Deep Space Nine upscale project [Archive]

Katie Boundary

4th April 2020, 22:36

My understanding is that DS9 has telecined 24p and interlaced video mixed together in frames (where VFX is used). I don't know how you can fix that correctly unless you use masks & treat each portion separately which would be totally impractical because the mask would have to change on a per frame basis.

Yes, but it's a mess. You can be in the middle of a telecined 24p sequence when all of a sudden some VFX work (which is i60) is composited into the scene for a few seconds. How do you handle that? Engage QTGMC for a few seconds, then turn it back off?

I solved that problem years ago:

mpeg2source("212.d2v")

Y=nnedi3(field=-2)

A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

Tweak other parameters to taste.

I applaud the person trying to tackle this project, but I fear he doesn't know enough to actually accomplish his goal. He talks about Handbrake and QTGMC at the same time, which makes me think he's just flopping about

He admitted to being an AVIsynth noob.

Unfortunately, we hobbyists don't have that option. So, the TNG 60i effects were redone to 23.976p. Our best option is then to deinterlace the 60i areas and choose specific frames, or more precisely, let TFM do it for us.

Or just bob to p60 and leave it at that.

wonkey_monkey

4th April 2020, 23:56

I solved that problem years ago:

You did not "solve the problem." You patched it up to your satisfaction, which is a very different thing.

Stereodude

5th April 2020, 00:34

You did not "solve the problem." You patched it up to your satisfaction, which is a very different thing.
I'm not even sure what that script's going to give you as the output. I'm not sure I'm curious enough to try it and see.

I mean I get the output for i60 input will be 60fps and shouldn't have combing, but beyond that I have no idea how the p24 content will be represented in the output.

I'm almost tempted to go get a DS9 DVD. Almost...

huhn

5th April 2020, 01:10

Stereodude

5th April 2020, 03:27

not sure what's so hard about this topic VFR encoding are made for like ever.

VFR is one way to get the correct frame rate for all parts the other is 120 HZ which is not an optimal choice for obvious reasons.

time didn't stop:
i never used this way to archive VFR but there are simply better people to answer how to do it but it's even an example for http://avisynth.nl/index.php/TIVTC

it can't be to hard to automatically deint the parts that are detected as hybrid in stead of treating them as PsF.
It's hard because both types of content are in the exact same frame. It's not some i60 and some telecined p24 content and it switches on a scene by scene or shot by shot basis.

They shot it on film, telecined the film to i60 and then composited into that i60 special effects.

huhn

5th April 2020, 08:50

sorry still don't get the issue or what's unusually about that.

i know that the mixed part will have some motion issues that's the creators intent. so motion adaptive deint should be the way to go for that part.

Katie Boundary

7th April 2020, 11:56

I'm not even sure what that script's going to give you as the output. I'm not sure I'm curious enough to try it and see.

I mean I get the output for i60 input will be 60fps and shouldn't have combing, but beyond that I have no idea how the p24 content will be represented in the output.

It looks a lot like it would if you ran it through QTGMC, except better because the fields are properly matched and then left the hell alone instead of bobbed and then run through a dozen rounds of blurring and sharpening.

bladerunner1982

7th April 2020, 12:48

I also had this upscale project in mind for the past three years. (taking long breaks in between :D)
From time to time I experimented a little bit with different scripts and
parameters using the first episode of season 6 (the first 20 minutes).
Here is the script I came so far using the NTSC DVDs:

myvideo = "ep1ntsc.d2v"
DGDecode_mpeg2source(myvideo, idct=5)

#deint = QTGMC().SelectEven() #using QTGMC as deinterlacer optional
deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

QTGMC(InputType=1) #using QTGMC on progressive frames again to clean up a little bit
#QTGMC( Preset="Slower", InputType=1 ) #optional

tweak(cont=1.3,sat=1.1)

ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

addgrainc()
trim(0,26000)+trim(49114,50000)
AssumeFPS(25)

If someone is interested I can provide a link to the encoded result via pm.
Maybe it´s not a pro solution but for me it was ok.
I´m always open for suggestions.

ryrynz

7th April 2020, 21:06

Stereodude

7th April 2020, 22:10

Stereodude

8th April 2020, 05:47

S06E01 is not a good sample IMHO. I just pulled it from the DVD to try bladerunner1982's script with my own twist. There are virtually no combed frames in the first 20 minutes that make it past the IVTC. The only real problem area in those 20 minutes is the wormhole opening at the end of the Intro. And, the script from bladerunner1982 does not handle it well. The default MI of 80 in TFM is way too high. It will pass visibly combed frames in that area. Cranking it down to 5 helps, but still doesn't get 100% of them. Basically you've got to take it to 1 which causes it to see lots small areas as needing deinterlacing in progressive sections, but the impact on the output seems negligible.

After playing around a bit I would suggest something like this:

source=MPEG2Source("VTS_02_1.d2v", idct=5, info=0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()

source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=false)

TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.3, display=false)

MCTemporalDenoise(settings="low", radius=3)

f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)
Then encode it with a SAR of 8:9

SaurusX

8th April 2020, 13:18

Sounds like the series would require TFM override files for every episode of the series. Any time theres a special effect the regular TFM approach would need to be examined to make sure it’s catching everything. If not, then add the frame sequence to the override file to be deinterlace. Not impossible, but very time consuming.

bladerunner1982

8th April 2020, 18:35

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?
So why not take enough computing power in the upscale process before encoding?
Ending up with a 1440x1080 video file, PAR 1:1

I only encoded 20 minutes because I only have an i5-7500, took me 2 hours to encode.
Tested mainly the parameters of SuperResXBR, trying to find the balance between smoothening and sharpening.

So what episode would be best for testing?

Stereodude

8th April 2020, 19:23

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?
I didn't say my preference was right for everyone.

So what episode would be best for testing?
I'm not sure. I haven't ripped them all. I've only looked at 2: S06E01 and S01E01. It might not be a bad episode for testing, but I didn't see it in the first ~20 minutes. It's only 88.29% film per the D2V. S01E01 is 97% film.

Maybe there aren't that many problem areas in the typical episode when using TFM. Much of the special effects are telecined film content. They're not all interlaced video. I do wish Tdecimate had some different parameters for controlling the blend vs. drop in the hybrid mode.

scharfis_brain

8th April 2020, 19:38

btw.: does anyone know wheter there is an NTSC DVD release of DS9/VOY with german audio?

Chir

9th April 2020, 02:02

Greetings --

I'm actually tackling this project. I am no expert on this, but I've developed workflow that produces decent results. I came here looking for solutions to my problem where it looks like the frame rate drops -- it gets all jerky in scenes. It's usually heavily CG scenes. You guys helped explain a lot, and I appreciate that. No I understand its the mix of video and film.

I just started to learn about working with video at this level, so please be patient. My current workflow is this:

1. Rip with dvddecryptor to get each episode into a vob file.
2. Open the VOB in DGIndex; enable forced film and save the D2V file.
3. Create an AVISynth script that consumes the D2V file with the following as my template:

LoadPlugin("D:\Video\Tools\DGMPGDec\DGDecode.dll")
SetFilterMTMode("QTGMC",2)
MPeg2Source("D:\DS9S1D1\VIDEO_TS\VTS_02_1 - 0xE0 - Video - MPEG-2 - 720x480 (NTSC) - 4~3.d2v")
ConvertToYV12()
AssumeTFF()
QTGMC(Preset="slower", FPSDivisor=2, Edithreads=4)
Prefetch(threads=6)

4. Pull the video out with ffmpeg:
ffmpeg.exe -i <full path to .avs> -map 0:v -c:v copy <destination.mkv>

6. At this point, I upscale the video with VideoEnhance AI. It spits out PNG files for all the frames in the video file (destination.mkv)

7. I then reassemble the PNGs into Video with FFMPG using the original framerate from the DVD
ffmpeg -r 23.976 -f image2 -s 1920x1080 -i <path to png files>\%06d.PNG -vcodec libx265 -preset medium -crf 28 -pix_fmt yuv420p EncodedOutput.mkv

8. And finally, I mux everything back together with the original audio from the disc, chapters, and subtitles.

Its far from perfect.... and I'm hoping to pick up some tips to streamline my project and get a better result. I'm going to look into the sample scripts above to try and get a better understanding of what they're doing. I have a feeling I'm in over my head and doing unnecessary steps -- but I'm willing (and trying!) to learn!

SaurusX

9th April 2020, 02:34

Using QTGMC to deinterlace the image essentially means you’ll be upscaling a 240p picture to 1080p or 4K or whatever. You need to inverse telecine as your main process and deinterlace only the scenes you need to.

manono

9th April 2020, 06:22

2. Open the VOB in DGIndex; enable forced film and save the D2V file.

Unless the source is 100% FILM (check the bottom of the D2V file), that is a very bad idea.
ConvertToYV12()
It's already YV12.

bladerunner1982

9th April 2020, 12:26

@Stereodude

I encoded your script combined with SuperResXbr (without the output_depth=10 parameter in f3kdb).
Looked good but the space scenes (starships) had motion judder.

@scharfis_brain
I don't think there is an ntsc version disc with german audio. I use the ntsc discs, speed them up after processing to 25fps at the end of my script, then I encode. Speed up the english track to 25fps as well, leave the german as it is.

@Chir
A.I. Gigapixel? You can feed the video directly in it using newest version. No need for frame based encoding.

There is now a folder at mega.nz which contains the testvideos generated by scripts in this thread.
Everyone is invited to have a look at it for comparison. PM me :)

If you want to contribute your testvideo send me a link. Please encode only first 20min of S01E01 or S06E01 without audio to 1440x1080 (I use x264,crf16,preset slow)

Stereodude

9th April 2020, 13:24

@Stereodude

...
Looked good but the space scenes (starships) had motion judder.
Can you give me an exact timecode for when you see judder? Pretty much all the special effects scenes appear to be telecined film when I skimmed through it.

bladerunner1982

9th April 2020, 16:59

Timecode 01:38 ....right after the look back scene...check PM...

Stereodude

9th April 2020, 19:56

Timecode 01:38 ....right after the look back scene...check PM...
It's a bit complicated. If you comment out the Tdecimate line in your script and use this TFM line instead:

source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=1, clip2=deint, display=true)

and then step through the section frame by frame in VD2. There's clearly a duplicated frame that should be removed by tdecimate.

In looking at this, I did discover that I misunderstood the meaning of this section of the TFM help. "...if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2." It doesn't replace only areas where combing was found it like I was thinking it did. It replaces anywhere that motion was found. In this case because of the motion this changes the output frames significantly (basically completely replaces them).

So, in this section the low MI value set in TFM causes TFM to pull some of the frames from the deinterlaced version which then causes areas of irregular motion in the final output.

Changing the TFM line to:
source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=true, flags=5)

fixes the problem in this area. Flags=5 will prevent TFM from looking for combing in any part of the source (except scene changes) that's encoded as 24p with a soft pulldown flag on the DVD. This still leaves areas that are encoded as video.

To address those I would suggest increasing the MI some. Some minor combing will get through, but it will cut down on unnecessary frame replacement with softer deinterlaced frames.

Short of making a full manual override file where you manually and visually inspect every frame with a MIC > 0 you have the trade off of letting minor combing through vs. unnecessary frame replacement. The MI value sets the threshold.

I'm prepping a sample to upload and maybe we'll get some better ideas from others.

Edit: At least for S06E01 bumping up the MI to 24, adding flags=5 to TFM, using an override file for TFM, and an override file for tdecimate seems to basically fix the episode. (Note, I didn't do a frame by frame examination of the whole show.)
TFM Override:
13329,13945 -
16383,16504 +
The first keeps the comet at the start of the "intro" from being seen as combed. The second fixes the combing in the wormhole at the end of the "intro".

Tdecimate Override:
79880,81234 v
This forces the end credits to be blended.

Other episodes will need the same areas fixed with overrides, but of course the frame numbers will be different.

Stereodude

9th April 2020, 20:15

Well the pro solution would be something similar to this that would be created with the help of people here. The project creator has far less knowledge of Avisynth filters. Wouldn't hurt to flick that across to him for starters. You could also maybe provide a, sample for us with this to evaluate and maybe improve upon?
Here's a sample. It's starts with a section that's video and then it's a section that alternates between film/video as the titles appear on screen.

https://mega.nz/file/1g9DBSpC

and what you'll need to get it:
64cZaOQ8mHnbvDRTthMf6M0RenYwdb3M0A7I_ufaQqc

mbcd

11th April 2020, 01:10

My try, looks good, but not perfect ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u

LWLibavVideoSource("VTS_02_1_sample.m2v")
TFM(mode=1)
TDecimate(mode=0,hybrid=1)
AssumeFps(23.976)

There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

mbcd

11th April 2020, 01:44

My try, looks good, but not perfect, no magic done ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u

There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

Look like field oder is changing, I get mostly duplicated bad frames on those ...

I dont think you will get a good result, they layed two different framerate together, title changes, but background is still (doubled frames) ...

bladerunner1982

11th April 2020, 13:49

@mbcd
Thanks for participation. That's a rather basic approach. But where's the upscaling part? :)

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole.
After some experimenting I came up to this:

LoadPlugin("d:\gordianknot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\LSMASHSource.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\TIVTC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\vinverse.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\Shader.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\AddGrainC.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 2)

myvideo = "ep1ntsc.d2v"

DGDecode_mpeg2source(myvideo, idct=5)

## choose deinterlacer ##
#deint = QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()
deint = tdeint()

## IVTC hybrid ntsc with override filles ##
tfm(d2v=myvideo, PP=6, MI=24, clip2=deint, flags=5,ovr="s06e01_tfm_ovr.txt")
TDecimate(hybrid = 1,ovr="s06e01_tdecimate_ovr.txt")

## clean up ##
#MCTemporalDenoise(settings="low", radius=3)
#f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false)
#or
#QTGMC(InputType=1) #using QTGMC on progressive frames again to clean up a little bit
#or
QTGMC( Preset="Slower", InputType=1 )

## color tweak ##
tweak(cont=1.3,sat=1.1)

## Upscale ##
ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

## grain only use if you didn't use f3kdb
addgrainc()

trim(0,15000)

## PAL Speedup for Europe
AssumeFPS(25)

Result is version 3 in this folder https://mega.nz/folder/A10lULqT

Stereodude

11th April 2020, 16:07

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole...
FWIW, I don't think there is any difference between 6 or 7 if you're using clip2.

I'm also on the fence about using mild deblocking in the MPEG2 source filter on the luma channel. For example, the comet in the show intro has significant blocking when decoded with no deblocking.

Stereodude

11th April 2020, 16:37

Result is version 3 in this folder https://mega.nz/folder/A10lULqT
You made some post processing decisions that I personally wouldn't have made, but that doesn't mean they're wrong. To me it mostly looks like there is a static grain pattern sitting on top of the image much of time that I don't care for.

However, I don't think the MPEG-2 source could be handled much better getting it to 480p24 in a basically automated way. FWIW, you could crop about 4 pixels off the right side at the source filter.

DGDecode_mpeg2source(myvideo, idct=5).crop(0,0,-4,-0)

bladerunner1982

11th April 2020, 20:57

TFM options:
"if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2."

PP6 = motion-adaptive cubic interpolation deinterlacing... gives to me a sharper picture than PP7 (maybe it´s just my imagination)

And therefore I tend to QTGMC for post processing...MCTemporalDenoise and f3kdb seem to clean to much so there is less detail left for upscaling.

I compared both solutions eye by eye...but tastes differ...

Anyone with a suggestion for a better upscaler?

Stereodude

11th April 2020, 22:53

TFM options:
"if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2."

PP6 = motion-adaptive cubic interpolation deinterlacing... gives to me a sharper picture than PP7 (maybe it´s just my imagination)
It's your imagination. The deinterlacing routine isn't used. clip2 is used. The same detection algorithms are used for either.

Anyone with a suggestion for a better upscaler?
You could try waifu2x

https://forum.videohelp.com/threads/394851-Script-to-scale-videos-with-waifu2x-(AviSynth)

(I haven't tried this)

Chir

12th April 2020, 01:05

Stereodude

12th April 2020, 04:43

So the problem that we have is that some frames need to be reverse telecined and some need to be deinterlaced, but not both. Would that be a correct statement of the problem?

To deal with this, we'd need to create an OVR file (two actually), which if I understand it right, allows us to choose which process to apply to the frame.

If that is a correct statement -- are there any tools to help create the OVR files? Or is this something that you have to step through frame by frame to determine the type and manually create the OVR file?
bladerunner1982 and myself have basically hammered out a pretty decent template/theory of handling the episodes.

You want to IVTC all the parts of the original DVD that are encoded as film with a soft pulldown. You still want to IVTC as much of the rest of the video portions as possible. You only want to deinterlace any frames that have residual combing or are interlaced video.

TFM checks for residual combing in a frame and comes up with a number based on how much combing it measures. You set the threshold for how high number that needs to be before it will deinterlace the frame.

You only need an override file when that threshold doesn't work for a section. In S06E01 I found 3 problem areas. 1) At the start of the show "intro", the comet generates a high MIC number despite not being combed. 2) The wormhole opening at the end of the "intro" has combing but doesn't generate a high MIC number. 3) The credits appear to be pure interlaced video.

So, you make an override file with the frame ranges and what it should do for those ranges.13329,13945 -
16383,16504 +
79880,81234 p
I expect these issues will happen in every episode. Of course the frame numbers will vary for each episode.

The second issue is that tdecimate doesn't always make the correct decision on whether to drop a frame restoring 24p or to blend the 30p video section to 24p. The most flagrant area it got wrong was the credits, so I made an override to force them to be treated as video (blended).79880,81234 vI expect this issue will also happen in every episode and the frame numbers will need to be adjusted. I'm also undecided if the wormhole opening at the end of the "intro" should be blended or treated as film, but it's something to consider.

So you want to do something like this:
source=MPEG2Source("VTS_02_1.d2v", idct=5, moderate_h=40, moderate_v=40, cpu2="xxoooo").crop(0,0,-4,-0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0).SelectEven()

#for testing
#source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=24, clip2=deint, display=true, flags=5)
source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=24, clip2=deint, display=false, flags=5, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")

#for testing
#TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=true)
TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=false, ovr="D:\DS9S6D1\E01\td_ovr.txt")

You can visually inspect what TFM is doing and the see the metrics for each frame by uncommenting the first TFM line and commenting out the 2nd. You can to the same for tdecimate by uncommenting the first tdecimate line and commenting out the 2nd. You check TFM and tdecimate separately.

I do not think you need to go frame by frame. I would think a quick skim of the video sections of the episode will be sufficient. The only portions that are video are fades to and from black, the intro, where text is overlaid on the screen, and the credits. The intro and credits should have the same behavior from episode to episode, just update the frame numbers. Fades in and out of black I would just leave be, which really means the only parts you have to check are the text overlays for a few minutes after the intro where they give the guest actors, director, etc.

SaurusX

12th April 2020, 18:54

The interlaced fade-ins and fade outs can be handled with the smartfade() routine right after the IVTC. I think I pulled it out of the AnimeIVTC script. Might as well get rid of those if you can.

#based on smartfade v0.2 by martino - Aimed at removing interlaced fades in anime. Uses luma difference between two fields as activation threshold.
#mod by thetoof : removed degrainmedian post-processing,
# changed how the fields are blended together (average of 2 nnedi interpolations + contra-sharpening to retain more detail) and
# added cache parameter to forward the setting of animeivtc to this function
# renamed global parameter to something less general than "threshold"

function smartfademod(clip c, float "sfthr", bool "sfshow", int "cache") {

global sfthr = default ( sfthr, 0.4 ) # Threshold for fade detection.
show = default ( sfshow, false ) # Displays luma difference between fields without processing anything.
cache = default ( cache, 10 ) # Cache the frames to help seeking and other functions requesting many frames

show ? scriptclip(c, "subtitle(string(abs( separatefields().selectodd().averageluma() \
- separatefields().selecteven().averageluma())))") : \
scriptclip(c, "sep = separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > sfthr ? interleave(sep.selecteven(),sep.selectodd()).weave().mcdaa3() : last")

cache<0 ? last : last.RequestLinear(8, cache, 5, false, false)

return last

}

Chir

12th April 2020, 23:00

I'm trying to read up and understand so I don't ask too many questions, but I'm a little lost.

I understand what we're trying to do, but I'm struggling to put it all together. I grabbed Season 6, Episode 1 so I could use your examples to learn, but it's not clicking for me.

I started with DVD Decryptor and pulled the epsiode off into a VOB file. I opened the VOB file with DGINDEX and set it to Honor Pulldown flags, then saved the D2V file. I fired up avspmod and pasted the script in that you provided.

I commented things out to do "testing":

source=MPEG2Source("VTS_02_1.d2v", idct=5, moderate_h=40, moderate_v=40, cpu2="xxoooo").crop(0,0,-4,-0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0).SelectEven()

#for testing
source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=24, clip2=deint, display=true, flags=5)
#source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=24, clip2=deint, display=false, flags=5, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")

#for testing
TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=true)
#TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=false, ovr="D:\DS9S6D1\E01\td_ovr.txt")

When I display the first frame, I the frame, the type of match, and (D2V). I believe this mean that no filters are applied and its just displaying the D2V -- is that correct?

When I get to frame 832 (Kira standing at a control panel, but the start of a Scene change to a CG scene) The (D2V) tag goesaway and it has a MIC=9. At 838, it switches to a Full CG Scnene and goes from Clean Frame to Combed Frame. This continues for frame 1017. I'm assuming this is a frame that TFM detected that it needs to process -- is that correct?

As I progress into the video, I see a few mic values pop up -- but only for a frame or two. I think this is ok, as it appears that a changing scene or lots of movement causes this. However, at 7102 (black screen) the (D2V) indicator goes away again -- is this TFM detecing a frame to process again? It still says CLEAN FRAME with a rather low mic value until frame 7352 (as the comet leaves the screen to the right). You mentioned this part of the video in your comments above, and its where I see problems with my previous attempts -- there is a HUGE jump right here). I'm expecting TFM is picking this up, so I don't need to do anything in the overrride file... is that correct>

You provided a sample ovr for TFM. If I jump to that frame, I'm expecting to see something in the stats change, or something on the screen, that idicates what I'd be looking for to include in the ovr file. At 13329, I see Odo talking to Quark. This appears just after the opening credits are finished (after the cg intro). I don't see anything -- either in the image itself or the TFM debug output that would have clued me in to add this to my ovr file. Any hints at what I should be looking for in these scenes to add them to the override?

I have a hunch that I'm not working with exactly the same media and your examples might be a little off from mine. avspmod claims that I have 81773 frames before decimating.

Chir

12th April 2020, 23:41

oh, my. I feel like a complete idiot. I had my D2V file pointing to the wrong VOB, so I was actually looking at a different episode...

Time for me to take a break!

Stereodude

13th April 2020, 04:53

When I display the first frame, I the frame, the type of match, and (D2V). I believe this mean that no filters are applied and its just displaying the D2V -- is that correct?
The D2V means TFM is applying an IVTC and assuming the output is progressive (not looking for combing) per the information/flagging in the D2V file. flags=5 forces this behavior (which is desirable here).

When I get to frame 832 (Kira standing at a control panel, but the start of a Scene change to a CG scene) The (D2V) tag goesaway and it has a MIC=9. At 838, it switches to a Full CG Scnene and goes from Clean Frame to Combed Frame. This continues for frame 1017. I'm assuming this is a frame that TFM detected that it needs to process -- is that correct?
When D2V disappears you're in a video section of episode and TFM will manually look for a pattern to IVTC. It will also look for combing (possible interlacing) after applying the IVTC. The MIC number is a measure of how much combing it finds.

bladerunner1982

13th April 2020, 14:45

@Chir
Don't worry, I made that error myself...Frame 13329 corresponds to the first VOB file of S06E01 (VTS_02_1.VOB) without any TFM or Tdecimate applied..where the comet is starting. Maybe we should set that file as a reference point

Stereodude

13th April 2020, 14:51

SaurusX

13th April 2020, 16:15

Unless I'm misunderstanding something I don't think works very well. In order to get it to work well on the fades it needs a sfthr value that is low enough it affects lots of non-fading frames.

It works well enough as-is and makes a noticeable improvement.

Stereodude

13th April 2020, 16:35

It works well enough as-is and makes a noticeable improvement.
With the default of 0.4?

I'm going to have to disagree with your conclusion. In S06E01 of DS9 the first several fades in and out of black it wouldn't get a single frame. The first one where it would find any affected frames it only gets about 30% of the frames you'd want it to get. And, there are plenty of static scenes where it detects ~.25.

So you have static scenes with no interlaced fading going on with a higher detected value than actual interlaced fades. It doesn't work for this type of content.

bladerunner1982

13th April 2020, 17:55

How about this dirty tweak?

function smartfademod_ds9(clip c, float "sfthr", bool "sfshow", int "cache") {

global sfthr = default ( sfthr, 0.4 ) # Threshold for fade detection.
show = default ( sfshow, false ) # Displays luma difference between fields without processing anything.
cache = default ( cache, 10 ) # Cache the frames to help seeking and other functions requesting many frames

show ? scriptclip(c, "subtitle(string(LumaDifference(separatefields().selectodd().crop(150,180,-150,-16),separatefields().selecteven().crop(150,180,-150,-16))))") : \
scriptclip(c, "sep = separatefields() \
diff = LumaDifference(sep.selectodd().crop(150,180,-150,-16),sep.selecteven().crop(150,180,-150,-16)) \
abs(diff) > sfthr ? interleave(sep.selecteven(),sep.selectodd()).weave().mcdaa3() : last")

cache<0 ? last : last.RequestLinear(8, cache, 5, false, false)

return last

}

It looks only in the parts of the half-frame, in which the credits are and calculates the absolute Luma difference in it. Then use a sfthr of maybe 1.2...

Stereodude

13th April 2020, 18:25

Chir

14th April 2020, 01:38

Stereodude

14th April 2020, 02:36

@Stereodude

A couple questions, if you don't mind. I'm still struggling how I'm supposed to identify the frames that I need to add to the OVR files, and I think I might have identified them. I just want to run it by you.

You gave me the following script -- one to test, and one for production.

Everything looks prety good with the display on, and no override files -- so I flip over to the production one and things start to go south. I end up seeing, during transitions, weird effects -- like portions of the images are blacked out or leftover from the previous frames. It looks like this happens during really quick, brief scene changes. Are these the frames that should get added to the OVR file?
Can you give me a frame number where you see this? Is it repeatable?

Are you sure your deinterlaced video from QTGMC is working correctly? You can check this by putting deint at the bottom of your script so that it returns that instead of the output from tdecimate.

Also, for almost the entire episode if both tdecimate lines are commented out you should have one duplicated frame out of every 5. This is normal and expected. tdecimate will remove this duplicated frame and that's how you get 23.976fps content from it.

You shouldn't have to add much of anything to the ovr file for S06E01. I've subsequently looked at the intro more closely and the metrics and have a somewhat different idea on how to treat it that addresses all the fades and largely doesn't rely on the MI to kill off combed frames (since it missed some and flagged some that it shouldn't have).

I'd use this override:;intro higher MI override
13331,16497 i 45

;comet override
13331,13947 -

;14243 DS9 title first frame where visible - for alignment

;fade in intro
14717,14786 +----

;fade in intro
15097,15171 +----

;fade in intro
15472,15541 +----

;fade in intro
15821,15872 +-++-

;fade in intro
16145,16186 ----+

;wormhole open override
16382,164502 +

;override for title fade in
17069 -

;override for title fade in
18203 -

;override for title fade in
18478 -

;override for title fade in
18982 -

;override for title fade in
19843 -

;credits override
79880,81234 p
You can use a simple spreadsheet to adjust the frame numbers in the override for the intro for another episode by using the frame number where the DS9 title is first visible to calculate an offset.

I was also wondering why PP=1 was in the test line and PP=7 was in the production line. Was that just a mistype?
PP=1 calculates the MIC, but doesn't replace any part of the frame. PP=7 uses some parts of the deinterlaced video. It's not a typo. It's intentional so you can see if the source actual has interlacing that should be replaced or not. If you use 7 (or 6) in that line part of the frame will be replaced and you won't be able to see if it should have been or not.

edit: You should not have to change cthresh. Something else much be wrong and that's not the right fix.

Chir

14th April 2020, 03:52

You were spot on with my QTGMC not working. It was returning a mostly black screen. Once I fixed that, the issue went away.

I'm off to play around with your ovr file and see if I can pick up on your method!

I so appreciate all the help and patience!!!

Stereodude

14th April 2020, 04:36

Oh, you'll want to change the MI in your TFM call to: MI=33. Otherwise there are two frames, 898 and 900, that need to be added to the override file with a -.

Stereodude

14th April 2020, 14:20

I've been trying to apply the same processing method to S06E02 and S06E03. S06E02 is largely no issue. S06E03 is a bit more complicated. In it most of the space VFX shots will need to forced (as not combed) in order to avoid false positives and the MIC calculation of quite a few of the video segment after the intro throw high MIC values and will have to be forced (as not combed).

Also, I found a mistake or two in the "intro" portion of the override file I posted last night for S06E01 that I've fixed.

bladerunner1982

15th April 2020, 10:27

I found them too :)

;wormhole open override
16382,164502 +

better:
;wormhole open override
16382,16505 +

Right? Anything else?
Is this your way of testing if people are still there? :D

bladerunner1982

15th April 2020, 11:27

StainlessS

15th April 2020, 11:46

I always use DvdShrink whether episodes or not.
Also, set up DvdShrink preferences, eg set output DVD9 [avoids need to switch to "No Compression" on each title], also select required Subtitles lang etc.
DVDShrink usage helps avoid some minor user mistakes and ReDo's.

EDIT: Some disks may have eg 27 differenct versions of a title, all different length with only a single one that plays properley.
To select correct one [in DS], see how long title is [duration] in a DVD player, and select the title in DS that corresponds to same length,
can be as simple as that [often is as simple as that] when using DS.

Stereodude

15th April 2020, 13:19

Another thing:

Maybe its better to reauthor the DVDs first to get vob files for every episode. Makes dubbing the audio later a little bit easier...

I used DVDshrink https://www.videohelp.com/software/DVD-Shrink

Steps:
1. Load Disc
2. Re-author
3. Drag titles from right to left
4. Backup to a new folder

Now you have vob files for every episode split correctly.
Then making ovr files for every episode...
If you assign seasons to diffrent users it would be much time saving ;)
I just use DVD Decrypter in IFO mode to rip each episode to a separate folder on my HDD. I use the stream processing to demux the video and audio streams, but I leave the subtitles in the .vob so I can use SubRip to convert the subtitles to a .srt (it wants a .vob).

Stereodude

15th April 2020, 17:09

Right? Anything else?
Is this your way of testing if people are still there? :D
It depends on the episode, but you also have to watch that the titles fading in and out after the intro don't mess up the pattern detection of the underlying video during the fade. It mainly happens when the scene is basically static, or has very low motion. If it does, you get to add more things to the TFM override file.

I didn't notice this until several episodes later. I went back and looked at S06E01 closer specifically for this and found 3 areas where this happens.

;override for title fade in
17069 -

;override for title fade in
17757 -

;override for title fade in
18203 -

;override for title fade in
18478 -

;pattern override for title fade in
18972,19055 pcccp

;override for title fade in
18977 -

;override for title fade in
18982 -

;pattern override for title fade in
19839,19922 pcccp

;pattern override for title fade in
19964,20047 pcccp

A possible alternative way handle the video and not have this issue occur (that I specifically haven't tested on DS9 yet, but have on other things) is to duplicate the video, mask out the area where the titles are fading in, stack the two vertically, and then use the y0 and y1 options for TFM to only look at the portion of the clip with the masking for pattern detection. That way the titles can't affect pattern or duplicate frame detection.

something like: (not currently working as expected)
#deint may be need to be stacked, like deint=StackVertical(deint, deint). Not sure yet because I hit a problem before getting to that point.
h = source.Height()

moverlay = ImageSource("overlay.png", end=1000).ShowRed ("YV12")
mmask = ImageSource("mask.png", end=1000).ShowRed ("YV12")

c1 = mt_merge(inputvid, moverlay, mmask, luma=true)

comp=StackVertical(c1, inputvid).TFM(d2v="VTS_02_1.d2v", y0=(h), y1=(h*2-1), PP=1, flags=5, MI=24, clip2=deint, display=true, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")
#comp=StackVertical(c1, inputvid).TFM(d2v="VTS_02_1.d2v", y0=(h), y1=(h*2-1), PP=7, flags=5, MI=24, clip2=deint, display=false, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")

full=comp.crop(0, h, -0, -0)
masked=comp.crop(0, 0, -0, -h)

#masked.tdecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.3, display=true, ovr="D:\DS9S6D1\E01\td_ovr.txt", clip2=full)
#masked.tdecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.3, display=false, ovr="D:\DS9S6D1\E01\td_ovr.txt", clip2=full)
Edit: It doesn't work here. When fed the stacked clip and excluding the bottom clip with y0 and y1 it does not work. It seems y0 and y1 do not work as expected. The bottom clip still is affecting the matches which confuses me because I've used this method before on videos and it has worked (AFAIK, though I didn't look at it this closely).

Edit2: I made a quasi bug report for this apparent issue with TFM here (https://forum.doom9.org/showthread.php?p=1907771#post1907771). Hopefully Pinterf can take a look.

Chir

19th April 2020, 05:23

I think I've learned enough to start going through the episodes and creating the TFM OVR files. I've gone all the way back to the beginning and started from season 1.
If anyone wants to contribute or use the files, I've started a project on Github.

https://github.com/Chireven/DeepSpaceNine

If there is a better place for these, please let me know and I'll share what I'll remove the project and move the work over there.

Stereodude

20th April 2020, 04:37

Katie Boundary

21st April 2020, 10:01

So the problem that we have is that some frames need to be reverse telecined and some need to be deinterlaced, but not both. Would that be a correct statement of the problem?

Not quite. See below...

You want to IVTC all the parts of the original DVD that are encoded as film with a soft pulldown. You still want to IVTC as much of the rest of the video portions as possible. You only want to deinterlace any frames that have residual combing or are interlaced video.

Inverse telecine involves converting the footage back to 24 fps, which is not only unnecessary, but also makes this project about 10x as complicated as it needs to be because suddenly you're in the world of mixed-framerate video which is incompatible with the entire VFW ecosystem.

You will, however, need to do a mix of field-matching and bob-deinterlacing. I provided the script for that earlier.

To deal with this, we'd need to create an OVR file (two actually),

Whoa whoa whoa, no. There is no need to drag OVR files into this. You're making it way more complicated than it needs to be.

I think I figured this out. Going over the docs for TFM, I stumbled across the cthresh parameter. It defaults to 9, and suggested values are from 8-12 -- I had to bump it up to 25.

Oh god no, that is the exact opposite of what you should be doing. Cthresh should be 1-2 if you want to kill all visible combing. Whatever problem you're trying to solve, you're creating an even worse one

You were spot on with my QTGMC not working. It was returning a mostly black screen. Once I fixed that, the issue went away.

QTGMC shouldn't even be used in this sort of situation to begin with.

It depends on the episode, but you also have to watch that the titles fading in and out after the intro don't mess up the pattern detection of the underlying video during the fade.

Or, you could use a method that doesn't require any pattern-detection at all.

mpeg2source("212.d2v")

Y=nnedi3(field=-2)

A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

Stereodude

21st April 2020, 13:26

Stuff it Katie... Your method returns soft looking video that has nasty cadence judder for nearly all of it. The method Chir is using has no cadence judder retains the original sharpness of the source over as many frames as possible.

wonkey_monkey

21st April 2020, 20:07

You will, however, need to do a mix of field-matching and bob-deinterlacing.

[...]

QTGMC shouldn't even be used in this sort of situation to begin with.

Make your mind up.

bladerunner1982

22nd April 2020, 21:22

Stereodude

22nd April 2020, 23:58

Well, I did a full encode of S01E03 with my script from page 2 (Tdeint as deinterlacer) and Stereodude's script (with QTGMC as deinterlacer). Both with and without ovr Files from Chir.
IMHO I would strongly recommend NOT using QTGMC for deinterlacing, because it kills too much details that are later missing in the upscale process.
I sit here 7 feet away in front of my well tuned 50" plasma TV connected to well tuned htpc and I really can see a difference between the two versions.
With QTGMC as deinterlacer (not as a cleaner as I use it after IVTC !) the picture looks to me like an upscale only, but when using Tdeint instead I have a pseudo HD feeling.
And adding grain at the end of the script is necessary in my opinion, because it forces the x264 encoder (btw. crf 16 !) to encode those details in the upscaled 1080p image.
And it really does looking at the file sizes of both encodes (3.7GB vs. 2.9GB)
I don't really understand your conclusion since virtually none of the episode is deinterlaced. Almost all of the episode is handled strictly by the IVTC. With no ovr files there are only 228 frames that will have any deinterlacing used in them out of 81758 frames. Additionally, QTGMC should have minimal noise reduction with the options I posted.

If you're seeing visible differences something else is going on.

I think the ovr file chir put on github doesn't handle the intro correct for S01E03. Frankly, the intro looks so bad, I'd find another version of it from another episode, perhaps in another season, and use that instead (if it's the same number of frames / same content).

And I must say that not using ovr files and just standard settings

deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

isn`t that bad either.
I mean for those short sequences (comet,wormhole,exlosions,teleporter) I personally can live with not using ovr files.
I never had the feeling that without them makes some scenes unwatchable.

So my question is: Is it really worth the effort to crawl through every episode only to have the best IVTC version for those short moments, which are only 1-2% of the episode?

Don't take me wrong, I really appreciate the work and time Chir and Stereodude are putting into these files, but is it really worthwhile?
If you're going to go through the effort to encode all the episodes don't you want to get the most out of it? Further, if chir is doing all the work for you with the .ovr why not use them?

Boulder

23rd April 2020, 05:29

Additionally, QTGMC should have minimal noise reduction with the options I posted.

I think the tr2 parameter is what causes the denoising. I would try setting tr2=0, it will then bypass the MDegrain phase at the end.

wonkey_monkey

23rd April 2020, 12:18

Boulder

23rd April 2020, 12:25

Has anyone had a look at the PAL DVDs? I think they used something called something like DEFT or DFET to convert the film sequences correctly and to do their best with the video sequences.

I think I've got seasons 1-3 somewhere. Might see if I can dig them out.

That's what was done with Star Trek TNG so it's a possibility that the same approach was used with DS9 a well. SRestore works against that with good enough results.

StainlessS

23rd April 2020, 12:27

Think DEFT, no idea what it is though. [Have looked for it before and never found any adequate explanation]

DFET variation of Field Effect Transistor, dont know what the D stands for. [EDIT: guessing that D may stand for Dual]
Other variations, JFET [Might be Junction, not sure], and MOSFET [Metal Oxide Semiconductor Field Effect Transistor].
CMOS that you've probably heard of, Complimentry Metal Oxide Semiconductor. [Low power alternative to TTL Transistor Transistor Logic]
DDL, very old pre TTL Diode Diode Logic.
FET, sort of like a transistor, but behaves more like a vacuum tube or valve [I think] very high impedance & sensitive to static electricity can blow them just by touching them.

Boulder

23rd April 2020, 12:41

Think DEFT, no idea what it is though. [DFET variation of Field Effect Transistor, dont know what the D stands for].

It's a marketing name. Standards conversion by Snell & Wilcox, apparently a bit less worse than simple field blending things we've seen.

StainlessS

23rd April 2020, 12:46

Arh, that explains why I never found it, probably saw that it was company specific and thought it not what I was looking for, thanks.

scharfis_brain

23rd April 2020, 21:08

Deft stands for
Digital
Electronic
Film
Transfer

Essentially it is a speedup of 59.94i to 62.5i
then doing 3:2 pulldown to 25p
and if that fails fieldblend conversion from 62.5i to 50i

They however completely borked the vertical field alignment during the 3:2 pulldown scenes.

TNG PAL DVD were undergone this procedure.

Essentially DEFT is a more clever way to do NTSC to PAL converssions that have a lot of 23.976p scenes

Chir

24th April 2020, 02:40

I don't think anyone has attempted to share their work in this way, so that's as good of a place as any.

FWIW, I don't think your handling of the credits in S1E01 is correct. They look to be progressive to me, so the override should be p, not -.

Are you also planning to include a tdecimate override file for each episode (usually needed for the credits)?

Yeah; I was completely wrong with S01E01 (and 03). I've since changed it, but I haven't pushed the changes to the project yet. I'll do that in a few. The wife and I have been battling that nasty virus going around and I've not had much time lately -- sorry if anyone wasted time with those...

In regards to the OVR files for TDECIMATE, I'm not entirely sure where they should be used. I kind of have a handle on the TFM ovr files, but I know I don't have it perfect yet.

Is there a more logical way to know which frames to look at for either of the OVR files? I've been doing it by just watching the video and looking for flaws.

I also plan to move on to Voyager after this, so hopefully the foundation that gets built here translates well to that (which I expect it will). I really don't mind doing the work to create them as long as people don't mind asking a million questions and helping me figure it out!

Stereodude

24th April 2020, 03:30

The credits at the end are 30p, however they have low enough metrics that tdecimate will drop a frame (treating it as film with a duplicated frame) instead of blending them to 24p. So, the stars behind the text end up moving irregularly. The override forces it to blend them to 24p instead. I didn't find other areas of episodes in my limited testing that were 30p that were incorrectly handled requiring and override. The wormhole opening and closing in the intro might be, but it's pretty much a toss up.

I would expect Voyager is similar, but I haven't actually examined an episode to see.

Chir

24th April 2020, 05:13

Oh - I get it now.

I'll move the credits from the TFM overrides to the TDecimate and mark them as P...

Stereodude

24th April 2020, 05:52

Oh - I get it now.

I'll move the credits from the TFM overrides to the TDecimate and mark them as P...
You need the credits in both. You need them marked as p in the TFM override and marked as v in the td override.

Katie Boundary

26th April 2020, 19:38

Stuff it Katie... Your method returns soft looking video

Wrong. It returns the frames exactly as they appear on the DVD, with no blurring, whenever field matches can be found.

that has nasty cadence judder for nearly all of it. The method Chir is using has no cadence judder

I believe what you mean to say is that my method allows conversion to VFR to be done as a second step, rather than making everything 10x more complicated than it needs to be by trying to deinterlace while in a VFR workflow.

Stereodude

26th April 2020, 23:33

That's very rude. You should be nice to the ladies.
Katie might have XX chromosomes, but she's no lady. Katie reaps what she sows.

wonkey_monkey

27th April 2020, 13:18

I just tried Katie's script on the sample clip provided. No idea what the aim actually was, but I get a 60fps output with up to four repeating frames at some points, plus a lot of misordered frames.

manono

27th April 2020, 19:57

Please stop with the gratuitous insults, Stereodude. If you wish to rebut what she says, then maybe do something like wonkey_monkey and try one of her scripts.

huhn

29th April 2020, 06:46

so i looked at the sample and i couldn't find a single part that is interlaced.

it's just 3:2 telecine with PsF credits.

i mean ok the the credits are sometimes only in 1 field but that doesn't change that i couldn't find a single part where the credits fade with 60 FPS not 30.

so what do i miss here?

Stereodude

29th April 2020, 13:16

I guess that all depends on how you're defining "interlaced". You can't simply IVTC the thing to p24 and call it a day. There are i60 post telecine fades (in/out of black and into other scenes) on top of the telecined content and the text titles are faded in and out as i60 on top of the telecined underlying image.

huhn

29th April 2020, 21:18

not here to say this makes it trivial but this is stuff that's been dealt with all the time it's now similar to broadcast where sometime text with 30 FPS is shown.

Katie Boundary

5th May 2020, 06:25

Katie reaps what she sows.

Oh really?:rolleyes:

What I sow:

my method allows conversion to VFR to be done as a second step, rather than making everything 10x more complicated than it needs to be by trying to deinterlace while in a VFR workflow

What I reap:

Stuff it Katie
Katie might have XX chromosomes

Hmmm, those things don't seem very similar.

By the way, Chir, this is exactly the sort of hysterical poop-flinging that I told you about in that PM I sent you. Now you have proof :)

ryrynz

5th May 2020, 12:15

Oh really?

up to four repeating frames at some points, plus a lot of misordered frames.

TBH that result speaks for itself, just drop it.

hello_hello

5th May 2020, 13:21

You can fix the out of order frames when using the Katie method on the sample, by swapping the field arguments used by TFM, but her method is quite bizarre. I'd be keen to learn how she decimates it for a VFR encode.

Y=nnedi3(field=-2)
A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

When I tried her method in an older thread it resulted in duplicate frames that probably didn't need to be duplicates, but whatever makes her happy. She's been told why it isn't as clever as she thinks it is in several threads.
I think Yadif is a new addition to the workflow though. I'm not sure I understand the benefit of telling TFM to replace the combed pixels using a clip de-interlaced by Yadif, which is replacing the combed pixels by taking them from a clip de-interlaced with nnedi3. Maybe I'm missing the obvious, but why wouldn't you use the "Y" clip for TFM's Clip2?

Y=nnedi3(field=-2)
A=Y.selecteven()
B=Y.selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

zapp7

8th May 2020, 16:16

Hi all, thanks so much for developing this workflow. I've started working on producing override files and have finished S01E01-02 (The Pilot). The output definitely looks better than my old workflow, which was rip with MakeMKV and then detelecine and deinterlace with Handbrake.

One thing I'm not 100% sure about is when to use the p, c overrides versus the + for the TFM ovr file. I'm guessing the p and c ones should be used to correct combing only when the frame fits in with an obvious 3 progressive-2 interlaced pattern. Is that correct?

Another question I have is what is the best way to save this output at the highest quality for input into the Topaz Video Enhance AI software? Right now I'm using ffmpeg with the command:

ffmpeg -i input.avs -aspect 640:480 -c:v copy output.mkv

I tried to fix the aspect ratio because it looked off. From what I understand, this is a lossless file? The output for one episode is about 31GB, which is fine with me as I have a lot of hard drive space.

(Note to @Chir: I'm the one that contacted you on GitHub.)

SaurusX

9th May 2020, 13:57

ffmpeg -i input.avs -aspect 640:480 -c:v copy output.mkv

I tried to fix the aspect ratio because it looked off.

Decreasing the resolution before upscaling seems like a wrong move IMO. You want to use all the original details you can. You should use Topaz to expand everything so that your minimum dimensions are met and then downsize after that.

zapp7

9th May 2020, 19:17

Decreasing the resolution before upscaling seems like a wrong move IMO. You want to use all the original details you can. You should use Topaz to expand everything so that your minimum dimensions are met and then downsize after that.

Yeah, I think I misunderstood what I needed to do. I figured out that I need to encode with the original NTSC sample aspect ratio (SAR) of 8:9. I'm now using:

ffmpeg -i input.avs -c:v libx264 -preset ultrafast -crf 0 -vf scale=-2:480,setsar=8/9 output.mkv

The original res is 720x480, but I noticed there are black strips on the sides of the image so I trim a few pixels off of the left and right sides in avisynth. The -2 accounts for the slight variance in the x dimension from episode to episode (i.e. 710 vs 706, etc.) depending on how much I trim it.

Chir

12th May 2020, 02:19

Yeah, I think I misunderstood what I needed to do. I figured out that I need to encode with the original NTSC sample aspect ratio (SAR) of 8:9. I'm now using:

ffmpeg -i input.avs -c:v libx264 -preset ultrafast -crf 0 -vf scale=-2:480,setsar=8/9 output.mkv

The original res is 720x480, but I noticed there are black strips on the sides of the image so I trim a few pixels off of the left and right sides in avisynth. The -2 accounts for the slight variance in the x dimension from episode to episode (i.e. 710 vs 706, etc.) depending on how much I trim it.

I think this would produce a more lossless result:
ffmpeg -i <your avs file> -map 0:v -c:v copy <your output file>

If you are happy with your results, feel free share your results with the github project that I started. Anyone can contribute!

https://github.com/Chireven/DeepSpaceNine

zapp7

12th May 2020, 02:57

I think this would produce a more lossless result:
ffmpeg -i <your avs file> -map 0:v -c:v copy <your output file>

Do you correct for the SAR at any point later in the process? I ask because this command appears to produce a horizontally stretched image. It's not super obvious but it's evident if you compare your output it to the original DVD footage. I used the codec because it's my understanding that SAR can't be corrected using the "copy" command. I read that CF 0 H.264 is lossless so I was hoping that would be sufficient.

I have about 10 episodes "complete", and I put that in quotes because I feel as though I am still learning the whole process and making refinements. There are undoubtedly errors in what I have. Should I share anyway? What do you think?

JoelHruska

12th May 2020, 03:17

Hey all,

Just thought I'd post here and introduce myself. I'm Joel Hruska, from ExtremeTech, and I've been working to upscale Deep Space 9.

Some of the workflows I'm currently evaluating are below: When I refer to "my" QTGMC filter, it's nothing exotic -- just a simple repair script, lightly tweaked from the AviSynth Wiki. This isn't all of them -- I'm not in front of my PC at the moment -- so this is just what I can recall from memory.

Handbrake Source -- DaVinci Resolve Studio (framerate conversion to 119.88 fps using dummy frames) -- StaxRip (AviSynth to apply my own QTGMC filter ) -- Topaz VEAI upscale.

Handbrake Source -- DaVinci Resolve Studio (framerate conversion to 119.88 fps using interpolated frames) -- StaxRip (AviSynth to apply QTGMC and decimate the frame rate back to 23.976) -- Topaz VEAI upscale.
)
Handbrake Source -- StaxRip (AviSynth to apply a Doom 9 Deinterlacing filter suggested on the first page + my own QTGMC repair filter + daa3mod) -- Upscale 4x in TVEAI .

Handbrake Source -- StaxRip (AviSynth to apply my own QTGMC repair filter) -- Topaz VEAI 4x upscaling -- DaVinci Resolve Studio ingestion and 119.88 fps conversion -- 23.976 fps decimation in a second run through AviSynth.

Handbrake Source -- StaxRip (AviSynth to apply my own QTGMC repair filter) -- DaVinci Resolve Studio Ingestion and 119.88 fps conversion -- AviSynth decimation back to 23.976 fps -- Topaz VEAI upscaling.

Note that all of the above assume that I'm using a CFR MKV or M4V file created in Handbrake. VBR files have their own dependencies and must be processed in a different order. The five workflows above are more representative than static -- while I'm actively experimenting with all of them, I'm also trying some ideas that aren't listed above.

I do not currently have an option to use MakeMKV. StaxRip will not / cannot match A/V if handed a VFR file -- at least not that I can figure out. I would love to solve this problem by feeding the MakeMKV file directly into DaVinci Resolve Studio, but DRS will not ingest MPEG-2 files. (DRS won't ingest MKV files either, and StaxRip cannot mux M4V files properly. There's a merry hell of file dependencies depending on what kind of source you start with in all this).

In my last article I referred to an encode process I call Rubicon. Rubicon is created by ripping the DVD in Handbrake into H.264 Max Production, CFR (I'm evaluating this workflow in VFR right now, but Rubicon as demonstrated was created with CFR):

After ripping, I ran it through a QTGMC repair filter, set for Input Type 2 on the first pass and Input Type 3 on the second. I then run the QTGMC repair function (daa3mod is used for AA as well). This fixes the interlacing and 3:2 pulldown problems, but doesn't do a great job on the credits.

Once the StaxRip encode is finished, I upscaled the video in Topaz VEAI, dropped the upscaled footage into DaVinci Resolve Studio, interpolated 119.88 frames of information into the video stream (Optical Blend, Large Motion Vectors, Enhanced Better), then ran it back through AviSynth with a ChangeFPS(24000, 1001) command.

This produced the following video (Please set to 4K):

https://www.youtube.com/watch?v=8DwfHADU0QY&feature=youtu.be

For comparison, here's the exact same clip from an upscaled MKV of a DS9 episode that I ran through Topaz Video Enhance AI with no further processing:

https://www.youtube.com/watch?v=DzeENB9JKmE&feature=youtu.be

When I started this project I said I wanted to provide a free software workflow and I mean that. Katie Boundary's deinterlacing filter from Page 1 fixed most of the problems I had in Sacrifice of Angels, and the simple QTGMC repair script I'm using appears to fix the rest. It produces a rather nice 48fps version of the show. There's also the option to rip the VOB directly at 29.97 fps. When the same script is used on them, they become 60fps files, and I'm evaluating that option as well just to see how well it compares.

Finally, I'll be evaluating the impact of leaving the show at 119.88 fps without ever decimating the frame rate down at all. (This only applies to the versions of the show that use dummy frames -- the interpolated "supersmooth" variants are not being used.

Right now, I'm wondering what the best way is to get moire out of the early show credits and out of Episode 1-1, Emissary, where the Borg cube is absolutely riddled with it. Is there a way to remove the rainbow effect with a filter, or does it have to be done by hand?

One question I'd really like to know the answer to, if anybody does: Everyone talks about H.265 in terms of its ability to deliver high-quality results in a smaller file size. I don't really care about that aspect of it. What I'm curious about is this: Does H.265 ever produce better visual results than H.264? Are there times when one should use H.265 rather than H.264 because it results in a higher quality picture *in absolute terms, regardless of file size?*

Happy to discuss results and any of the above. Apologies that it's a bit of a muddle.

Link to my last story, if anyone wants to see the video samples. Set for 4K in all cases when available. I upscaled the same clips from a DVD rip, an MKV file, and my own Rubicon upscale method.

https://www.extremetech.com/extreme/310176-deep-space-nine-upscale-project-season-finale-what-we-brought-ahead

SaurusX

12th May 2020, 15:57

What are your QTGMC calls specifically? QTGMC is great and can be used on progressive material, but most of this series does not need it. There are a handful of 60i sequences throughout and that's it. There has already been much discussion in this thread as to the "best" workflow for resolving the interlaced sections of this series. Don't get married to Handbrake.

JoelHruska

12th May 2020, 16:51

What are your QTGMC calls specifically? QTGMC is great and can be used on progressive material, but most of this series does not need it. There are a handful of 60i sequences throughout and that's it.

My intended tutorial workflow doesn't require scene-by-scene editing, and applying QTGMC in repair mode smooths certain AA issues that otherwise crop up. For example, the horizontal grid on the bottom of the Miranda-class saucer:

https://www.youtube.com/watch?v=RC_VQ_n0kA0

Set for 4K, look at the 4-second mark, and you'll see the shimmering pattern. This can be completely eliminated.

https://www.youtube.com/watch?v=oSOR2Uk1aCg

It can actually be eliminated even better than *that*, but with worse judder control. But regardless, QTGMC repair is a functional necessity for that kind of visual improvement. I've tested leaving it off. Unfortunately, I'm losing a bit of detail to YT in a rather important spot, but for a comparison between running just QTGMC in repair mode and using it in combination with the aforementioned D9D interlace filter, see here:

https://www.youtube.com/watch?v=IjVBE0GQvuo

This is the encode I call "5Sharp." QTGMC in repair-mode only. Note the two Jem'Hadar ships with the series of lines on the bottom that pivot into view towards the end of the clip. Before it goes through YouTube upload, on 5Sharp, the bottom of those ships is completely clean. No shimmer. YT is putting some of the distortion back, unfortunately.

Using what I call the "Doom 9 Deinterlacer" (posted on Page 1) in combination with QTGMC makes motion smoother but also loses a bit of clarity on lines. There's also more shimmer on the Galaxy-class windows in the upper-left-hand corner of the screen (check 0:15 - 0:17 seconds) when the D9D solution is used in combination with QTGMC.

https://www.youtube.com/watch?v=FKI9vO4FacQ

The reason to use QTGMC and D9D together is that the D9D deinterlacer alone produces errors and reversed movement in some of the frames during Sacrifice of Angels in a way that is visible during normal playback. Adding QTGMC repair resolved the problem.

I need to actually go post test scripts from downstairs, so I'll drop them in a separate post below. I don't have them duplicated on this system.

I am not "attached" to Handbrake. I am using Handbrake for two specific reasons:

1). DaVinci Resolve Studio will not ingest an MPEG-2 file in any guise *or* an MKV file of any type. Any workflow that begins with DaVinci Resolve Studio must therefore use a non-MKV file (M4V is acceptable to DRS).

2). I don't have a solution for rendering a VFR properly in AviSynth that keeps the A/V aligned when processed via StaxRip, despite repeated attempts to resolve this problem.

So right off the bat, we have two different sets of dependencies that must be resolved:

Beginning with DaVinci requires a non-MPEG-2, non-MKV source: Handbrake.

Beginning with StaxRip and MakeMKV results in a misaligned file. Using commands like "ntsc_film" or setting the file output to 23.976fps in the H.264 encoder will pull the frame rate down from the default 24.66 fps (don't ask me why StaxRip picks 24.66 fps, it just does) down to the appropriate 23.976 fps, but the audio is still improperly synchronized from minute one.

Beginning with StaxRip and an M4V file created in Handbrake will automatically fail; StaxRip cannot mux the audio back together properly.

One final thing to keep in mind: I have absolutely zero control over any encode settings whatsoever from Topaz Video Enhance AI. The application strongly prefers encodes in the 2GB - 4GB range based on my observations. Feeding it 12-24GB source files will still result in a final output of 2-4GB.

One other tidbit from before I go get code: Topaz Video Enhance AI is capable of performing single-stage ingestion if configured to do so using AVFS. I've done the entire AviSynth encode + upscale process in the application at once before. I don't actually know how this would have handled the audio, however, because I used Version 1.1.1 of the program, which did not feature audio pass-through support. The new version does, but I have not repeated this test yet.

Be advised that TVEAI quality continues to shift from version to version. I have all of the releases archived so I can pick and choose which filter implementation I want for the final product.

JoelHruska

12th May 2020, 17:14

Here's the QTGMC repair script I use. This works very well, as far as repairing various source errors and resolving a great deal of aliasing and flickering lines.

I have *extensively* tested QTGMC's repair functions as they apply to DS9 in general and Sacrifice of Angels in particular.

The only downside of this filter is that it does absolutely nothing to fix the credits -- in fact, the juddering it introduces has to be fixed itself through the application of further processing:

5Sharp QTMGC

QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")
Repair(QTGMC2, QTGMC3, 1)

The 0.5 sharpness is where the name "5Sharp" comes from. I tested every single option (0.1 - 1.0) and settled on 0.5 for best practices. Increasing this further can help if you apply additional filters that create blur. I tested Interframe, for example, as a means of offsetting judder, as well as adding MotionBlur to the episode to offset it. Ultimately I've stuck with neither approach, but if you wanted to use them, I'd tweak the sharpness setting here to compensate.

I have tested every combination of "InputType" that exist within this formula (1,1 - 1,2 - 1,3 - 2,1 - 2,2 - 2,3 (etc, etc) and experimented with various other custom settings as well. This is the best filter set I've come up with for keeping the frame rate at 23.976fps, smoothing motion, and eliminating the occasional 3:2 pulldown and deinterlaced frames.

5Sharp + D9D

This results in ~48 fps output (60 fps if used on a VOB extracted via DVD Encrypter and then rendered in StaxRip / AviSynth as below):

Y=nnedi3(field=-2)
A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)
QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.7, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")
Repair(QTGMC2, QTGMC3, 1)

The advantage of this method is that it solves the judder issues in 5Sharp. The disadvantage is that it walks back some of the AA improvements. StaxRip only includes native support for three AA methods that are classified as such (SangNom, DAA, MAA), and of the three, I have found DAA3Mod more effective. I have noticed that there are more options available on the VapourSynth side of things, but I do not know how to code for VapourSynth yet and have not tested them.

The D9D deinterlacer introduces some flicker on windows that I haven't figured out how to eliminate without using DaVinci Resolve Studio, but I'm trying to avoid doing that for my completely free tutorial pathway.

I have also tested every single "Input Type" shift that exists within the above, meaning I rendered the D9D encode path 9 different ways to see the impact of repairing the footage with each different method. I also experimented with accelerating and decimating this footage in QTGMC before using secondary repair methods to smooth it. Ultimately, there are better ways to accomplish this kind of conversion, so I abandoned the attempt. You can, amusingly, use QTGMC to interpolate a 119.88 fps frame rate out of 23.976 fps, and then use various decimation methods (Even, Even, Odd, Even, Odd, Odd) to calculate which frames to dump and which to keep. Lots of interesting results and visual patterns came out of this test series, but nothing that really moved the project forward).

I've experimented extensively with various denoise and degrain filters, attempts to fix moire (the Borg ship in Emissary is a horror show of crappy video quality), color tweaks, and such, but all of that fits into "experimentation" and isn't really part of the core work I'm doing. I've shied away from much in the way of noise removal, because TVEAI already implements its own denoise / deblocking, and I don't want to lose data before I chuck the files in for processing.

One point I am considering is whether earlier seasons require a different approach than later seasons. As weak as the overall DVD quality is, the early seasons are worse than the later by a significant margin.

JoelHruska

12th May 2020, 17:39

One last point: I'm aware that I'm throwing far more firepower at a project of this nature than is typically available to most people. I don't bring that up to brag. I bring it up because I've got far more freedom to run test encodes to see what things look like than most people do.

I don't care about encode times. I literally don't care if it takes 4 days to run one encode. If that's a problem, I'll just launch more encodes, some of which will take less than four days. I am aware that threading support can be dicey, so while I've tested the impact of using multi-threading in AviSynth+ (which is what StaxRip requires), I mostly just let stuff run in default threading modes, then run more encodes to compensate.

I am happy to launch 20 - 40 encodes of the same episode at once with completion times between 2 hours and 4 days to give myself enough to do while waiting for long-term tests to complete.

I also don't care at all about file sizes right now. That's something I'll trim down later, once I know what the best-achievable quality is.

poisondeathray

12th May 2020, 17:49

Right now, I'm wondering what the best way is to get moire out of the early show credits and out of Episode 1-1, Emissary, where the Borg cube is absolutely riddled with it. Is there a way to remove the rainbow effect with a filter, or does it have to be done by hand?

Aliasing and moire patterns can be difficult to remove without degrading the footage or oversmoothing.

One way to approach the credits is to mask /roto them out and filter them differently

Has anyone compared the intros in terms of aliasing (not the text)? Are they exactly the same in say episode1,2,3 ... ? There are ways of signal averaging that might help if there are minor differences, then you can add back the text

Rainbows can be filtered with derainbow filters (most essentially blur the chroma, or some variations)

One question I'd really like to know the answer to, if anybody does: Everyone talks about H.265 in terms of its ability to deliver high-quality results in a smaller file size. I don't really care about that aspect of it. What I'm curious about is this: Does H.265 ever produce better visual results than H.264? Are there times when one should use H.265 rather than H.264 because it results in a higher quality picture *in absolute terms, regardless of file size?*

In absolute terms, both can be lossless, so your question is not phrased very well.

If you use lossless or near lossless with h265, but not with h264, the h265 encode is going to be better in all categories except filesize

Better compression efficiency implies higher quality at a given filesize (bitrate). (But "quality" is difficult to measure or define)

My intended tutorial workflow doesn't require scene-by-scene editing, and applying QTGMC in repair mode smooths certain AA issues that otherwise crop up.

But are you blindly applying that to everything ?

It's counter productive if your goal is to upscale. It's quite destructive to details. You should only use selectively it on problem areas

Are you aiming for ideal results, or more for ease of use workflow?

One final thing to keep in mind: I have absolutely zero control over any encode settings whatsoever from Topaz Video Enhance AI. The application strongly prefers encodes in the 2GB - 4GB range based on my observations. Feeding it 12-24GB source files will still result in a final output of 2-4GB.

Export an image sequence like PNG or TIFF instead . Then compress that properly afterwards. That's the ideal lossless workflow out of of Enhance

zapp7

12th May 2020, 17:52

applying QTGMC in repair mode smooths certain AA issues that otherwise crop up.

Doesn't the use of QTGMC degrade the spatial resolution? My limited understanding of de-interlacing is that half the resolution is lost.

I'm not sure if it's because of YouTube's encoding, but those ships look a bit smoothed for my tastes and I find they have more of a cartoon-ish look. Too each their own, though.

I just went through Sacrifice of Angels using the override file method, and I'm happy with the results. The pulldown effect is removed by TFM in over 99% of the episode, and QTGMC takes care of the few combed frames that are left. The majority of the frames are pretty much unprocessed and left intact. A lossless input is fed to Topaz and there are no audio sync issues with the final result. Making override files is a bit annoying, but not super time consuming and once they're made, they can be shared with others.

It may just be that I'm not observant enough to notice the issues you're trying to address.

One point I am considering is whether earlier seasons require a different approach than later seasons. As weak as the overall DVD quality is, the early seasons are worse than the later by a significant margin.

I noticed this as well! I think the change may have coincided with the revamped intro they introduced in Season 4. The Topaz output looks far superior in later seasons. I wonder if a different Topaz setting (i.e. HQ vs LQ) should be applied before and after this change?

JoelHruska

12th May 2020, 18:24

But are you blindly applying that to everything ?

It's counter productive if your goal is to upscale. It's quite destructive to details. You should only use selectively it on problem areas

Are you aiming for ideal results, or more for ease of use workflow?

You can see the results for yourself. That's why I'm uploading the clips, is so people can see the outcomes in ways that aren't just theoretical.

(I realize that YT takes its own pound of flesh, so I may start using Vimeo, but I can't fix that problem right now): Set for 4K viewing in all cases when available.

First Fleet Attack:

DVD:

https://youtu.be/f56nFLey724

Upscaled MKV:

https://youtu.be/DzeENB9JKmE

Rubicon (My own upscaling workflow, including QTGMC applied as above):

https://youtu.be/8DwfHADU0QY

The first few seconds of this video, in comparison to the upscaled MKV, are the perfect selling point for Rubicon as compared to the other methods (in my own opinion). The approach is far smoother. There's far less shimmer.

Second Fleet Attack:

DVD:

https://youtu.be/MzJx0uuaTko

Upscaled MKV:

https://www.youtube.com/watch?v=RC_VQ_n0kA0

Rubicon:

https://youtu.be/oSOR2Uk1aCg

Conversation on Defiant:

DVD:

https://www.youtube.com/watch?v=niwY_UkcKAA

Upscaled MKV:

https://www.youtube.com/watch?v=1ePspekugbs

Rubicon (100ms audio offset in this video, since corrected):

https://www.youtube.com/watch?v=QQbX48HSA2U

These clips compare the original DVD source against an MKV file that's been fed through Topaz VEAI using the Gaia-CG preset against "Rubicon" -- a DVD source file that I've processed extensively, including temporarily converting it to a 119.88 fps file.

Do you think the look of Rubicon reflects an overall loss of detail or quality level compared to either the MKV upscale or the DVD?

Export an image sequence like PNG or TIFF instead . Then compress that properly afterwards. That's the ideal lossless workflow out of of Enhance

Theoretically possible, but it'll be murder on storage requirements. The output from Topaz VEAI is about 4.8MB per frame per PNG, and about 16MB per frame per TIFF. At 65,000-odd frames of content, that's about 320GB per episode if you use PNG, and about 1TB per episode for TIFF.

I could build a workflow around sizes that large, but I'm not planning to do it.

poisondeathray

12th May 2020, 19:02

You can see the results for yourself. That's why I'm uploading the clips, is so people can see the outcomes in ways that aren't just theoretical.

(I realize that YT takes its own pound of flesh, so I may start using Vimeo, but I can't fix that problem right now): Set for 4K viewing in all cases when available.

It's not very useful to compare on youtube; vimeo is slightly better but still not ideal.

I'm not disputing the usefulness reduction in shimmer on sections that require it. I'm saying you're going to lose details in other sections that do not have shimmer to begin with. It's a destructive filter - it's not some theoretical loss of details; this is how the filter works. Actual details are discarded before you upscale.

It doesn't make sense in my mind to jump through all these hoops and they dummy it down with crappy encoding quality, or to discard actual details before upscaling. It's your choice, you decide where you are willing to make compromises

videoh

12th May 2020, 19:03

I'm aware that I'm throwing far more firepower at a project of this nature than is typically available to most people. Wow, you're so awesome. Where is the 'bow down peons' smilie?

JoelHruska

12th May 2020, 19:41

Read the next sentence. The point is not braggery. The point is to arrive at a better final result by being willing to evaluate every potential outcome, partly because, as I have written publicly in numerous stories, I am a giant n00b.

I have the following hardware available to me for this project:

AMD Ryzen Threadripper 3990X
AMD Ryzen Threadripper 3970X
AMD Ryzen 9 3950X
Intel Core i9-10980XE
Intel Core i9-9980XE
Intel Core i9-9900K
Intel Core i9-9980HK

I also have: Two RTX 2080s, two RTX 2070s, three RTX 2060s, a 1080 Ti, and a GTX 1080.

I can bring 3-4 of these systems up at once. Currently, I'm using two of them -- the 3990X and the 10980XE. The 9980HK is on "hot standby" I suppose you could say. I'm willing to put more into service if it allows for more effective / efficient testing.

I'm not sitting here trying to preen. I'm trying to deploy available hardware resources to best test potential outcomes. If you're willing to engage with that, by all means, feel free. If you want to turn the facts of my job into some kind of pretended and pretentious value judgment by me, you can have that discussion with yourself. I won't be participating.

JoelHruska

12th May 2020, 19:51

It's not very useful to compare on youtube; vimeo is slightly better but still not ideal.

I'm not disputing the usefulness reduction in shimmer on sections that require it. I'm saying you're going to lose details in other sections that do not have shimmer to begin with. It's a destructive filter - it's not some theoretical loss of details; this is how the filter works. Actual details are discarded before you upscale.

It doesn't make sense in my mind to jump through all these hoops and they dummy it down with crappy encoding quality, or to discard actual details before upscaling. It's your choice, you decide where you are willing to make compromises

So, what you propose is something more along these lines:

MakeMKV or Handbrake directly into Topaz VEAI. Upscale.
Test everything else *post* upscale.

I hadn't done this, for the simple reason that Topaz VEAI has no idea how to handle a VFR file. Given the input from MakeMKV, it will read a file with some 80,000+ frames and encode the entire output at 29.97 fps. The output from Topaz VEAI if handed MakeMKV source will have to be pulled through another framerate conversion. This is another hidden dependency / problem.

If I encode losslessly in Handbrake and use CFR at that point, I could pull the file directly into Topaz VEAI. However, remember, I have no control over Topaz VEAI's settings at this point. Any detail it throws away is going to be lost forever.

So...

Your proposed workflow appears to be:

Handbrake Source (CFR file, to allow for proper Topaz ingestion). ~6GB initial file.
Topaz VEAI upscale. (~2GB output file, despite 4x nominal resolution upscale).
Followed by either:

DaVinci Studio Resolve (for interpolated or dummy frame rate conversion) + AviSynth (for decimation and QTGMC repair).

or

AviSynth (for QTGMC repair) followed by DaVinci Studio Resolve (for interpolated / dummy frame rate conversion)

With an optional final AviSynth run to yank the frame rate back down, if desired.

Obviously the DaVinci runs can be left out entirely if desired, but I'm fitting this to the workflow I'm actually using at the moment.

I'll get started on this today. The upscale alone will take about 10 hours, so it'll be tomorrow before I have results. I'll test both workflows and see how they compare.

poisondeathray

12th May 2020, 20:43

You're shooting yourself in the foot and leaving details on the table before you even upscale.

The amount of detail loss will be approx. proportional to the amount of motion in the scene. The more motion in the shot, the more details you will lose . With very low or no motion, you might not lose much.

IVTC, vs. your "5sharp". Here is an animated webp. Most browsers should be able to display it. It's 1:1 crop , non par corrected, and I brightened it up a bit

https://forum.videohelp.com/threads/395895-animated-webp-test#post2582929

or if you prefer png (hit 1 or 2 number keys) . I couldn't get animated png to embed on d9
http://www.framecompare.com/image-compare/screenshotcomparison/9JC2JNNU

Look at the blurred away skin textures,loss of wrinkles around the eye, neck textures blurred away - this guy looks 3 years younger now. The specular highlights in the hair are reduced. Costume (leather?) textures are softened .

It's not show in this frame comparison, but there are temporal artifacts in adjacent frames - the teeth are discolored from ghosting , the ear ridges are distored along a different direction

The point is - if 99% of it is 23.976p, it doesn't make any sense to dumb down 99% to fix 1% . You should apply the damaging filters selectively.

So, what you propose is something more along these lines:

MakeMKV or Handbrake directly into Topaz VEAI. Upscale.
Test everything else *post* upscale.

I hadn't done this, for the simple reason that Topaz VEAI has no idea how to handle a VFR file.

No.

Proper preprocessing is essential prior to upscaling

If 99% of it is 23.976, it doesn't make any sense to dumb down 99% to fix 1% . You should apply the filters selectively.

The way you handle VFR is external timecodes . Each program does not need to know it's VFR, or what the frame rate is, they just look at "frames"

Best practices is going to be VFR . Everything runs at the proper framerate. The only problem is mixed sections with multiple frame rates within the same frame (eg. text overlays over 23.976p sections) . Those I would mask out and treat differently. I imagine the only differences should be the guest stars per episode

I use RGB images into upscale because you can control the RGB conversion (chroma upscaling algorithm used) and you can scale it full range (and scale it back when converting back to YUV) . It makes a difference on things like space scenes. Or even indoor scenes with bright lights , or dark shadows regions. AI Upscale will use a standard range conversion if you feed it YUV - it will clip Y16-235 instead of preserving all the data 0-255.

You can choose where you want to make trade offs in the workflow, but the one thing you should not ever do is apply a damaging filter like QTGMC in progressive mode to every frame.

videoh

12th May 2020, 21:16

If you want to turn the facts of my job into some kind of pretended and pretentious value judgment by me, you can have that discussion with yourself. I won't be participating. That's great. I can pontificate at will. You know you sound like a condescending and entitled prima donna, right? Is that working for you in life? Bluster and bullying can take you only so far.

poisondeathray

12th May 2020, 21:30

Export an image sequence like PNG or TIFF instead . Then compress that properly afterwards. That's the ideal lossless workflow out of of Enhance

Here's a demo illustrating what you "leave on the table" by using mpeg4 output. Note it's using mpeg4-asp (xvid)

Maybe in future versions, they will have more control over the export settings, such as using AVC, or higher bitrates, but right now, it' s a TERRIBLE idea. You're shooting yourself in the other foot.

This is non par corrected. It's just a straight IVTC and upscale from one of the samples posted. No preprocessing. Export out of Upscale AI

mpeg4 output
https://i.postimg.cc/qByRNhSM/mpeg4.png

png output
https://i.postimg.cc/d0gs1w9y/png.png

Notice the blocky compression artifacts and poor shadow detail. Any details that you upscale in darker areas are going to become blocky artifacts. That's before you even upload to YT or whatever you're planning on doing.

It's a complete waste of time to upscale if you're going to use mpeg4 output

So those 2 things are the absolute must not do's

SaurusX

12th May 2020, 21:35

divx had its day, but it's long past. With Topaz you have to export to PNG if you want quality. And if you don't then what's the point of this project?

zapp7

12th May 2020, 21:59

I use RGB images into upscale because you can control the RGB conversion (chroma upscaling algorithm used) and you can scale it full range (and scale it back when converting back to YUV) . It makes a difference on things like space scenes. Or even indoor scenes with bright lights , or dark shadows regions. AI Upscale will use a standard range conversion if you feed it YUV - it will clip Y16-235 instead of preserving all the data 0-255.

I'm quite interested in this - how would one convert to RGB with this workflow? (and then back to YUV)

DVD Decrypter -> DGIndex -> AviSynth(TFM, TDecimate) -> Topaz

mpeg4 output
https://i.postimg.cc/qByRNhSM/mpeg4.png

png output
https://i.postimg.cc/d0gs1w9y/png.png

Notice the blocky compression artifacts and poor shadow detail. Any details that you upscale in darker areas are going to become blocky artifacts. That's before you even upload to YT or whatever you're planning on doing.

Damn, you've convinced me to switch to image output. I'll have to redo the 10 or so episodes I've already done. One question: Is .png or .tiff better? I've heard competing claims in that regard. Which one preserves more quality?

JoelHruska

12th May 2020, 22:10

*snip*
No.

Proper preprocessing is essential prior to upscaling

If 99% of it is 23.976, it doesn't make any sense to dumb down 99% to fix 1% . You should apply the filters selectively.

The way you handle VFR is external timecodes . Each program does not need to know it's VFR, or what the frame rate is, they just look at "frames"

Best practices is going to be VFR . Everything runs at the proper framerate. The only problem is mixed sections with multiple frame rates within the same frame (eg. text overlays over 23.976p sections) . Those I would mask out and treat differently. I imagine the only differences should be the guest stars per episode

I use RGB images into upscale because you can control the RGB conversion (chroma upscaling algorithm used) and you can scale it full range (and scale it back when converting back to YUV) . It makes a difference on things like space scenes. Or even indoor scenes with bright lights , or dark shadows regions. AI Upscale will use a standard range conversion if you feed it YUV - it will clip Y16-235 instead of preserving all the data 0-255.

You can choose where you want to make trade offs in the workflow, but the one thing you should not ever do is apply a damaging filter like QTGMC in progressive mode to every frame.

Alright. So how, then, does one generate / extract a proper timecode for this kind of operation? I attempted to do so via several methods, but never managed it successfully when using MakeMKV source.

Well, either I never successfully managed it, or I managed it but the audio was still misaligned. I genuinely forget which. I pivoted to investigate other methods of frame rate adjustment because I was having trouble with VFR ingestion.

I would be more than happy to test the impact of applying QTGMC and other repair methods on a frame by frame scripted basis, at least in a trial basis. I do not necessarily anticipate writing custom scripts for each and every episode, but I would be very happy to test the impact of this method of repair versus the more global application method I've been using.

JoelHruska

12th May 2020, 22:23

That's great. I can pontificate at will. You know you sound like a condescending and entitled prima donna, right? Is that working for you in life? Bluster and bullying can take you only so far.

That's your interpretation. Another interpretation is that I'm compensating for my complete lack of knowledge by running tons of encodes while painfully aware that if I knew what I was doing, I wouldn't have to. This interpretation has the benefit of being supported by evidence, including quotes from my own articles like these:

"I’m never going to be able to make these old DVDs look as good as what Paramount could do. Heck, I’m never going to improve them as much as a professional video editor could do. But Paramount has no plans to upgrade DS9 itself, and that means the only way to restore the TV show to some semblance of how it could look is with a lot of elbow grease, filter testing, and one exhausted RTX 2080. I think the work deserves to be done, even if Paramount disagrees."

https://www.extremetech.com/extreme/308505-star-trek-deep-space-9-upscale-project-update

"despite having no previous experience or education in video editing."

"Paramount could build a better version of the show than I could even hope to create — but since they aren’t going to bother, I figure I’ll keep up my own efforts."

I have referred to myself as "Barely video literate."

https://www.extremetech.com/extreme/309653-deep-space-nine-upscale-project-update-variable-frame-rate-dvds-can-burn-in-hell

I have talked about various issues which have confused and perplexed me (in the same link). I have admitted to previous mistakes made in earlier articles (in the same link).

My own project goals include statements like: "Be willing to laugh at some of the ridiculously bad quality encodes you will occasionally create, especially if it takes 1-2 days to create them."

https://www.extremetech.com/extreme/310224-deep-space-nine-upscale-project-ds9up-technical-goals-and-faq

But sure. Having repeatedly called myself a newb, the most obvious explanation for why I made that statement about available hardware is to somehow establish myself as having enormous credibility in front of a group of people I made it clear I don't expect to find me credible.

Newsflash: I don't consider owning a 3990X to be worth much in the way of credibility. The only thing the CPU is good for, absent knowing *what* code to run on it, is running lots and lots of code at once. I'm aware of that. I assumed you were, too.

Alternately -- and I'm just spitballing here -- you might conclude I emphasized my willingness to run a lot of different render tests as a way of illustrating my good faith and commitment to a project, despite the fact that I'm completely new at this. I didn't come here to impress anyone with the size of my CPU cores. I didn't come here to impress anyone at all. I came here to find better information on how to rip DVDs to create a better effective remaster of Deep Space 9, because this is the forum to come to for that kind of information.

Because I didn't want to be viewed as an imposter, I waited until I'd put in three months of work and rendered out a great deal of footage to even start having the conversation. Because I take what you guys know seriously. Because I take your expertise seriously. Because I, also, want to be taken seriously as someone who is here to learn and who cares about this content. So I emphasized what I can do and how seriously I'm taking it.

You can decide which of these reasons you find more persuasive. I'm done with the topic.

JoelHruska

12th May 2020, 22:27

I'm quite interested in this - how would one convert to RGB with this workflow? (and then back to YUV)

DVD Decrypter -> DGIndex -> AviSynth(TFM, TDecimate) -> Topaz

Damn, you've convinced me to switch to image output. I'll have to redo the 10 or so episodes I've already done. One question: Is .png or .tiff better? I've heard competing claims in that regard. Which one preserves more quality?

It's not just going to be a question of quality. Upscaled episodes are about 16MB per frame at 400%, from my own testing. PNGs are about 4MB. That's going to take 335GB - 1TB of storage per uncompressed episode, assuming you do the entire episode at once.

That is how you get lossless, however.

I have not compared the quality difference between 16-bit TIFF and PNG and cannot comment on which is better.

SaurusX

12th May 2020, 23:15

It's not just going to be a question of quality. Upscaled episodes are about 16MB per frame at 400%, from my own testing. PNGs are about 4MB. That's going to take 335GB - 1TB of storage per uncompressed episode, assuming you do the entire episode at once.

That is how you get lossless, however.

I have not compared the quality difference between 16-bit TIFF and PNG and cannot comment on which is better.

I don't think the difference between PNG and TIFF can be appreciated with human eyeballs. And the episode is only lossless until you use ffmpeg (or whatever) to encode that string on pictures into a final video. Then you're looking at 2-4GB per ep depending on the quality settings.

poisondeathray

12th May 2020, 23:32

I'm quite interested in this - how would one convert to RGB with this workflow? (and then back to YUV)

DVD Decrypter -> DGIndex -> AviSynth(TFM, TDecimate) -> Topaz

If you want full range, in avisynth you would use the "PC matrices" . This "maps" Y 0-255 <=> RGB 0-255 . So the contrast will look lower , washed out. But that doesn't matter, you'll fix it later. The goal is to preserve all the details, instead of clipping some data prior to upscaling.
ConvertToRGB24(matrix="PC.601")

You can also control the upsampling algorithm. By default "bicubic" is used for avisynth. Maybe you want sharper chroma scaling, or maybe less sharp. Maybe you want to use some more exotic chroma interpolation upscaling methods before the Upscale AI stage.

(This is f^&%^ slow, but can work ok on some types of content)
https://forum.doom9.org/showthread.php?t=173005

After the upscaling stage, it's better to adjust it in another application, such as resolve or any NLE. You have more control to adjust more shadow details or super bright details and to bring the contrast back.

Is .png or .tiff better? I've heard competing claims in that regard. Which one preserves more quality?

In theory both should be the same . PNG just uses lossless compression. TIFF can use lossless or lossy compression, but most variants use uncompressed lossless. Another potential issue is some applications write colr tags in PNG, this can cause some programs to display the colors slightly shifted. For example ffmpeg writes PNG with this when the input file has colorimetry tags. Upscale AI does not do this (untagged), so PNG output is fine.

poisondeathray

12th May 2020, 23:36

Alright. So how, then, does one generate / extract a proper timecode for this kind of operation? I attempted to do so via several methods, but never managed it successfully when using MakeMKV source.

Well, either I never successfully managed it, or I managed it but the audio was still misaligned. I genuinely forget which. I pivoted to investigate other methods of frame rate adjustment because I was having trouble with VFR ingestion.

I would be more than happy to test the impact of applying QTGMC and other repair methods on a frame by frame scripted basis, at least in a trial basis. I do not necessarily anticipate writing custom scripts for each and every episode, but I would be very happy to test the impact of this method of repair versus the more global application method I've been using.

Timecode VFR is too big of a discussion to go into for this thread - but the information is out there, examples this forum and others. There are guides around as well. 2pass TIVTC VFR is a commonly used method for avisynth. But if you 've looked at some earlier posts, some overrides might be necessary for some sections

Definitely there are some bad aliasing sections, and those need attention. But I'm suggesting you filter those separately. It's way too damaging to apply to the 99% of content for scenes with motion.

It's easy in avisynth to apply different filters to different sections (it's not really "frame by frame") - But how do you "automatically" identify those with any accuracy without human eyes? I don't know.

zapp7

13th May 2020, 00:28

If you want full range, in avisynth you would use the "PC matrices" . This "maps" Y 0-255 <=> RGB 0-255 . So the contrast will look lower , washed out. But that doesn't matter, you'll fix it later. The goal is to preserve all the details, instead of clipping some data prior to upscaling.
ConvertToRGB24(matrix="PC.601")

You can also control the upsampling algorithm. By default "bicubic" is used for avisynth. Maybe you want sharper chroma scaling, or maybe less sharp. Maybe you want to use some more exotic chroma interpolation upscaling methods before the Upscale AI stage.

So before feeding into Topaz, I do ConvertToRGB24(matrix="PC.601"), and after upscaling with Topaz I convert back with ConvertToYV12(matrix="PC.601"). Is that right?

I'm also not using any other upscaling algorithm other than the Topaz software.

Chir

13th May 2020, 00:56

Do you correct for the SAR at any point later in the process? I ask because this command appears to produce a horizontally stretched image. It's not super obvious but it's evident if you compare your output it to the original DVD footage. I used the codec because it's my understanding that SAR can't be corrected using the "copy" command. I read that CF 0 H.264 is lossless so I was hoping that would be sufficient.

I have about 10 episodes "complete", and I put that in quotes because I feel as though I am still learning the whole process and making refinements. There are undoubtedly errors in what I have. Should I share anyway? What do you think?

I wasn't aware of the SAR issue -- I'll take a peek at my results and see if I notice anything. Thanks for the heads up!

Please! Share what you have in the git hub! If github is intimidating (and it really can be!) feel free to just send me an email and I'll put them in for you. What I have out there is not yet complete, either -- and I think the best way is just to share share share. I'd like to see a "signoff process" where a couple people go through each episode and give a thumbs up to cross check.

poisondeathray

13th May 2020, 01:18

So before feeding into Topaz, I do ConvertToRGB24(matrix="PC.601"), and after upscaling with Topaz I convert back with ConvertToYV12(matrix="PC.601"). Is that right?

I'm also not using any other upscaling algorithm other than the Topaz software.

Yes, that essentially does the reverse operation back to YV12 in full range

But you will get generally better results if you take the time to adjust it in another program. Often the way these TV shows are graded , they are "harsh", and the shadow detail is all clustered together instead of gradual tonal separation

JoelHruska

13th May 2020, 02:41

Timecode VFR is too big of a discussion to go into for this thread - but the information is out there, examples this forum and others. There are guides around as well. 2pass TIVTC VFR is a commonly used method for avisynth. But if you 've looked at some earlier posts, some overrides might be necessary for some sections

Definitely there are some bad aliasing sections, and those need attention. But I'm suggesting you filter those separately. It's way too damaging to apply to the 99% of content for scenes with motion.

It's easy in avisynth to apply different filters to different sections (it's not really "frame by frame") - But how do you "automatically" identify those with any accuracy without human eyes? I don't know.

All of my work to shift to 119.88 fps has been undertaken to deal with the VFR problem, that being the strategy described as "most widely compatible" in the AviSynth Wiki.(keeping in mind that I'm writing a tutorial for people to follow, so it needs to as simple as possible).

Right now, the single greatest roadblock to refining my method in the manner you suggest is trying to figure out how to properly generate timecodes in a manner that allow me to keep the A/V synchronized and use MakeMKV in the first place. If I can't handle the VFR output in AviSynth from the start, then I have to perform a framerate conversion no matter what application I pull it into.

But the two things I'm getting out of QTGMC that I functionally care about are:

1). Repair for the interlaced and 3:2 pulldown frames scattered throughout the content.

2). Rather excellent horizontal AA performance, which is particularly helpful in some otherwise jagged scenes.

If I can duplicate those two functionalities in other filters and use a lighter / better touch to fix the errors in the source elsewhere, I'll certainly take a crack at it.

I'll see if I can get some usable timecodes.

poisondeathray

13th May 2020, 03:36

All of my work to shift to 119.88 fps has been undertaken to deal with the VFR problem, that being the strategy described as "most widely compatible" in the AviSynth Wiki.(keeping in mind that I'm writing a tutorial for people to follow, so it needs to as simple as possible).

Right now, the single greatest roadblock to refining my method in the manner you suggest is trying to figure out how to properly generate timecodes in a manner that allow me to keep the A/V synchronized and use MakeMKV in the first place. If I can't handle the VFR output in AviSynth from the start, then I have to perform a framerate conversion no matter what application I pull it into.

It's a bit misleading. "119.98" CFR because "23.976" is evenly divisible into it (x5) , and so are all common framerates of 29.97 (x4), 59.94(x2). The frames are "evenly" spaced apart in time, so it's smooth. I dislike it because many displays refresh at 60Hz anyways. And if you have 120Hz or similar judderless display then the point is moot anyways too. Also you encode 5x more frames - it's not insignificant. And 119.98fps UHD is not fun to play back on many systems - so not really that "compatible" in that sense. And if youtube is one of your "goals" for an audience, it's not supported either.

I'm talking about timecode VFR. Each section plays back at it's original framerate. 23.976p sections playback at 23.976 ; 29.97 at 29.97p ; 59.94 at 59.94, or whatever at whatever (some content can have different base frame rates other than those). The only problem is mixed rates in 1 frame - ie. the text and credit overlays. Personally I would take the underlying BG frame rate , but there are ways to interpolate one or the other. I dislike motion interpolation because it's prone to artifacts. If you want to do a good job it takes lots of manual work and motion tracking to guide the motion estimation, along with compositing/masks. (That's a tradeoff I'd be willing to make, I'd just take the BG framerate, and mask out the text to filter differently)

You don't need VFR "within" avisynth, or any program. You can assume any temporary framerate. It's the timecodes afterwards that you mux into the container that control the playback FPS . Each frame is unique, instead of encoded 5x the number of frames with duplicates. Unique frames also helps with temporal filters, such as QTGMC, or others such as temporal denoising. Duplicate frames often cause problems with temporal filters.

The most popular method is the 2pass TIVTC VFR, it's also mentioned in the wiki. That generates timecodes that can be used in MKV (or other containers like MP4). There are more complete instructions in the filter documentation

If there are just a few sections and you know exactly what sections have what (e.g. the intro has aliasing/flicker problems maybe needs some QTGMC in mode 2 or 3, maybe some other section etc...) , then you can just manually specify those, and the timecodes file. It will actually be faster than a 2pass TIVTC run. Have a look at the mkvmerge documentation under the "external timestamp" section. (it used to be called "timecodes", they refer to the same thing). The formatting is very easy for v1 timestamps

But the two things I'm getting out of QTGMC that I functionally care about are:

1). Repair for the interlaced and 3:2 pulldown frames scattered throughout the content.

2). Rather excellent horizontal AA performance, which is particularly helpful in some otherwise jagged scenes.

If I can duplicate those two functionalities in other filters and use a lighter / better touch to fix the errors in the source elsewhere, I'll certainly take a crack at it.

I'll see if I can get some usable timecodes.

If 99% of these DVD's is mostly clean 3:2 pulldown (that's what others are saying), then you should IVTC 99%. That's how you return the original high quality frames to start with before your upscale. Anything other than that will yield lower quality. Period. If you try QTGMC in progresive mode, anything with lots of motion will lead to moderate quality loss before you even start upscaling.

Absolutely QTGMC can play a role in that 1% problem areas, but I would not use it anywhere else - way too damaging

JoelHruska

13th May 2020, 03:52

I'm talking about timecode VFR. Each section plays back at it's original framerate. 23.976p sections playback at 23.976 ; 29.97 at 29.97p ; 59.94 at 59.94, or whatever at whatever (some content can have different base frame rates other than those). The only problem is mixed rates in 1 frame - ie. the text and credit overlays. Personally I would take the underlying BG frame rate , but there are ways to interpolate one or the other. I dislike motion interpolation because it's prone to artifacts. If you want to do a good job it takes lots of manual work and motion tracking to guide the motion estimation, along with compositing/masks. (That's a tradeoff I'd be willing to make, I'd just take the BG framerate, and mask out the text to filter differently)

You don't need VFR "within" avisynth, or any program. You can assume any temporary framerate. It's the timecodes afterwards that you mux into the container that control the playback FPS . Each frame is unique, instead of encoded 5x the number of frames with duplicates. Unique frames also helps with temporal filters, such as QTGMC, or others such as temporal denoising. Duplicate frames often cause problems with temporal filters.

The most popular method is the 2pass TIVTC VFR, it's also mentioned in the wiki. That generates timecodes that can be used in MKV (or other containers like MP4). There are more complete instructions in the filter documentation.

I was never able to get this method to work. The Wiki begins by stating: "For this you'll need TIVTC and avi_tc."

Only problem is, AVI_TC is a dead program. It's no longer online. It's no longer available anywhere. I did track down a version of it installed as part of a different third-party application, but was completely unable to get it to do anything useful.

Since I couldn't ever find a useful version of the application I was supposed to use, I dropped this line of approach. I was unable to determine how to actually use the methods described in the wiki. I also wasn't very pleased with the quality I was getting from Vdub.

If there are just a few sections and you know exactly what sections have what (e.g. the intro has aliasing/flicker problems maybe needs some QTGMC in mode 2 or 3, maybe some other section etc...) , then you can just manually specify those, and the timecodes file. It will actually be faster than a 2pass TIVTC run. Have a look at the mkvmerge documentation under the "external timestamp" section. (it used to be called "timecodes", they refer to the same thing). The formatting is very easy for v1 timestamps.

I'll see what I can do. I've had precious little luck with this method before.

If 99% of these DVD's is mostly clean 3:2 pulldown (that's what others are saying), then you should IVTC 99%. That's how you return the original high quality frames to start with before your upscale. Anything other than that will yield lower quality. Period. If you try QTGMC in progresive mode, anything with lots of motion will lead to moderate quality loss before you even start upscaling.

Absolutely QTGMC can play a role in that 1% problem areas, but I would not use it anywhere else - way too damaging

It's not 3:2 pulldown at all. According to DGIndex, it's about 93% film progressive and 7% "Other." The "Other" runs the gamut from single interlaced frames (usually between different clips, though there are some exceptions) to multiple instances of 3:2 pulldown that are encoded directly into the source DVD.

So, for example.

You'll get: Progressive Frame 1, 2, 3, 4, 5, 6,.... 200. Then, the sequence of frames from 201 - 2010 will be in 3:2 pulldown.
(201, 202, 203 progressive, 204/205 interlaced, 206, 207, 208 progressive, 209/2010 interlaced). Then 211 - 500 will be progressive. 501 - 600 will be a short clip of 29.97 fps, which may be interspersed between two other clips of 23.976 fps, all of which take place on the same set. From 601, it'll all be progressive again. (Numbers are strictly representative, not a literal example).

The credits are always 29.97 fps but they are not in 3:2 pulldown either. They're just really pig ugly in the DVD source.

zapp7

13th May 2020, 04:26

It's not 3:2 pulldown at all. According to DGIndex, it's about 93% film progressive and 7% "Other." The "Other" runs the gamut from single interlaced frames (usually between different clips, though there are some exceptions) to multiple instances of 3:2 pulldown that are encoded directly into the source DVD.

It is 3:2 pulldown in most of those film sections though. You can easily apply TIVTC to the film sections that are pulldown and it works very well. The only problem sections are where, like you said, there is some other framerate like credits on the screen. The existence of a frame here and there that goes against the pattern doesn't mean that TIVTC won't work for the majority of the episode. The approach to preserve as much quality as possible is to TIVTC and then only QTGMC where it's needed.

For the end credits we have just been treating them as video and blending the frames. I don't really care about the end credits, anyway.

poisondeathray

13th May 2020, 04:43

Only problem is, AVI_TC is a dead program. It's no longer online. It's no longer available anywhere. I did track down a version of it installed as part of a different third-party application, but was completely unable to get it to do anything useful.

Here is a mirror for the avi_tc_package .

https://forum.doom9.org/showthread.php?p=1445996#post1445996

It's not 3:2 pulldown at all. According to DGIndex, it's about 93% film progressive and 7% "Other." The "Other" runs the gamut from single interlaced frames (usually between different clips, though there are some exceptions) to multiple instances of 3:2 pulldown that are encoded directly into the source DVD.

So, for example.

You'll get: Progressive Frame 1, 2, 3, 4, 5, 6,.... 200. Then, the sequence of frames from 201 - 2010 will be in 3:2 pulldown.
(201, 202, 203 progressive, 204/205 interlaced, 206, 207, 208 progressive, 209/2010 interlaced). Then 211 - 500 will be progressive. 501 - 600 will be a short clip of 29.97 fps, which may be interspersed between two other clips of 23.976 fps, all of which take place on the same set. From 601, it'll all be progressive again. (Numbers are strictly representative, not a literal example).

The credits are always 29.97 fps but they are not in 3:2 pulldown either. They're just really pig ugly in the DVD source.

So it's mixed, 93% 3:2 soft pulldown, and "other" is 3:2 hard pulldown, along with 29.97p sections - but that does not really change what you do . The goal - always - is to return the original progressive frames. That's what field matching does (TFM). You have 23.976p and 29.97p sections in that example.

And I think it was mentioned there were some pure interlaced sections somewhere, those become 59.94p, and it's appropriate to use QTGMC there , in normal mode. This is more tricky to do, because you have to use TFM mode 6, and there are some timecode issues with it. IMO it's easier to edit the script and timecodes for 59.94p sections if there aren't that many

JoelHruska

13th May 2020, 04:54

So it's mixed, 93% 3:2 soft pulldown, and "other" is 3:2 hard pulldown, along with 29.97p sections - but that does not really change what you do . The goal - always - is to return the original progressive frames. That's what field matching does (TFM). You have 23.976p and 29.97p sections in that example.

I don't know what you mean by "soft pulldown." The overwhelming majority of the footage in later episodes like Sacrifice of Angels is 23.976 progressive.

I am looking at the MakeMKV source in MPC-HC with all deinterlacing disabled. The application is set to show progressive frames.

I am staring at sequences of hundreds of frames in length, each of them a precise distance from the next (in terms of motion), with no interlacing or 3:2 pulldown artifacts or anything of the sort.

Interspersed with this, as I've said, is the occasional clearly visible pattern of 3:2 pulldown and the odd interlaced frame or two. Beyond that, there are none. I can upload a sequence of frames in the AM if you want visual proof.

This applies to both space combat scenes and in-person shots.

poisondeathray

13th May 2020, 05:08

I don't know what you mean by "soft pulldown." The overwhelming majority of the footage in later episodes like Sacrifice of Angels is 23.976 progressive.

Soft pulldown is 23.976p content, but encoded progressive. Repeat field flags output a 29.97i signal in a 3:2 pattern . If you ignore the flags, then you see 23.976p . Some software players IVTC on the fly, others just ignore flags. Some can be set to honor pulldown flags

Hard pulldown is 23.976p content, but encoded interlaced. Instead of repeat field flags, actual repeated fields are encoded in a 3:2 pattern

Note both have 23.976p content. Hard pulldown is just stored differently. In both cases IVTC will return the original 23.976p frames.

hello_hello

13th May 2020, 06:29

I am not "attached" to Handbrake. I am using Handbrake for two specific reasons:

1). DaVinci Resolve Studio will not ingest an MPEG-2 file in any guise *or* an MKV file of any type. Any workflow that begins with DaVinci Resolve Studio must therefore use a non-MKV file (M4V is acceptable to DRS).

2). I don't have a solution for rendering a VFR properly in AviSynth that keeps the A/V aligned when processed via StaxRip, despite repeated attempts to resolve this problem.

So right off the bat, we have two different sets of dependencies that must be resolved:

Beginning with DaVinci requires a non-MPEG-2, non-MKV source: Handbrake.

Beginning with StaxRip and MakeMKV results in a misaligned file. Using commands like "ntsc_film" or setting the file output to 23.976fps in the H.264 encoder will pull the frame rate down from the default 24.66 fps (don't ask me why StaxRip picks 24.66 fps, it just does) down to the appropriate 23.976 fps, but the audio is still improperly synchronized from minute one..

I'm pretty sure the reason for the odd 24.66fps frame rate, is when you rip with MKV and open with StaxRip, it's using a method for indexing and opening the video (probably lsmash) that doesn't honour repeat flags by default. So it sees the interlaced/hard telecined sections as 29.97fps and the film sections as 23.976, but as Avisynth needs a constant frame rate, it decodes at the average frame rate. As the source is mostly 23.976fps, the average works out at 24.66fps. The interlaced parts are too slow and the film parts too fast, so the A/V sync is lost.

If you open a vob file and get a constant 29.97fps, it's because the repeat flags are being honoured so the interlaced sections are 29.97fps and the film sections are telecined to 29.97fps. That's where you should start, even though eventually you'll be reversing the soft telecine created via repeat flags on playback, as you'd do for the hard telecined parts (the telecine is encoded). If you've already ripped with MakeMKV, you can open the MKVs with TSMuxer and remux them as TS files. Using TS files as the source, I suspect StaxRip will index with DGIndex as it probably does for vob files, and the repeat flags will be honoured.

Katie's method bob de-interlaces the whole lot to 60fps, I assume to avoid problems where interlaced content has been overlaid on hard-telecined content, but you'd end up with lots of repeated frames in the film sections.

AnimeIVTC has a mode for dealing with interlaced stuff overlaid on telecined stuff. I used a sample from an earlier post and encoded with a script using AnimeIVTC for the opening credits/CGI part, and TIVTC for everything else. The output is a constant 23.976fps.

I didn't try to fix the remaining aliasing. I think it's encoded into the video anyway. I followed AnimeIVTC() with AddGrainC() as I thought it might look more natural.
The source is Stereodude's sample from here.
https://forum.doom9.org/showthread.php?p=1906994#post1906994
There's a link for the encodes at the end of this post.

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

Trim(0,734).TFM().TDecimate().MCDegrainSharp()++\
Trim(735,3000).AnimeIVTC(Mode=3, cbob=4, aa=2).AddGrainC()++\
Trim(3001,0).TFM().TDecimate().MCDegrainSharp()

Resize8(640,480)
GradFun3()

A modified version of the Katie method to remove the unnecessary use of YadifMod, fix the TFM field order, and decimate to 23.976fps. I'd also suggest trying her method after replacing the nnedi3 de-interlacing line with QTGMC to see how it compares. ie
Y = QTGMC()
That might prevent the problems you're trying to fix with QTGMC later, by de-interlacing with it in the first place, or it might not.

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

Clip = last

Y = nnedi3(field=-2)
A = Y.Selecteven()
B = Y.Selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)

QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.7, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")
Repair(QTGMC2, QTGMC3, 1)

E = last

F = Clip.TFM().TDecimate().MCDegrainSharp()

E.Trim(0,75)++\
F.Trim(76,2971)++\
E.Trim(2972, 0)

Resize8(640,480)
GradFun3()

Because Katie's method restricts the field matching it might potentially produce smoother motion at the expense of more "wobble" where there's already aliasing, although QTGMC does help with that. The AnimeIVTC method resulted in very slight jitter in a couple of places, which might be fixable by specifying the field order with AssumeTFF() or AssumeBFF() beforehand, or playing with some of it's options, or even by starting the AnimeIVTC section on a different frame, as the IVTC process works on frames in groups of five, but either way, I think outputting a constant 23.976fps from Avisynth is the way to go, even if it requires a bit more work to create the scripts.

Nothing was "interlaced" in the source as far as I could tell, aside from maybe the credits overlaid on hard telecined video (it's encoded that way), with "baked in" aliasing. And there's possibly hard telecined stuff overlaid on other hard telecined stuff, but without lining the fields up as such, making the field matching process harder. The actual film sections appear to be straight forward soft-telecine (repeat flags), except where there's credits and it's hard telecine. At least I assume... as the sample I used had credits throughout, but decoding while honouring the repeat flags and reversing the telecine with TIVTC should be all that's required for the film sections.

If the source was combinations of purely interlaced 29.97fps and film at 23.976fps, you could extract the timecodes, de-interlace the interlaced parts and use the timecodes for a VFR encode, but for the hard telecined parts if you remove the telecine you'll have 29.97fps with repeated frames and jerky motion, and if you decimate to 23.976fps, the original timecodes won't work. Unless I'm missing something....

VTS_02_1_samples.zip (https://www.sendspace.com/file/8smu6l) (75.3 MB)

By the way, I strongly suspect the correct SAR for the original DVD video is 10:11, not 8:9. I feel it in my bones. :)
Have a look here:
https://forum.doom9.org/showthread.php?p=1058927#post1058927
If you encode anamorphically, try 10:11, and when using an mpeg4 SAR, if you crop 8 pixels each side, or a total of 16 pixels from the sides, what you're left with will be exactly 4:3, so from there you can resize to any 4:3 dimensions.

hello_hello

13th May 2020, 12:59

Bluster and bullying can take you only so far.

You read that on neuron2's headstone, didn't you? :)

zapp7

13th May 2020, 17:14

By the way, I strongly suspect the correct SAR for the original DVD video is 10:11, not 8:9. I feel it in my bones. :)
Have a look here:
https://forum.doom9.org/showthread.php?p=1058927#post1058927
If you encode anamorphically, try 10:11, and when using an mpeg4 SAR, if you crop 8 pixels each side, or a total of 16 pixels from the sides, what you're left with will be exactly 4:3, so from there you can resize to any 4:3 dimensions.

Wow, this makes way more sense than the episode-specific cropping that I was doing. Many thanks!!

JoelHruska

13th May 2020, 18:10

I'm pretty sure the reason for the odd 24.66fps frame rate, is when you rip with MKV and open with StaxRip, it's using a method for indexing and opening the video (probably lsmash) that doesn't honour repeat flags by default. So it sees the interlaced/hard telecined sections as 29.97fps and the film sections as 23.976, but as Avisynth needs a constant frame rate, it decodes at the average frame rate. As the source is mostly 23.976fps, the average works out at 24.66fps. The interlaced parts are too slow and the film parts too fast, so the A/V sync is lost.

If you open a vob file and get a constant 29.97fps, it's because the repeat flags are being honoured so the interlaced sections are 29.97fps and the film sections are telecined to 29.97fps. That's where you should start, even though eventually you'll be reversing the soft telecine created via repeat flags on playback, as you'd do for the hard telecined parts (the telecine is encoded). If you've already ripped with MakeMKV, you can open the MKVs with TSMuxer and remux them as TS files. Using TS files as the source, I suspect StaxRip will index with DGIndex as it probably does for vob files, and the repeat flags will be honoured.

Katie's method bob de-interlaces the whole lot to 60fps, I assume to avoid problems where interlaced content has been overlaid on hard-telecined content, but you'd end up with lots of repeated frames in the film sections.

AnimeIVTC has a mode for dealing with interlaced stuff overlaid on telecined stuff. I used a sample from an earlier post and encoded with a script using AnimeIVTC for the opening credits/CGI part, and TIVTC for everything else. The output is a constant 23.976fps.

I didn't try to fix the remaining aliasing. I think it's encoded into the video anyway. I followed AnimeIVTC() with AddGrainC() as I thought it might look more natural.
The source is Stereodude's sample from here.
https://forum.doom9.org/showthread.php?p=1906994#post1906994
There's a link for the encodes at the end of this post.

A modified version of the Katie method to remove the unnecessary use of YadifMod, fix the TFM field order, and decimate to 23.976fps. I'd also suggest trying her method after replacing the nnedi3 de-interlacing line with QTGMC to see how it compares. ie
Y = QTGMC()
That might prevent the problems you're trying to fix with QTGMC later, by de-interlacing with it in the first place, or it might not.

Because Katie's method restricts the field matching it might potentially produce smoother motion at the expense of more "wobble" where there's already aliasing, although QTGMC does help with that. The AnimeIVTC method resulted in very slight jitter in a couple of places, which might be fixable by specifying the field order with AssumeTFF() or AssumeBFF() beforehand, or playing with some of it's options, or even by starting the AnimeIVTC section on a different frame, as the IVTC process works on frames in groups of five, but either way, I think outputting a constant 23.976fps from Avisynth is the way to go, even if it requires a bit more work to create the scripts.

Nothing was "interlaced" in the source as far as I could tell, aside from maybe the credits overlaid on hard telecined video (it's encoded that way), with "baked in" aliasing. And there's possibly hard telecined stuff overlaid on other hard telecined stuff, but without lining the fields up as such, making the field matching process harder. The actual film sections appear to be straight forward soft-telecine (repeat flags), except where there's credits and it's hard telecine. At least I assume... as the sample I used had credits throughout, but decoding while honouring the repeat flags and reversing the telecine with TIVTC should be all that's required for the film sections.

If the source was combinations of purely interlaced 29.97fps and film at 23.976fps, you could extract the timecodes, de-interlace the interlaced parts and use the timecodes for a VFR encode, but for the hard telecined parts if you remove the telecine you'll have 29.97fps with repeated frames and jerky motion, and if you decimate to 23.976fps, the original timecodes won't work. Unless I'm missing something....

VTS_02_1_samples.zip (https://www.sendspace.com/file/8smu6l) (75.3 MB)

By the way, I strongly suspect the correct SAR for the original DVD video is 10:11, not 8:9. I feel it in my bones. :)
Have a look here:
https://forum.doom9.org/showthread.php?p=1058927#post1058927
If you encode anamorphically, try 10:11, and when using an mpeg4 SAR, if you crop 8 pixels each side, or a total of 16 pixels from the sides, what you're left with will be exactly 4:3, so from there you can resize to any 4:3 dimensions.

Really interesting stuff here. I will launch some test encodes this afternoon to try some of these ideas.

TBH, part of the reason I continue to use QTGMC in the repair mode as I do is because every IVTC solution I've tested has created problems as well as solving them. Motion in the credits might be improved, but I'll see increased blur caused by out-of-order frames. Applying QTGMC to repair this sort of *occasional* damage seemed like a smart way to limit the impact.

I'm going to run through this later today. Also, while PoisonDeathRay is recommending the use of timecodes and scripting to only target specific frames with QTGMC (and I fully intend to also explore this), I am *also* testing the impact of upscaling before performing any of my traditional workflow.

I want to see what the following looks like:

DaVinci Resolve Studio used to change frame rate to 119.88fps
Upscale to 4x DVD resolution. From this point, apply various frame processing techniques.

I'm also doing a lossless upscale with PNG files to see what the outcome is. That'll be compared to an MP4 output from Topaz with the same base file to see comparative output once the video is reassembled.

Finally, I'll be tackling the advice you and others have given, to see what I can make of this timecode issue and bring MakeMKV source back into the running.

hello_hello

13th May 2020, 18:58

Wow, this makes way more sense than the episode-specific cropping that I was doing. Many thanks!!

Because there's very little black down the sides, and assuming I'm correct and the total display aspect ratio is a little wider than 4:3, you could reduce the side cropping to a total of 10 pixels and crop 2 pixels from the top and bottom. Upscaling to 720p that'd give you resizing of 976x720 (or at 1080p it'd be 1464x1080) with a display aspect ratio of roughly 1.356 instead of 1.33. At 720p it's only 16 pixels wider than 4:3, but it'd fill a bit more of a 16:9 display.

Using my script to show you the result of the calculations.

Cropping 16 pixels from the width.
CropResize(0,720, 8,0,-8,0, InSAR=10.0/11.0, Info=true)

https://i.postimg.cc/SYqBMfkS/1-333.jpg (https://postimg.cc/SYqBMfkS)

Cropping 10 pixels from the width & 4 from the height.
CropResize(0,720, 4,2,-6,-2, InSAR=10.0/11.0, Info=true)

https://i.postimg.cc/q6L9X6v8/1-356.jpg (https://postimg.cc/q6L9X6v8)

So if you cropped and resized the following way (976x720 or 1464x1080), the aspect error would only be off by one quarter of a pixel. Just a thought, but I'd probably take the "slightly wider than 4:3 option" if the sample posted earlier is typical, and assuming 10:11 is the correct SAR.

Crop(4,2,-6,-2)
SomeResizing(976, 720)

BTW, I haven't seen a script that includes it yet, but when upscaling it's probably a good idea to convert the colors from standard definition colorimetry to HD colorimetry so the upscaled version displays the same way. The easiest way to do it is with the ColorMatrix plugin. Just thought I'd mention it....

ColorMatrix(mode="Rec.601->Rec.709", clamp=0)

hello_hello

13th May 2020, 19:21

TBH, part of the reason I continue to use QTGMC in the repair mode as I do is because every IVTC solution I've tested has created problems as well as solving them. Motion in the credits might be improved, but I'll see increased blur caused by out-of-order frames. Applying QTGMC to repair this sort of *occasional* damage seemed like a smart way to limit the impact.

I had a couple of thoughts regarding the IVTC problems, and about to test them....

It occurred to me some of the cause of the wobble might be the de-interlaced clip, and maybe QTGMC should be creating that after TFM has field matched it, rather than de-interlacing the original clip. So it'd look something like:

DeintClip = TFM(PP=1).QTGMC().SelectEven()

TFM(Clip2=DeintClip).TDecimate()

The idea being that the de-interlaced clip is field matched by TFM but without it fixing any combing, and then it's de-interlaced by QTGMC. Hopefully that'll help reduce the wobble. It's a working theory.

I've just started playing around with making DeintClip quite bright so I can see what TFM is doing and fiddle with it's options accordingly. The white bits in the screenshot are the pixels taken from DeintClip. I'll play around and post back if anything comes of it.

https://i.postimg.cc/hfh0D7RN/Deint-Clip.jpg (https://postimg.cc/hfh0D7RN)

zapp7

13th May 2020, 20:03

BTW, I haven't seen a script that includes it yet, but when upscaling it's probably a good idea to convert the colors from standard definition colorimetry to HD colorimetry so the upscaled version displays the same way. The easiest way to do it is with the ColorMatrix plugin. Just thought I'd mention it....

ColorMatrix(mode="Rec.601->Rec.709", clamp=0)

I'm a total noob when it comes to all of this. What I'm doing currently, based on comments upthread, is converting to RGB24 with the PC.601 matrix, then feeding into Topaz. I then convert the Topaz output back to YV12 using the PC.601 matrix. How would the 709 fit in to that? Also, I'm upscaling to 2160p, if that makes a difference.

hello_hello

13th May 2020, 21:41

JoelHruska,
The QTGMC repair you ran was way too much for me. In the end I only ran a QTGMC repair on the first close-up shot of the space station. It's the one that really needed it. The rest of it is fairly wobble free now. You could try running QTGMC over it all to see if it fixes the baked in aliasing, but I'd be tempted to leave it as it's not over the top.

Three little functions to make life simpler.

# IVTC of the opening credits
function QIVTC(clip Video)
{
DeintClip = Video.TFM(Mode=6, PP=1, MI=20, Slow=2, CThresh=4).QTGMC().SelectEven()
Video.TFM(Mode=6, Clip2=DeintClip, MI=20, Slow=2, CThresh=4).TDecimate()
}

# To IVTC the film sections with a bit of denoising
function IVTC_Denoise(clip Video)
{
Video.TFM().TDecimate().MCDegrainSharp()
}

# QTGMC repair
function QTGMC_Repair(clip Video)
{
Video.QTGMC(Preset="Very Slow", InputType=1)
}

It's split into 4 sections. The first part is film, then the shot of the comet (to avoid turning it to mush), the CGI crap, then film again.
It looks better than the encodes I uploaded earlier (assuming you've seen them as it appears SendSpace is down).

Another VTS_02_1_sample.mkv (https://ufile.io/4dtvdk6e)

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

Trim(0,65).IVTC_Denoise() ++ \
Trim(66,736).TFM().TDecimate() ++ \
Trim(737,3000).QIVTC() ++ \
Trim(3001,0).IVTC_Denoise()

Trim(0,928) ++ \
Trim(929, 1258).QTGMC_Repair() ++ \
Trim(1259, 0)

Resize8(640,480)
GradFun3()

zapp7,
HD is normally converted to RGB for playback using rec.709, so if you're upscaling SD you'd normally convert from rec.601 to rec.709. It can be done without converting to RGB using the ColorMatrix plugin. Or you could convert from RGB using rec.709 when coming out of Topaz, I assume.
UHD is rec.2020, but it uses different color primaries and I don't think it's worth it. If you label the video stream by specifying rec.709 in the x264 command line for the color matrix, you'd probably be fine converting to rec.709 for UHD too. A modern player should read it and convert to RGB that way for display on a monitor/TV.

Mind you upscaling standard definition to UHD seems like over-kill to me. I rarely upscale SD to even 720p myself. I usually just clean it up if need be, re-encode, and let the player/TV do the upscaling.

JoelHruska

14th May 2020, 00:15

hello_hello,

*sigh*

Clearly I've got a lot of work to do before I can even attempt the scripts you've been kind enough to write. Multiple filters you call are, apparently, either invoked differently by StaxRip and the AviSynth+ installation it uses or must be downloaded and added. I've not had much luck doing this manually to-date.

There is no MCDegrainSharp. When I remove the call to that filter, StaxRip tells me "Splice - Video Frame Rate Doesn't Match." I'm not sure why. StaxRip uses AviSynth+, not AviSynth, and some of the calls are different, but I don't know which.

I continue to learn things that are not particularly helpful. Apparently, calling "ffms2" instead of "FFVideoSource" will work in StaxRip, but I'm not sure it changes anything in the other errors when I do.

*pause*

Nope. Doesn't.

At any rate, apologies. I'll have to do some more surgery on the OS, clearly, before I can test any of this.

EDIT #2: I'm getting exactly the same error on Handbrake that I am on MakeMKV. I've tried both a CFR and a VFR MKV file ingest. Probably has something to do with using StaxRip as a front-end or having AviSynth+ installed instead of AviSynth.

JoelHruska

14th May 2020, 00:18

PS -- All of the credits encodes you provided look truly phenomenal. I have to wait for the sun to go down before I can really proof dark-screen content on this system (Yeah, I know, goofy reason, but it's true), but they look gorgeous from what I can see.

poisondeathray

14th May 2020, 00:36

I'm a total noob when it comes to all of this. What I'm doing currently, based on comments upthread, is converting to RGB24 with the PC.601 matrix, then feeding into Topaz. I then convert the Topaz output back to YV12 using the PC.601 matrix. How would the 709 fit in to that? Also, I'm upscaling to 2160p, if that makes a difference.

PC.709 when going from your upscaled RGB images to YUV

(unless you were planning on using Rec2020. Many UHD productions still use 709 for legacy reasons)

zapp7

14th May 2020, 05:14

This MCDegrainSharp function is awesome! I applied it to the season 1 intro (which is horrible quality) and it looks way better. Is there any down side to applying this to the entire episode, aside from longer processing time?

poisondeathray

14th May 2020, 05:48

Apparently, calling "ffms2" instead of "FFVideoSource" will work in StaxRip, but I'm not sure it changes anything in the other errors when I do.

*pause*

Nope. Doesn't.

At any rate, apologies. I'll have to do some more surgery on the OS, clearly, before I can test any of this.

EDIT #2: I'm getting exactly the same error on Handbrake that I am on MakeMKV. I've tried both a CFR and a VFR MKV file ingest. Probably has something to do with using StaxRip as a front-end or having AviSynth+ installed instead of AviSynth.

The wrong framerate is probably from ffms2. You probably have jitter in the MKV timestamps

If you're using ffms2, FFVideoSource, you also need to set rffmode=1 to honor the repeat field flags .

But ffms2 is less reliable, less consistent than DGIndex, MPEG2Source. MPEG2Source is what everyone uses for DVD's. It's rock solid

To use DGIndex on a DVD MPEG2 in MKV, you'd have to demux it first. (DGSource will work on a MKV , but it requires a compatible Nvidia card and is not free)

StainlessS

14th May 2020, 07:38

There is no MCDegrainSharp. When I remove the call to that filter, StaxRip tells me "Splice - Video Frame Rate Doesn't Match."

Looks to me like you are saying that you dont have McDegrainSharp(), and when you comment it out you get wrong framerate warning.

I'm guessin' that what you have actually commented out are calls to IVTC_Denoise() which calls McDegrainSharp, if so then

# To IVTC the film sections with a bit of denoising
function IVTC_Denoise(clip Video)
{
Video.TFM().TDecimate().MCDegrainSharp()
}

If call to above commented out then no TDecimate and would indeed be mismatch in framerate. [commenting out MCDegrainSharp alone would not affect framerate]

Here is McDegrainSharp, try again without commenting out the filter.
https://forum.doom9.org/showthread.php?p=1737045#post1737045

Also, if above guesswork was correct, then maybe be a little more clear so as to avoid the guessin' bit.

EDIT:
Is there any down side to applying this to the entire episode, aside from longer processing time?
I use it on pretty much everything (although mostly with Frames=1[default=2], and quite often csharp=0.3 [default=0.6]), downside ?, me dont think so, me likee :)

EDIT: I've also just added this edit to McDegrainSharp() post,
EDIT: Perhaps McDegrainSharp() default bblur should also be raised a little for HD stuff.

hello_hello

14th May 2020, 09:29

Clearly I've got a lot of work to do before I can even attempt the scripts you've been kind enough to write. Multiple filters you call are, apparently, either invoked differently by StaxRip and the AviSynth+ installation it uses or must be downloaded and added. I've not had much luck doing this manually to-date.

First thing:
I discovered while writing these instructions that I messed up with the frame numbers. The CGI section should have ended on frame number 3714 (or 3175), not 3000. I'm not sure what happened there. Maybe I incorrectly used the frame number after IVTC, not before. I'll run another encode again later.

Also, if you compare the credits in the film section between my AnimeIVTC and KatieIVTC encodes, you see some of them "blink out" a little for the Katie encode whereas they're smoother in the other. Exactly the same IVTC was used for both though. I think the difference was starting the IVTC for the film section on a different frame number, which can effect the result, because the IVTC process works on groups of five frames. Theoretically, you'd keep the Trims to multiples of 5, or start a new Trim on a scene change, but I found adjusting the end/start frames up or down by a frame could change the result, especially in the CGI section (where the field order appears to change), so if there's jitter etc, that's something to try. Anyway...

MeGUI has an AVS cutter under the Tools menu. You create a script to open the source (MeGUI can do that from the File/Open menu) then save and open it with the AVS Cutter. It has a preview displaying the frame numbers, and you use it to set the start and end points for multiple instances of Trim. It looks a bit different to the way I've been doing it manually, but the result is the same.

This was the initial script:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

Using the AVS Cutter to add the first lot of Trims looks like this (MeGUI calls them "cuts").

__film = last
__t0 = __film.trim(0, 65)
__t1 = __film.trim(66, 735)
__t2 = __film.trim(736, 3714)
__t3 = __film.trim(3715, 6858)
__t0 ++ __t1 ++ __t2 ++ __t3

As those cuts are going to be used to apply IVTC, they'll change the frame count, so if you want to add a second lot of cuts you need to add IVTC first. You can just add basic IVTC to make it fast if you want to, if all you're concerned about is the correct frame numbers after IVTC. In fact you could probably just leave the trims as they are and follow them with a single IVTC.

TFM().TDecimate()

If you want to add additional filtering that doesn't cover the same frame ranges as the previous Trims, you'd open the script in the AVS Cutter again and add a second lot of Trims. When you're done, save them to the script and edit it to add the appropriate filtering to each (also removing the temporary IVTC). After the Trims you can add additional filtering that'll apply to the whole video. In my case I just added some resizing and dithering.

__film = last
__t0 = __film.trim(0, 65).IVTC_Denoise()
__t1 = __film.trim(66, 735).TFM().TDecimate()
__t2 = __film.trim(736, 3714).QIVTC()
__t3 = __film.trim(3715, 6858).IVTC_Denoise()
__t0 ++ __t1 ++ __t2 ++ __t3

# TFM().TDecimate() This line can now be removed

__film = last
__t0 = __film.trim(0,928)
__t1 = __film.trim(929, 1258).QTGMC_Repair()
__t2 = __film.trim(1259, 0)
__t0 ++ __t1 ++ __t2

Resize8(640,480)
GradFun3()

There is no MCDegrainSharp. When I remove the call to that filter, StaxRip tells me "Splice - Video Frame Rate Doesn't Match." I'm not sure why. StaxRip uses AviSynth+, not AviSynth, and some of the calls are different, but I don't know which.

As StainlessS said, you probably removed the IVTC with it, so the frame rates no longer match, and each spliced Trim must have the same frame rate.

Install AVisynth+ if you don't have it installed. You can put additional plugins and avsi functions in the "installed" Avisynth plugins folder. You could save the little functions from my earlier post with an avsi extension and put them in the "installed" Avisynth plugins folder too, if you wish. When a programs runs it's portable version of Avisynth, that folder is still checked, and any plugins/functions inside are automatically loaded, so you can just use them in scripts without having to load them first. And with Avisynth installed you can open scripts in other programs.
I assume StaxRip comes with it's own collection of plugins and functions which it loads in scripts as required, but using additional plugins/functions from the "installed" Avisynth plugins folder should work the way I described. I haven't used StaxRip for quite a while and I can't remember how to modify it's scripts or how easy that is.

I don't quite understand what you meant regarding the Handbrake error. Can Handbrake open Avisynth scripts these days?

Edit: Everything needs to be 64 bit or 32 bit, and I gather StaxRip is 64 bit these days? I'm using the 32 bit MeGUI, but there is a 64 bit version, and you'll probably have to install 64 bit Avisynth+ and use 64 bit plugins. You might be able to install 32 bit and 64 bit versions of Avisynth at the same time, and use 32 bit and 64 bit programs with them, each Avisynth installation with the appropriate 32 bit and 64 bit plugins. I'm not sure though. I'm still using XP. Someone else will no doubt be able to tell you.

I continue to learn things that are not particularly helpful. Apparently, calling "ffms2" instead of "FFVideoSource" will work in StaxRip, but I'm not sure it changes anything in the other errors when I do.

FFMS2 is FFVideoSource and FFAudioSource combined, but the audio is disabled by default.

The MCDeGrainSharp function can be found here:
https://forum.doom9.org/showthread.php?p=1855907#post1855907
I discovered it through StainlessS.

zapp7,
I use MCDeGrainSharp quite a bit, but only if there's light to medium noise, otherwise I'd use something else, but if you like the result, there's probably no downside.

I'll run another encode shortly with the correct Trim ranges to see what it looks like. That might fix the slight jitter I noticed somewhere towards the end of the CGI section.

StainlessS

14th May 2020, 09:46

HH, NOTE that McDegrainSharp updated some time ago, [updated compared to your posted link, I update that linked post, whenever it may happen].

Again McDegrainSharp() here:- https://forum.doom9.org/showthread.php?p=1737045#post1737045

hello_hello

14th May 2020, 10:20

StainlessS

14th May 2020, 10:56

hello_hello

14th May 2020, 11:14

I did not touch TrueMotion,
remind me, was it suggested to turn off for HD, (cant remember, nor remember where was suggested).
I only changed default blocksize, and added Limit stuff as per hi-lites in BLUE.

Also should maybe (guessin) raise bblur default 0.6 where HD (due to the 'grittiness' of HD compared to SD). (by how much, dont know)

Remember the screenshots in the TemporalDegrain2 thread where I compared MCDegrainSharp with and without TrueMotion? They were 720p.
https://forum.doom9.org/showthread.php?p=1855869#post1855869

Or the earlier comparisons, where QTGMC was looking more clever than TemporalDegrain2 because the latter had True Motion enabled?
https://forum.doom9.org/showthread.php?p=1855055#post1855055

They were the inspiration for my McDegrainSharp mod to disable it (the version I linked to).

Edit: I missed your edit before posting so now I'm repeating what you know. :)

I'm fairly sure it's disabled for resolutions above SD in SMDegrain and I think the default is disabled for MCTD, regardless of resolution, but it might pay to check. I can't remember when QTGMC enables it, but I'm sure the default is disabled. QTGMC mightn't have an argument to change it directly as such. It might be tied to another setting or a preset. I can't remember.

hello_hello

14th May 2020, 13:27

I don't know what I was thinking. A milder QTGMC repair over the whole CGI section is worth a tiny bit of lost detail, in my opinion.

A new sample, only requiring one set of Trims as the QTGMC repair is included in the QIVTC function. The QTGMC repair "may" look smoother due to the additional motion blur arguments.

function QIVTC(clip Video)
{
DeintClip = Video.TFM(Mode=6, PP=1, MI=20, Slow=2, CThresh=4).QTGMC().SelectEven()
Video.TFM(Mode=6, Clip2=DeintClip, MI=20, Slow=2, CThresh=4).TDecimate()
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

function IVTC_Denoise(clip Video)
{
Video.TFM().TDecimate().MCDegrainSharp()
}

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
Crop(8,0,-8,0)

Trim(0,65).IVTC_Denoise() ++ \
Trim(66,735).TFM().TDecimate() ++ \
Trim(736,3714).QIVTC() ++ \
Trim(3715,0).IVTC_Denoise()

Resize8(640,480)
GradFun3()

As an experiment, I modified the Katie method again to see how it'd compare if I gave it better de-interlaced clips to use. It's slow, as it has to de-interlace twice with QTGMC before repairing with QTGMC, whereas the more normal IVTC method only de-interlaces once, but where the CGI fades from one shot to another and the field order changes there are differences. I think the more standard IVTC generally looks a bit smoother through those transitions, but I've watched them so many times I can barely remember my name, so I've uploaded both encodes in case anyone else wants to look. The sort of transition I'm referring to is the fade from one shot to the next, a little bit after frame number 1800. The transition just after frame 2300 looks smoother for the Katie Mod encode, but I'm not sure that's always the case.

This is what I used as a modified version of the Katie method for IVTC in the CGI sections.

function QIVTC(clip Video)
{
A = Video.Tfm(field=0,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=1,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(field=0,mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

Both encodes can be download here (https://ufile.io/f/til02).

JoelHruska

14th May 2020, 14:15

Also, if above guesswork was correct, then maybe be a little more clear so as to avoid the guessin' bit.

I do not know how I could possibly have been more clear than "I removed the call to that filter." Previously, there was a call to that filter. I removed it.

Specifically, I changed the following line of code:

F = Clip.TFM().TDecimate().MCDegrainSharp()

to the following:

F=Clip.TFM().TDecimate()

This produces the video splice error.

I will quote my code changes in the future to avoid the ambiguity.

JoelHruska

14th May 2020, 14:56

Hello Hello,

Sorry for the ambiguity. References to Handbrake and MakeMKV in the same sentence should be read as a reference to using Handbrake or MakeMKV-derived source. I apologize for being unclear and will change how i refer to this in the future.

The reason I referenced both applications is because I was having the weird problem with MakeMKV playback speed in the first place. I wanted to make it clear that I tested both a CFR and a VFR ingestion of the file, and got the same "incorrect speed" error on both, in order to illustrate that the problems were unconnected.

JoelHruska

14th May 2020, 16:54

hello_hello

14th May 2020, 17:00

This produces the video splice error.

MCDegrainSharp doesn't touch the frame rate.

I have a theory....
I assume you removed MCDegrainSharp because of a "no such function" error. Once that error was removed you worked your way up the error chain.

The following line was used because in mode 7, TDecimate can't always decimate to the exact frame rate. It's generally so close it doesn't matter, but Avisynth needs exact, so AssumeFPS adjusts it to the correct frame rate.

TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)

This line will give you exactly the right frame rate (23.976), if the source frame rate is exactly right (29.97).

F = Clip.TFM().TDecimate()

If you opened the source with ffms2 without honouring the repeat flags, the source frame rate would be different, the "F" line above wouldn't have a frame rate of 23.976fps, instead it'd be relative to the source frame rate, while the earlier line would still be 23.976fps, because AssumeFPS told Avisynth to speed it up or slow it down to 23.976fps, regardless of the actual frame rate, so removing MCDegrainSharp fixed the "no function" error, only to expose the frame rate mismatch when splicing.

That's my best guess, after letting the last bit of info rattle around in my head for a while, but only because the additional details made me realise you were talking about a different script than the one I (and StainlessS) thought you were referring to.

FFMS2/FFVideoSource needs it's RFFMode argument enabled to honour repeat flags. That way it should decode the video ripped with MakeMKV at 29.97fps. ie

FFVideoSource("E:\VTS_02_1.mkv", RFFMode=1)

For LSmash, I think the latest version has it's repeat option enabled by default, but if not, or for older versions, it's Repeat=true instead.

DGIndex honours repeat flags, which is one of the reasons it's the recommended method for mpeg2 video, but it can't open MKVs. If you remux the video from the MKV as a ts file with TSMuxer as I suggested, StaxRip will probably open the ts file with DGIndex.

I'm still not clear on the Handbrake part. As far as I know it's CFR/VFR choice only effects the output. It should handle the input from VFR a MKV correctly. If you have the correct filters enabled (the Decomb filter I think, I don't use Handbrake) it should output a VFR if you tell it to.
In fact for Handbrake it's probably the best choice. If the input is VFR (combinations of interlaced/hard telecine and soft telecine) and the filtering is enabled, it'll decimate and de-interlace as normal, but if you tell it to output a constant frame rate of 29.97fps (for example), the interlaced parts will be okay, but the film parts will have lots of duplicate frames to increase the frame rate to 29.97fps, and that generally looks terrible because motion gets quite choppy. For hybrid sources it's generally better to let it output a VFR. If you want to use the output as the input for another program though....

hello_hello

14th May 2020, 17:23

Attempting to use MPEG2Source instead of FFVideoSource produces the following error in StaxRip: "Invalid D2V File, It is Empty!"

So, obviously I need to create a D2V file for each MKV if I want to use MPEG2Source.

When ffms2 indexes, it creates an index file with an ffindex extension. For Lsmash it's LWI and for DGIndex it's D2V. They're not interchangeable. StaxRip probably does it for you but DGIndex can't index MKVs.

After DGIndex has created an index file it's opened by DGDecode with mpeg2source. They're two separate programs but designed to work together. The index file is normally created by a GUI via the command line, as they can't run DGIndex via a script, although DGIndex has it's own GUI. The index files are normally opened directly as they contain the location of the source file:

mpeg2source("D:\VTS_02_1.d2v")

For ffms2 and lsmash it's an Avisynth plugin to both index and decode. If you open a file with either and there's no index file, they'll create one automatically.

FFVideoSource assumes the index file is in the same folder as the source. If it's not you can use the cache argument.

FFVideoSource("E:\VTS_02_1.mkv", RFFMode=1, cachefile="D:\SomeFolder\VTS_02_1.ffindex")

For LSmash you can do it either way these days. If you were to open an MKV it'd look for an lwi index file in the same folder. If it doesn't find one it'll index, unless you use it's cache argument, but you can also open index files directly as they contain the location of the source file.

LWLibavVideoSource("D:\VTS_02_1.lwi")

You need to remux the MKVs as TS files so StaxRip can open them instead, index with DGIndex and create a script to decode with mpeg2source, if you want to use DGIndex/DGDecode (unless you have the DVD files ripped and handy). I assume that's how StaxRip will open TS files, but it can't open MKVs with DGIndex/mpeg2source.

There's newer versions of DGIndex that can index MKVs and use a video card to decode rather than your CPU. I think DGIndexNV and DGDecNV are the Nvidia versions. I don't know a lot about them, but StaxRip probably supports the current ones. They're pretty cheap, but they're not free. http://rationalqm.us/dgdecnv/dgdecnv.html

videoh

14th May 2020, 18:02

I think DGIndexNV and DGDecNV are the Nvidia versions. I don't know a lot about them, but StaxRip probably supports the current ones. They're pretty cheap, but they're not free. http://rationalqm.us/dgdecnv/dgdecnv.html Yes, they will index and frame serve MKV. I believe they can be used in staxrip, but they are not bundled in staxrip.

JoelHruska

14th May 2020, 22:37

Hello_Hello,

Your assumption was accurate. I removed the call out to McDegrainSharp because of the error calling it created. I have managed to get your initial script (the one from back on Page 7) to run properly, though I had to remove the "resize8" command (don't have that filter installed at the moment, either, though I'll see about getting it).

I have a question for you.

Allowing for the fact that different people perceive quality differently, it has always been obvious that any effort to create the best possible version of Deep Space 9 or any other TV show would require scene-by-scene or frame-by-frame editing. I do not dispute this. Your version(s) of the credits are all better than what I'd produced.

My original goal, however, was not to create a scene-by-scene scripted guide to how to restore DS9. I do not believe I can afford the time required to painstakingly create a scene-by-scene idealized repair for every single episode -- and that assumes I had the skill to do so. I currently do not.

Here, then, is my question:

How would you approach the question of improving the DVD source if your goal was to improve as much of the underlying content as possible a relatively broad (though lightly applied) brush?

I realize this question may stand very much opposite to the way things are done, and I'm interested in exploring the type of repair you do at the scene-level and frame-level. This kind of low-level manipulation is incredibly interesting. But I do want to make sure that the project I've taken on strikes a balance between difficulty of setup (because I want people to be able to follow a tutorial), total processing time (obvious reason, if it hasn't been a concern for me personally to-date) and then the total difficulty of creating the tutorials in the first place, which lands on my shoulders.

I know that a lighter touch is always better on all but the most mucked-up video. Is it possible to find some light touches that can be run across the entire encode to repair various issues without causing the QTGMC impacts you dislike?

hello_hello

14th May 2020, 23:24

One thing before replying.... I looked at my last encodes again and I think the last modification of the Katie method for IVTC of the CGI section produces the smoothest result. I thought her method looked a bit awful originally (it did) but after experimenting and changing the dienterlaced clips it uses I think it's a little smoother through the fades between shots. The difference is very small, but that's probably the method I'd go with. Anyway....

The idea isn't so much to inspect every scene and apply scene specific filtering, it was to find a good way to IVTC the CGI stuff compared to the rest, assuming all the CGI has similar problems. I gather most episodes will predominantly be film, where standard IVTC can be applied, so if it was me, I would go through each episode (probably with MeGUI's AVS Cutter) and specify Trims to divide a script into film and CGI sections. I've done that sort of thing a lot myself. Once you've saved the Trims it's then easy to apply the same IVTC to each CGI section and the same IVTC to the film sections. You shouldn't need to create a new method of IVTC each time. That part's done now. That's the theory at least, having only seen one sample, but I think putting the extra time into creating the scripts in order to have a nice, clean CFR output will save you time and grief later.

I don't dislike using QTGMC to clean up video, I often run it in progressive mode for DVD sources to denoise, but I use it for that because often DVD video isn't very clean/stable, and as well as denoising, QTGMC can help with that. I'd run something like this over an entire video fairly regularly, even HD sources at times.

QTGMC(InputType=1, EzDenoise=1.5)

I just thought the two instances you were using to repair the CGI were a bit over the top and creating "mush" for want of a better word.

In the case of Deep Space Nine, the picture in the film sections is very stable, from what I've seen. There's a bit of noise, but the underlying picture is quite good. I did try running QTGMC on it at one stage rather than MCDegrainSharp, but the picture itself doesn't need stabilising, just some noise removal, so in this case I thought MCDegrainSharp looked better. Hopefully that makes sense. Aside from the horrid CGI, I haven't seen any issues in other sections yet. If the sample we've been playing with is any indication, standard IVTC and MCDegrainSharp will be fine for all the film sections.

In case it gets lost in the thread, here's the function to IVTC the CGI sections, assuming you want to split the scripts into CGI and film sections with Trim. It's the heavily modified Katie method, and a bit slow, but based on the sample, I think it'll be the best "one size fits all" method of dealing with the CGI parts.

function QIVTC(clip Video)
{
A = Video.Tfm(field=0,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=1,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(field=0,mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

hello_hello

15th May 2020, 00:36

Something important.... that I wasn't thinking about while testing..

If you do divide the script into film and CGI sections, the IVTC process converts five telecined frames into four progressive frames. When you IVTC a whole video that's fine, but when you divide the video up into sections the way I've been doing it for testing, you risk losing a frame in each section, unless you stick to multiples of five for each Trim. Here's a quick example:

This Trim contains 100 frames and at 29.97fps, it's duration is 4 sec 170ms.
Trim(0,99)
Apply IVTC and it now only contains 80 frames at 23.976fps but the duration is the same.
Trim(0,99).TFM()TDecimate()

This Trim contains 101 frames and it's duration is 4 sec 212ms.
Trim(0,100)
After IVTC it contains 80 frames and it's duration drops to 4 sec 170ms
Trim(0,100).TFM()TDecimate()

Sometimes you might create a frame, but with lots of trims the errors could add up and effect the audio sync.
I just checked the effect of the trims I added to the sample, as they weren't multiples of five, and they reduced the total duration by 90ms, which is unsatisfactory, so either stick to exact multiples of five for Trims used for IVTC, or do it by creating two clips instead.

First, add basic IVTC to a script, then add the trims.

TFM()TDecimate()

Trim(0,62) ++ \
Trim(63,715) ++ \
Trim(716,3682) ++ \
Trim(3683,0)

Replace the basic IVTC with two IVTC'd clips, then modify the Trims to splice the clips correctly.

SomeVideoSource()
Video = last
Clip1 = Video.QIVTC() # The CGI IVTC function is applied to the whole clip.
Clip2 = Video.IVTC_Denoise() # Standard IVTC and denoising function is applied to the whole clip.

Clip1.Trim(0,62) ++ \
Clip2.Trim(63,715) ++ \
Clip1.Trim(716,3682) ++ \
Clip2.Trim(3683,0)

It's a bit more work, but you can pick any frame range for the Trims without worrying about audio sync because IVTC has already been applied. I should have been thinking about that from the start... well I did it that way for the first few scripts I posted, then I got lost in the IVTC tests and forgot....but I really thought I should point this out.

JoelHruska

15th May 2020, 02:33

I do not yet understand the organizing principle behind this TV show's use of various frame rates and content sources. All of the statements below are 100% true and I can back them up with specific scenes.

1). Much of the CGI footage is encoded at 23.976 progressive in Season 6. In other places, however, the CGI footage is 29.97 interlaced. This appears to be more common in Season 7.

2). The non-CGI sections of the show are mostly -- but not entirely -- in 23.976 fps. There are at least two places in "Sacrifice of Angels," where footage bounces up to 29.97 fps and then back to 23.976 fps in the middle of shooting a conversation between characters.

3). Sometimes, space shots are interlaced footage. Sometimes they are not. Sometimes, they are 29.97 fps. Sometimes they are not.

4). The credit sequence of the show is the ugliest, *ugliest* part of it, by far. The original DVD looks like it was recorded off someone's VHS. Watch when the camera pans over the station, in particular: https://www.youtube.com/watch?v=xL8nPEmLKLE

Fixing the credits the way you did is amazing, if only because they were so awful before.

I can *show* you something of the effect I am trying, specifically, to create. I created both of the videos below (please set YT to 4K -- and yes, while video takes a hit, you can still see the effect I'm specifically going for, so I'm not worried about it).

https://www.youtube.com/watch?v=DzeENB9JKmE

That's an upscaled MKV file of DS9. I took a pre-existing encode of the show and ran it through Topaz VEAI Gaia-CG. That's the result. When the clip starts, watch the leading edge of the Galaxy ship and how smoothly it moves towards the viewer. Watch the level of noise on the ship hulls. (I mostly care about the first 10 seconds of the clip as far as a demonstration of what I'm looking for).

Now compare it to this video (set for 4K) -- particularly how smoothly the approaching edge of the Galaxy saucer moves, but also hull noise and the like.

https://www.youtube.com/watch?v=8DwfHADU0QY

The file I call "Rubicon" is much smoother than the upscaled MKV, even though it's still running at 23.976 fps. But the production process for this image quality is whack as *hell*, and it can't be good for the file.

Start with a Handbrake CFR rip. Encode in AviSynth with the QTGMC process I posted originally + daa3mod. Run the result through DaVinci Studio Resolve, increasing the framerate to 119.88 fps. Then, run the result of *that* encode through AviSynth again via "ChangeFPS(24000, 1001)" filter to change the frame rate back to 23.976 fps.

Then upscale in Topaz VEAI.

This is obviously rather tortured, and there's evidence. The very first frame of the video (you may need to rewind to see it) shows a distorted stardrive deflector shield on the approaching Galaxy-class ship for a single frame.

What I like about Rubicon, specifically, is how it removes noise, has excellent smooth motion (to my eye, anyway -- I am not saying it couldn't be better), reduces flicker on the hulls of the ships (check the Galaxy-class vessel in the upper left for window flicker), and antialiases the hulls. Is there a gentler way to achieve this kind of frame rate improvement via IVTC?

poisondeathray

15th May 2020, 04:43

Start with a Handbrake CFR rip. Encode in AviSynth with the QTGMC process I posted originally + daa3mod. Run the result through DaVinci Studio Resolve, increasing the framerate to 119.88 fps. Then, run the result of *that* encode through AviSynth again via "ChangeFPS(24000, 1001)" filter to change the frame rate back to 23.976 fps.

Then upscale in Topaz VEAI.

Sorry if I missed it, but what is the reason for Resolve ? Are you only using resolve for optical flow 23.976=>119.88 ? ChangeFPS is just dropping what you just interpolated (*5 , /5). But what is the purpose ?

If Resolve resamples all frames (does not keep the original set of frames in the interpolation), then you will get slightly different no matter where in the cycle you choose. If it keeps original frames, then it depends where in the cycle the you select (0,1,2,3,4) . You have more control over the offset with SelectEvery .

There might be some other options to get what you want, and avisynth has optical flow options if you needed it for some reason

This is obviously rather tortured, and there's evidence. The very first frame of the video (you may need to rewind to see it) shows a distorted stardrive deflector shield on the approaching Galaxy-class ship for a single frame.

You would expected contamination and blending, if there is no built in scene detection for some types of temporal filters and optical flow.

Or you can control the application of filters to specific sections to avoid those sorts of issues

What I like about Rubicon, specifically, is how it removes noise, has excellent smooth motion (to my eye, anyway -- I am not saying it couldn't be better), reduces flicker on the hulls of the ships (check the Galaxy-class vessel in the upper left for window flicker), and antialiases the hulls. Is there a gentler way to achieve this kind of frame rate improvement via IVTC?

Using IVTC only - will not smooth over the flicker from aliasing. But possibly IVTC + other filters you might get similar (or maybe better) results . Post the original clip and someone will give you some suggestions

JoelHruska

15th May 2020, 04:48

One more note. I tested the following script from back on page 7:

FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", rffmode=1)
#AssumeFPS(25)
Crop(8,0,-8,0)

Clip = last

Y = nnedi3(field=-2)
A = Y.Selecteven()
B = Y.Selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)

QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.7, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")
Repair(QTGMC2, QTGMC3, 1)

E = last

F = Clip.TFM().TDecimate().MCDegrainSharp()

E.Trim(0,75)++\
F.Trim(76,2971)++\
E.Trim(2972, 0)

GradFun3()

This did not work very well, as far as reducing judder or smoothing motion -- the judder was quite poor from frame to frame. Some of the CGI scenes were flawless, others were quite badly affected. I will obviously test the other code ideas -- just wanted to report back on the first one.

wonkey_monkey

15th May 2020, 07:54

2). The non-CGI sections of the show are mostly -- but not entirely -- in 23.976 fps. There are at least two places in "Sacrifice of Angels," where footage bounces up to 29.97 fps and then back to 23.976 fps in the middle of shooting a conversation between characters.

This may have been done to seamlessly alter the timing of the conversation and shorten a pause.

Katie Boundary

15th May 2020, 08:06

TBH that result speaks for itself, just drop it.

Any results that you might perceive of as wonky, are that way because that's exactly the way the DVD is encoded. My method perfectly preserves field-accuracy.

Hey all,

Just thought I'd post here and introduce myself. I'm Joel Hruska, from ExtremeTech, and I've been working to upscale Deep Space 9.

Hi Joel! World Domination Studios here. As you can see, most people here are hell-bent on making this process much more complicated than it really is. I see that you're still using Handbrake and Staxrip instead of Smartripper, DGindex, and AVIsynth, and you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?

Also, we did get your email about the retrograde field behavior. We apologize for not responding to it earlier. To use the trim command to get rid of it, the script would look something like this:

mpeg2source("arbitrary ds9 episode.d2v")

A=nnedi3(field=-2).selecteven()
B=nnedi3(field=-2).selectodd()
C=Tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=B,micmatching=0)
E=Tfm(field=1,mode=0,pp=5,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,micmatching=0)
F=Tfm(field=0,mode=0,pp=5,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,micmatching=0)

X=interleave(C,D)
Y=interleave(E,F)

Trim(X,0,82)+trim(Y,83,247)+trim(X,248,0)

...where frames 0 through 82 are in a "sane" order, frames 83 through 247 are the ones with the retrograde field behavior, and then frames 248 onward are clean again ("0" means "last frame" when used as the second parameter for trim). This will produce blended frames where the retrograde field behavior used to be, but it will at least preserve all of the original fields in their original locations in the timeline. If you're willing to sacrifice some orphaned fields to kill the blending, the script would look more like this:

mpeg2source("arbitrary ds9 episode.d2v")

A=nnedi3(field=-2).selecteven()
B=nnedi3(field=-2).selectodd()
C=Tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=B,micmatching=0)

X=interleave(C,D)
Y=separatefields().doubleweave().tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)

Trim(X,0,82)+trim(Y,83,247)+trim(X,248,0)

Just remember that your sole priority here is to deinterlace so that when you upscale the footage, you're not upscaling combed frames. Any and all other bullshit, like denoising or trying to convert to variable frame rate, is a distraction, and should be dealt with after deinterlacing - not as part of a single process that tries to do 20 different things at once and sucks at all of them.

https://i.pinimg.com/originals/90/ce/d6/90ced64d3a6a76cd714042e8f7276d8f.jpg

Groucho2004

15th May 2020, 08:59

you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI.You're funny.

Just in case you weren't joking and suggesting to stuff H.264 / H.265 into AVI - You appear to operate in a different reality than most of us.

wonkey_monkey

15th May 2020, 09:34

Any results that you might perceive of as wonky, are that way because that's exactly the way the DVD is encoded. My method perfectly preserves field-accuracy.

Your method failed spectacularly on the sample clip. Don't blame the DVD; you posted an untested script and falsely claimed that it worked.

JoelHruska, I really can't stress enough how little credence you should give to anything Katie Boundary says. This is not me being rude; it's simply that more often than not, they just don't know what they're talking about, often insist on using objectively worse methods for nothing other than arbitrary reasons, and simply refuse to listen to any dissenting advice.

Stereodude

15th May 2020, 16:12

So before feeding into Topaz, I do ConvertToRGB24(matrix="PC.601"), and after upscaling with Topaz I convert back with ConvertToYV12(matrix="PC.601"). Is that right?

1) Why would you want to limit yourself to 8-bit RGB color?

2) The built in functions have chroma positioning errors. Using z_ConvertFormat (http://avisynth.nl/index.php/Avsresize) would be preferred.

zapp7

15th May 2020, 16:40

1) Why would you want to limit yourself to 8-bit RGB color?

2) The built in functions have chroma positioning errors. Using z_ConvertFormat (http://avisynth.nl/index.php/Avsresize) would be preferred.

I'm a noob and still just learning about color spaces. It seems the original DVD footage is 8-bit (at least as reported by MediaInfo on the .M2V file), so is there any advantage to going higher?

huhn

15th May 2020, 16:51

a colorsapce doesn't have a bit deep but a DVD is 8 bit.
if you do math with pixel you can end up with fraction so you need more bit deep to store these result accurately or you will get side effect like banding.

the "colorspace" used on DVD is usually BT 601 and what was the name SMPTE 170M there was more i can't remember EBU or something. so you have to check first what it is the sample was matrix bt 601 and primaries SMPTE 170M.

JoelHruska

15th May 2020, 16:57

Your method failed spectacularly on the sample clip. Don't blame the DVD; you posted an untested script and falsely claimed that it worked.

JoelHruska, I really can't stress enough how little credence you should give to anything Katie Boundary says. This is not me being rude; it's simply that more often than not, they just don't know what they're talking about, often insist on using objectively worse methods for nothing other than arbitrary reasons, and simply refuse to listen to any dissenting advice.

I don't know who knows what. The attitude I'm taking is that the code will either prove to work or prove not to work. Right now, I haven't found an IVTC-based approach that could be run against the entire episode to both fix the random impact of interlacing and 29.97 fps but *doesn't* really obviously create other frame timing issues, like people walking backwards in the middle of movement. I'm trying to avoid having to program frame-by-frame comparisons, though I'm going to test this approach on Sacrifice of Angels over the weekend, because that's the episode I've worked on by far the most.

I have only barely started testing IVTC, however, and the functions written by hello_hello and others are far more complex than anything I tested, so I want to look at this carefully before I start saying what will or won't work. What I want, ideally, is a function that will fix the occasional instances of interlacing and 3:2 pulldown without overly changing anything else, and I'm *really* hoping for something I can run against every DS9 episode as a global script.

see that you're still using Handbrake and Staxrip instead of Smartripper, DGindex, and AVIsynth, and you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?

Tons. In fact, you should assume every single decision I've made has been made deliberately, in an attempt to solve a problem. That doesn't mean I'm solving it in the best way, but there's no part of my process that isn't deliberate.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.

Stereodude

15th May 2020, 17:04

I'm a noob and still just learning about color spaces. It seems the original DVD footage is 8-bit (at least as reported by MediaInfo on the .M2V file), so is there any advantage to going higher?
8-bit YUV isn't the same as 8-bit RGB. Converting back and forth between color spaces isn't a perfectly lossless operation in most cases. The color space conversion math is decimal based matrix math and doesn't map exactly to the 255 discrete steps 8-bit has. The more bits you keep through your process generally the better the final output will be. Additionally, you're upscaling the chroma in that conversion. RGB has full chroma resolution. DVD uses 4:2:0 YUV color, so the chroma resolution is 1/4 of the luma. The Luma is 720x480 and the chroma is 360x240. So you are upsampling the chroma in the step converting to RGB with a very rudimentary algorithm (compared to what Topaz uses).

I'm not familiar with Topaz and what color formats and bit depths it accepts. It may be better to feed it the 8-bit 4:2:0 YUV video directly (if it accepts it) and save the upscaled output from Topaz in >8-bit losslessly compressed video.

zapp7

15th May 2020, 17:41

I'm not familiar with Topaz and what color formats and bit depths it accepts. It may be better to feed it the 8-bit 4:2:0 YUV video directly (if it accepts it) and save the upscaled output from Topaz in >8-bit losslessly compressed video.

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.

The z_ConvertFormat is a bit over my head, but based on the doc, my guess is the command I need to go from YUV -> RGB is:

z_ConvertFormat(pixel_type="RGBPS",colorspace_op="470bg:601:470bg:full=>rgb:601:470bg:full")

After upscale, to go from RGB back to YUV with Rec.709:
z_ConvertFormat(pixel_type="YUV420PS",colorspace_op="rgb:601:470bg:full=>709:709:709:full")

I should also discuss the command I'm using to make a video out of the avs script (uses the override file workflow StereoDude helped develop). This video will then be fed to Topaz.

ffmpeg -i inputfile.avs -c:v libx264rgb -preset ultrafast -crf 0 -vf scale=-2:480,setsar=10/11 outputfile.mkv

I'm encoding instead of using the copy command because I want to set the SAR to 10:11. I read in the ffmpeg docs that -crf 0 is essentially lossless encoding. The libx264rgb is used since the format is rgb.

I'm sure I messed up a bunch and would appreciate any advice. :)

Stereodude

15th May 2020, 17:50

I don't know who knows what. The attitude I'm taking is that the code will either prove to work or prove not to work. Right now, I haven't found an IVTC-based approach that could be run against the entire episode to both fix the random impact of interlacing and 29.97 fps but *doesn't* really obviously create other frame timing issues, like people walking backwards in the middle of movement. I'm trying to avoid having to program frame-by-frame comparisons, though I'm going to test this approach on Sacrifice of Angels over the weekend, because that's the episode I've worked on by far the most.

I have only barely started testing IVTC, however, and the functions written by hello_hello and others are far more complex than anything I tested, so I want to look at this carefully before I start saying what will or won't work. What I want, ideally, is a function that will fix the occasional instances of interlacing and 3:2 pulldown without overly changing anything else, and I'm *really* hoping for something I can run against every DS9 episode as a global script.

Tons. In fact, you should assume every single decision I've made has been made deliberately, in an attempt to solve a problem. That doesn't mean I'm solving it in the best way, but there's no part of my process that isn't deliberate.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.
My personal opinion having played around with the first few episodes of Season 6 is that there's no compelling reason to get a VFR output. The show is for all intents and purposes is (24/1.001)fps. Just because the DVDs have p24 encoded sections with a soft pulldown applied and i60 encoded sections with a hard pulldown applied doesn't mean the output should be VFR.

There are some i60 text graphics inserted on top of it, fades in and out of black at the commercial breaks that are i60 (on top of the telecined p24), and the end credits are not 24fps. The show intro is i60 edited telecined film so you can't perfectly IVTC the intro where it fades between shots. However, no one is watching the show for the end credits or the intro (that's in every episode). You're not losing anything of importance with p24 (drop frame) output. You can blend the end credits to p24. So what if the star field behind the credits is blended? And the intro ends up with a few 5 frame segments that gets blended to p24 instead of decimated to p24. Hardly a loss. That leaves you with very few frames in an episode where an IVTC can't perfectly restore the underlying footage. The underlying footage is the concern not the titles that are faded in and out on top of it right?

I haven't found any actual i60 content (that should be turned into p60 by deinterlacing) in the show itself in the episodes I've looked at that. It's all p24 content that was edited as video (i60) after the p24 content was turned into i60 via a telecine.

The IVTC works. The problem is that the extra things done to the show when it was edited as i60 video occasionally break the IVTC pattern detection algorithms and require per episode override files if you want the best possible output to fix the few frames that it gets wrong per episode. That blows up your desire to be able to treat every episode the exact same and just run them through an entirely automated process.

You might also get marginally better output on the frames that get blended to p24 (instead of decimated to p24) if you waited to blend them until after they were upscaled by Topaz, but that further complicates the whole process.

Stereodude

15th May 2020, 18:11

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.
I will have to go back and look at the posts closer. I only skimmed some of the earlier posts. It may or may not be a problem. Or the fix may be worse than the problem.

The z_ConvertFormat is a bit over my head, but based on the doc, my guess is the command I need to go from YUV -> RGB is:

z_ConvertFormat(pixel_type="RGBPS",colorspace_op="470bg:601:470bg:full=>rgb:601:470bg:full")

After upscale, to go from RGB back to YUV with Rec.709:
z_ConvertFormat(pixel_type="YUV420PS",colorspace_op="rgb:601:470bg:full=>709:709:709:full")
Not quite...

z_ConvertFormat(pixel_type="RGBPS", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

&

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Also, I suspect that Topaz may not accept RGBPS video, but it might.

I should also discuss the command I'm using to make a video out of the avs script (uses the override file workflow StereoDude helped develop). This video will then be fed to Topaz.

ffmpeg -i inputfile.avs -c:v libx264rgb -preset ultrafast -crf 0 -vf scale=-2:480,setsar=10/11 outputfile.mkv

I'm encoding instead of using the copy command because I want to set the SAR to 10:11. I read in the ffmpeg docs that -crf 0 is essentially lossless encoding. The libx264rgb is used since the format is rgb.

I'm sure I messed up a bunch and would appreciate any advice. :)
I'm pretty sure this doesn't do what you think it does. You will still get a YUV encoded output.mkv file. It will not be RGB.

poisondeathray

15th May 2020, 18:15

poisondeathray

15th May 2020, 18:19

My personal opinion having played around with the first few episodes of Season 6 is that there's no compelling reason to get a VFR output. The show is for all intents and purposes is (24/1.001)fps. Just because the DVDs have p24 encoded sections with a soft pulldown applied and i60 encoded sections with a hard pulldown applied doesn't mean the output should be VFR.

At least one space battle definitely has 29.97p content. Someone posted in another forum a sample. How many scenes like this ? I never got into watching DS9 compared to the other series

https://forum.videohelp.com/threads/396652-Deinterlacing-Deep-Space-Nine-NTSC#post2578796

Some people reported "interlaced" . But is there actually any real interlaced content in this series ? By that I mean actual content with 59.94 different moments in time . Not some interlaced fade, not text overlays, or some orphan field.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content. VFR is the only way to have everything run at the correct frame rate.

My opinion #2 - One filter for everything is works good on some sections, but not so good on others. Definitely I would filter the problem aliasing sections selectively , otherwise you kill the details on 95% of the other sections. Counterproductive when upscaling. Analogy - A F1 car is great on a track, but sucks on city streets with potholes or offroading. A 4x4 is great for off roading but sucks on the track.

Katie Boundary

15th May 2020, 18:21

You're funny.

Just in case you weren't joking and suggesting to stuff H.264 / H.265 into AVI - You appear to operate in a different reality than most of us.

I never said anything about h.264 or h.265

1) Why would you want to limit yourself to 8-bit RGB color?

Because that's already more colors than the human eye and brain can distinguish

I'm *really* hoping for something I can run against every DS9 episode as a global script.

Unfortunately, if the episodes do contain retrograde field behavior as you say, then there's no script that you can copy and paste 176 times, changing nothing but the episode number each time, that will deliver anything close to a perfect result. The only methods that exist to fix retrograde field behavior are very destructive and should not be used on other kinds of interlaced video. You're going to have to choose a poison here: damaging perfectly salvageable content by blending it, damaging perfectly salvageable content by discarding fields, or going through every episode frame by frame to find the RFB and using Trim to fix it. RFB is a curse straight from the pits of Hell.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

If that's correct, then you could get rid of the vast majority of your problems by just making this a flat 59.94 FPS constant frame rate project. AVI works with everything and its only drawback that is relevant to this project is that it can't do VFR, which shouldn't even be a priority anyway.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content.

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

zapp7

15th May 2020, 18:26

I'm pretty sure this doesn't do what you think it does. You will still get a YUV encoded output.mkv file. It will not be RGB.

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?

There is a negligible difference between 8bit vs. 16bit in this workflow given this source. Chroma upsampling/down for 1 generation using internal avs function vs. zlib is pretty negligible too

I believe you, but I might as well use a better command if available, even if improvement is marginal.

Stereodude

15th May 2020, 18:28

Stereodude

15th May 2020, 18:30

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?
Maybe I misunderstood the FFMPEG help. I think you need to go to .tiff or .png to get the advantage poisondeathray is talking about.

zapp7

15th May 2020, 18:34

Just to be clear, that ffmpeg command is for the input video into Topaz. I'm saving the output from Topaz as .png images because it's lossless and the only video alternative is an antiquated H.263 encoding in mp4 format. We're all hoping they add better video options in the future.

Stereodude

15th May 2020, 18:41

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.
It looks like you are conflating two different things. You don't want to feed a lossy compressed intermediate file into Topaz or take a lossy output from Topaz. You would want to take the output from the IVTC script, losslessly compress it and feed that file into Topaz (presuming Topaz can't open the .avs). If you want to go the RGB route you would convert to RGB in the script, export the .avs to .tiff or .PNG, and run those through Topaz. Output from Topaz should be lossless also.

There are pros and cons to converting to RGB. If you want to color grade the episodes after upscaling, then going to RGB probably make sense (presuming Topaz can't accept >8-bit YUV video input). Otherwise, I'd probably steer clear unless you find that Topaz handles the YUV-RGB conversion poorly.

If you are planning to color grade it later those conversion lines I pasted are not the right ones to use. They are full to full. You would want to go full to limited in the first step. You will be compressing the colors range, so you definitely don't want to map it to 8-bit RGB.

z_ConvertFormat(pixel_type="RGBPS16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:l")

The step to come back is more complicated and depends on how you've color graded it (and where in the remaining steps you're going to color grade it). Which is why I haven't provided you the line because it's so highly variable on what is correct.

Just to be clear, that ffmpeg command is for the input video into Topaz. I'm saving the output from Topaz as .png images because it's lossless and the only video alternative is an antiquated H.263 encoding in mp4 format. We're all hoping they add better video options in the future.
Understood. Even if it real RGB, it's going to be limited to 8-bits, or 10-bits. If you're trying to get fancy with RGB you'd want to use Topaz on 16-bit .png or tiff files (for the input).

poisondeathray

15th May 2020, 18:50

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

Maybe "judder" would have been a better term

The reason some people in this thread should not do this is duplicate frames. Some filters (many temporal, "artemis" model in upscale AI) do not work as well with hardcoded duplicates. eg. If Joel wants to clean up aliasing sections with temporal antialiasing filters, duplicates will make it less effective

Another reason is why encode ~2x more frames if you don't have to? Upscale AI isn't the fastest, even if you have a monster hardware. You end up with larger filesizes too for intermediates and final versions

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?

It depends on what Enhance AI is using to decode.

You can check the levels before and after.

I believe you, but I might as well use a better command if available, even if improvement is marginal.

Fair enough, the main trade off there is the filesize for the intermediates.

If you zoom in 400% - you can barely see the difference. But the other things I mentioned - you can easily see the differences, those are worthwhile

Groucho2004

15th May 2020, 19:08

I never said anything about h.264 or h.265You did not, but your question to JoelHruska was:
...you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?And from his early posts you can clearly see that his target format is either H.264 or H.265.

Also, calling MKV an exotic container is simply ridiculous.

Groucho2004

15th May 2020, 19:14

1) Why would you want to limit yourself to 8-bit RGB color?
Because that's already more colors than the human eye and brain can distinguish
You must have a really crappy monitor. For example, I can clearly see differences (on my 10 bit Eizo screen) between 24 bit and 48 bit images, particularly when it comes to banding in gradients.

JoelHruska

15th May 2020, 20:08

At least one space battle definitely has 29.97p content. Someone posted in another forum a sample. How many scenes like this ? I never got into watching DS9 compared to the other series

https://forum.videohelp.com/threads/396652-Deinterlacing-Deep-Space-Nine-NTSC#post2578796

Some people reported "interlaced" . But is there actually any real interlaced content in this series ? By that I mean actual content with 59.94 different moments in time . Not some interlaced fade, not text overlays, or some orphan field.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content. VFR is the only way to have everything run at the correct frame rate.

My opinion #2 - One filter for everything is works good on some sections, but not so good on others. Definitely I would filter the problem aliasing sections selectively , otherwise you kill the details on 95% of the other sections. Counterproductive when upscaling. Analogy - A F1 car is great on a track, but sucks on city streets with potholes or offroading. A 4x4 is great for off roading but sucks on the track.

I don't know enough to want to make absolute pronouncements, yet, but those clips that were linked are from "Paradise Lost," (Season 4, Ep 12) which means there's just all sorts of fun stuff I haven't seen. At this point, I think the only sane thing to do is to assume that DS9 has basically everything in it, somewhere. We know that most of the show is in 23.976 fps, but even if that's 90-95%+, we also know that the last 4-5% is made of lots of other things. There are interlacing artifacts baked directly into source (I believe this is "hard" telecine?).

Deep Space 9 did not have a formula where non-CGI was 23.976 and CGI was 29.97 or 60i or what have you. It seems to have been more along the lines of "Mostly 23.976, but we do what we have to, in order to make the rest work."

I'm starting to appreciate why TNG, DS9, Stargate, and Babylon 5 are all called out as tricky in the AviSynth wiki.

JoelHruska

15th May 2020, 20:22

You did not, but your question to JoelHruska was:
And from his early posts you can clearly see that his target format is either H.264 or H.265.

Also, calling MKV an exotic container is simply ridiculous.

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."

I am completely agnostic about this entire process, and it's a good thing I am, because the applications I'm using have proven to be far more picky than I am. I output to H.264 because it's convenient. I had intended to test H.265 because it might offer better compression. I'll break out DIVX and .mpeg files if they turned out to be the best way to remaster the show.

Groucho2004

15th May 2020, 20:33

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."Pragmatic approach, I agree.

I'll break out DIVX and .mpeg files if they turned out to be the best way to remaster the show.Neither of those are very efficient nowadays. I'd say H.264 is your best bet. For low(ish) bitrates it seems that H.265 is more efficient. Just try them both at the bitrate you're comfortable with.

zapp7

15th May 2020, 20:45

Not quite...

z_ConvertFormat(pixel_type="RGBPS", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

&

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

So I converted to RGB using:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Saved that as 16 bit tiffs. Ran through Topaz. Then when I convert back to YV12 using:

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

it throws an error saying "clip must be planar", likely due to it being RGB48? How can this be tackled?

Also, since I'm saving as tiffs I'm not sure how to apply the SAR of 10:11. The result is that for 2160p, Topaz wants to produce a frame size of 3168x2160, instead of 2880x2160. What is the best point in this workflow to fix that, and how?

wonkey_monkey

15th May 2020, 21:02

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

The irony of you telling someone not to insist on doing something odd is off the charts.

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

It absolutely IS jerky to anyone with decent eyes.

It's not necessarily obvious - especially to someone who has grown up watching telecined video - but if you compare it to a video that's truly 24fps the difference is like night and day.

Stereodude

15th May 2020, 21:14

So I converted to RGB using:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Saved that as 16 bit tiffs. Ran through Topaz. Then when I convert back to YV12 using:

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

it throws an error saying "clip must be planar", likely due to it being RGB48? How can this be tackled?
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

What is your plan for the final output resolution? Regardless I would generally suggest that you should not convert to 8-bit YUV here. I would suggest keeping 16-bits (as above).

Use f3kdb to convert down to 10-bits for your HEVC compression.

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

Also, since I'm saving as tiffs I'm not sure how to apply the SAR of 10:11. The result is that for 2160p, Topaz wants to produce a frame size of 3168x2160, instead of 2880x2160. What is the best point in this workflow to fix that, and how?
You can't force a particular resolution for the output of Topaz? Does it have bars on the sides, or does it fill the entire frame?

If it fills the frame you will have to squeeze it horizontally. I would probably squeeze it in the z_ConvertFormat line.

like:
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

As to which scalar (resample_filter) you should use, I don't know. There will be lots of opinions.

Then it will have a SAR of 1:1 and you can encode it with x265 as 10-bit HEVC.

zapp7

15th May 2020, 22:17

ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

What is your plan for the final output resolution? Regardless I would generally suggest that you should not convert to 8-bit YUV here. I would suggest keeping 16-bits (as above).

Use f3kdb to convert down to 10-bits for your HEVC compression.

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

You can't force a particular resolution for the output of Topaz? Does it have bars on the sides, or does it fill the entire frame?

If it fills the frame you will have to squeeze it horizontally. I would probably squeeze it in the z_ConvertFormat line.

like:
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

As to which scalar (resample_filter) you should use, I don't know. There will be lots of opinions.

Then it will have a SAR of 1:1 and you can encode it with x265 as 10-bit HEVC.

Thanks! This all seems to work pretty well and I'm getting a 10 bit HEVC output now.

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.

JoelHruska

15th May 2020, 22:25

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

Not a "perceived need" as such. A desire to explore what outcomes looked like. I went to 119.88 to resolve VFR issues because that's one method of doing it. I also have tried various approaches with QTGMC to fix judder at 23.976 fps. I took the script you posted on Page 1 and combined it with my own repair formula to create a fairly good-looking 48 fps variant (if run through MakeMKV) or 60 fps (if used on a VOB).

Which one of these did I create? All of them. The only reason I didn't showcase far more footage than I did in my last article was because I ran out of time and the article had too many clips in it already. I rendered out six different clips from Sacrifice of Angels using native DVD, a generic upscaled MKV, and then my own work at 23.976 fps output, 48 fps, 60 fps, and 119.88 fps. Then I upscaled all of them in Gaia-CG and Gaia-HQ. (Obviously the DVD didn't get upscaled).

6 clips * 6 outputs * two upscale methods = Nearly 70 videos I created for that story. Turned out to be too much, so I dumped most of it. But I did the work.

The encode methodology I call "Rubicon" is not a 119.88 fps file. It's a 23.976 fps file that was temporarily pushed to 119.88 fps by DaVinci before being decimated back to 23.976 fps by AviSynth using ChangeFPS(24000,1001).

I'm not wedded to VFR. I'm wedded to nuking judder, preferably in a method I can apply across an entire episode at once. All of the container choices I have made have been attempts to work around problems I was having, not some attempt to use a particular file format or codec that I'm in love with.

Stereodude

15th May 2020, 22:26

Thanks! This all seems to work pretty well and I'm getting a 10 bit HEVC output now.

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.
Well, you can use AVSmeter to process the .avs as fast as possible vs. playing it real time presuming that's limiting factor.

I think you can also use ffmpeg to turn a .avs into a series of images but I've never done it and am not sure of the command syntax.

poisondeathray

15th May 2020, 22:39

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.

Probably

avsr64 script.avs

But it's slow, probably not much faster.

How are you loading the 16bit TIFF's ? If using ImageSource, don't forget pixel_type="RGB48", or they will load as 8bit

zapp7

15th May 2020, 22:59

How are you loading the 16bit TIFF's ? If using ImageSource, don't forget pixel_type="RGB48", or they will load as 8bit

Good catch! I've added that.

Seems like avsr64 is similar speed to AVSmeter, however I can run multiple instances of avsr64 and run several episodes in parallel!

Katie Boundary

16th May 2020, 03:37

Maybe "judder" would have been a better term

The reason some people in this thread should not do this is duplicate frames. Some filters (many temporal, "artemis" model in upscale AI) do not work as well with hardcoded duplicates. eg. If Joel wants to clean up aliasing sections with temporal antialiasing filters, duplicates will make it less effective

An interesting point, but the only frames that should require temporal filtering are the ones derived from orphaned fields to begin with.

Another reason is why encode ~2x more frames if you don't have to? Upscale AI isn't the fastest, even if you have a monster hardware. You end up with larger filesizes too for intermediates and final versions

That's actually a very good point. I'm not sure it justifies complicating the workflow to the edge of unusability, but rescaling the same frame 3x does seem like a huge waste of CPU cycles.

Also, calling MKV an exotic container is simply ridiculous.

It's not an industry standard like MP4 is, and it's not compatible with most video editing programs like AVI is. It's as exotic as a format can get without being some proprietary bullshit that you'd never see outside a digital camera.

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."

Well container format doesn't really have anything to do with how good the video looks. For working with the most software, AVI is pretty much the format that every program can import and every program can export. MPEG can be correctly interpreted by a very slightly broader range of hardware and software, but AVI can be exported by a vastly broader range of programs, like Premiere and Virtualdub; if you want to create a .mpg file, you'd have to create an AVI file or AVIsynth script or something and then load it into Tsunami MPEG Encoder, also known as TmpegEnc, to convert it to .mpg.

I'm not wedded to VFR. I'm wedded to nuking judder

That is unfortunate, because your approach of decimating an entire episode down to 23.976 FPS will create problems in 30 FPS or 60 FPS sections that are much worse than judder. Many years ago, I attempted to apply standard IVTC to an episode of War Planets, and noticed an odd skipping effect, as if decimate() was decimating the wrong frames. Looking at the source stream more closely, I discovered that the episodes were originally rendered at 25 frames per second, not 24, and were converted to NTSC by duplicating every fifth frame, not every fourth. I was decimating one frame per second too many, and it was already noticeable. Cutting 30 FPS content down to 24 fps kills 6 frames per second too many.

Judder in the 24 fps parts is the lesser of those two evils... by far.

If you think VFR is the lesser of all three evils, I won't try to dissuade you from that opinion, but conversion to VFR should be done as a separate step, after you've ripped, indexed, and deinterlaced in CFR. Worrying about judder when you're trying to deinterlace, or even just trying to get video from a VOB file into an AVS script, is like trying to decide what color your next Ferrari should be when your house is on fire.

poisondeathray

16th May 2020, 03:59

An interesting point, but the only frames that should require temporal filtering are the ones derived from orphaned fields to begin with.

"should" is not the same thing as reality.

The sad fact is this series has some garbage sections full with aliasing .

And I noticed some people were using some temporal filtering on "normal" sections as well

Whatever their reasons are, unique frames are going to be more conducive for their goals

That's actually a very good point. I'm not sure it justifies complicating the workflow to the edge of unusability, but rescaling the same frame 3x does seem like a huge waste of CPU cycles.

It's not for me; but for those people doing it, upscaling "x" amount AND having "y" times more frames is going to be worse on the CPU and GPU for both encoding and decoding

Katie Boundary

16th May 2020, 04:48

All right, PDR, you've made your case. Nonetheless, getting the video from the VOB file to the AVS script and then deinterlacing it can be more easily done in CFR. There's no benefit to bothering with VFR during those steps. Once that's done, it can be converted to VFR, and then temporal cleanup can be applied.

poisondeathray

16th May 2020, 05:39

Nonetheless, getting the video from the VOB file to the AVS script and then deinterlacing it can be more easily done in CFR. There's no benefit to bothering with VFR during those steps. Once that's done, it can be converted to VFR, and then temporal cleanup can be applied.

Yes, but "deinterlacing" should only be used on parts that need to be deinterlaced; otherwise you degrade >90% of the footage

It should be IVTCing parts that need to be IVTCed (>90%) , and additional filters for problem sections

Avisynth is technically CFR only anyways; it's the exported timecodes (timestamps) that make the final output VFR

If it's mostly 23.976p, and some 29.97p sections, it's easier. But some people said there were "interlaced" sections; if that' s true, (actual sections with 59.94 different moments in time represented, not fades, not text overlays ), it's harder because 2pass TFM using mode 5 won't pick those up

Katie Boundary

16th May 2020, 06:58

Yes, but "deinterlacing" should only be used on parts that need to be deinterlaced; otherwise you degrade >90% of the footage

It should be IVTCing parts that need to be IVTCed (>90%) , and additional filters for problem sections

IVTC is a form of deinterlacing. I think you mean that bob-deinterlacing should only be used on the parts that need to be bobbed, which is exactly how my method works.:cool:

If it's mostly 23.976p, and some 29.97p sections, it's easier. But some people said there were "interlaced" sections; if that' s true, (actual sections with 59.94 different moments in time represented, not fades, not text overlays ), it's harder because 2pass TFM using mode 5 won't pick those up

It's worse than that: supposedly, some sections suffer retrograde field behavior. We still haven't discussed how to handle that.

huhn

16th May 2020, 07:37

but if the source only has up to 30 fps parts then there is no part that needs bobbing so a bob deint will reduce the resolution in half.

zapp7

16th May 2020, 08:27

ryrynz

16th May 2020, 08:46

From the wiki.

Generally mode 1 will retain more detail, but repair less artefacts than modes 2,3. You may consider setting TR2 to a higher value (e.g. 2 or 3) when repairing progressive material.

I quite like the output of Fluxsmooth, Didée came up with a simple command that I use in real-time for noisy and shimmery video, see what you think..

FluxsmoothST(12,4).FluxsmoothT(4).Merge(Last,0.49)
feel free to play with the values, I haven't done any hardcore non real-time denoising so others might have some better options.

zapp7

16th May 2020, 09:25

I quite like the output of Fluxsmooth, Didée came up with a simple command that I use in real-time for noisy and shimmery video, see what you think..

FluxsmoothST(12,4).FluxsmoothT(4).Merge(Last,0.49)
feel free to play with the values, I haven't done any hardcore non real-time denoising so others might have some better options.

Thanks, I'll try it out. I've tested a few filters so far... MCDegrainSharp, TemporalDegrain2, QTGMC, but none really give me good results. One issue is that some will look good for the intro but not for film sections with characters.

Starting to feel burnout and diminishing returns at this point. I might just leave the footage as is, since the differences are subtle. (to my eyes, anyway)

Groucho2004

16th May 2020, 11:03

Seems like avsr64 is similar speed to AVSmeter, however I can run multiple instances of avsr64 and run several episodes in parallel!It should be about the same speed. As for multiple instances of AVSMeter - RTFM :sly: (there's a setting in AVSMeter.ini called "AllowOnlyOneInstance", set it to "0").

poisondeathray

16th May 2020, 15:43

IVTC is a form of deinterlacing. I think you mean that bob-deinterlacing should only be used on the parts that need to be bobbed, which is exactly how my method works.:cool:

No, that's not I mean.

Deinterlacing (any kind) primarily involves resizing a single field (+/- temporal filtering) . If you use it on progressive content, you lose ~1/2 the resolution of a full progressive frame. A full progressive frame consists of 2 fields from the same moment in time - they just need to be matched and weaved.

Double rate deinterlacing should only be used for the 59.94 sections

Single rate deinterlacing of any form (or QTGMC in progressive mode for temporal antialiasing) should only be used for problem sections, orphan fields

IVTC is used for progressive content. This means primarily field matching to get back the full progressive frames (+ decimation if 23.976p), + /- post processing for residual combing (e.g. orphan field, that single field can get deinterlaced) .

If you deinterlace, (single or double) with any method, you will degrade >90% of the content. The main distinction is interlaced vs. progressive content

It's worse than that: supposedly, some sections suffer retrograde field behavior. We still haven't discussed how to handle that.

Maybe someone should post a sample, and if verify if there really is 59.94 content. Some people say "interlaced" when it's really only combed for 3:2 hard telecine, or really only an orphan field.

poisondeathray

16th May 2020, 15:45

I'm experimenting with different denoisers for the first season of DS9. I have seen other people advocating for QTGMC as a denoiser run in progressive mode (InputType=1). Is QTGMC run in this fashion still destructive to the resolution of the footage?

QTGMC(InputType=1, Preset="Medium", EzDenoise=0.1)

Yes it's destructive

It might be appropriate for the 2-3% problem sections, but you will destroy details in >90% of the other scenes

I've tested a few filters so far... MCDegrainSharp, TemporalDegrain2, QTGMC, but none really give me good results. One issue is that some will look good for the intro but not for film sections with characters.

Exactly. You degrade the 90-95%. No amount of neural net processing can return the lost details

JoelHruska

16th May 2020, 18:25

Katie,

I feel like you still don't understand what I'm trying to do, here.

That is unfortunate, because your approach of decimating an entire episode down to 23.976 FPS will create problems in 30 FPS or 60 FPS sections that are much worse than judder.

So. Let me try to be even clearer.

I don't care about the output frame rate.
I don't care about the output container.
I don't care about the file extension.

Why, then, am I mucking around with 23.976 fps footage? Because it exists. Why have I converted to every other frame rate? To see what they look like. To see which gives the best output.

Why did I output to 119.88 fps? Because the AviSynth Wiki declares that 119.88 fps output is the most-compatible way to fix judder between 23.976 and 29.97 fps content.

I want good-looking video. Coming into this project, I had no idea what I needed to address in order to get it.

You keep talking about this project like I have declared: "I am going to create the best 23.976 fps encode of DS9 because it MUST BE IN 23.976.

I couldn't give two s**** and a whistle if the final output frame rate is 23.976, 29.97, 35, 42, 49, 60, or 119.88 fps. I don't really care if the output is 1fps, but we play back the content 23.976x faster than normal, if the end result was good-looking footage.

I will admit to one practical boundary: I just spent eight days waiting for an RTX 2080 to finish upscaling "Emissary" after I did a 119.88 fps conversion on it but *before* I applied any other filters or processing. I did it once, so I'd be able to compare the output to future attempts to process the footage, but there's no way I'm waiting four days to process each and every episode of the television show.

I'd like to keep the total processing time below the 24-hour mark per episode, keeping in mind that VEAI imposes a 10 hour encode time all of its own. The DaVinci Studio runs currently take about two hours, so my current process is a minimum of 12 hours of processing per episode.

But that's it. That's all I care about. I'm not wedded to 23.976 fps. I have worked on a 23.976 fps version because my intent is to publish instructions for upscaling the show depending, in part, on what frame rate you want to target. Do you want 23.976 fps? Then I want a workflow for it. I was creating simultaneous comparison shots for 23.976, 29.97, 48, 60, and 119.88 so that people could choose what worked best for them.

I'd like nothing more than to be able to tell people: "Use this process to arrive at the best final project frame rate that will look better than [list of options previously enumerated]. Don't bother screwing around with this other stuff. Just do this."

But I can't possibly tell people that if I don't know what all of the other speeds look like. If I can find one solution that looks better than anything else, I'll recommend it. If I can't, I'll recommend multiple solutions and show people the outputs so they can choose for themselves.

I haven't just explored one frame rate. I've explored like, six frame rates simultaneously. I don't care which one of them the final project uses. I care which one of them makes the final project look the best.

Katie Boundary

16th May 2020, 19:00

I just want to be clear. By retrograde field behavior, we're talking about content that, when run through separatefields().doubleweave() to simulate what it looks like on an NTSC TV, looks perfectly fine, aside from the fact that it's interlaced:

https://imgur.com/a/b62ndpK

But, when you look through it one field at a time using a dumb, temporally naive, double-rate bob-deinterlacer like bob(), it's obvious that the fields are NOT in the right order:

https://imgur.com/a/JqHkaHL

Joel, can you confirm that this kind of pathological content exists in DS9?

Deinterlacing (any kind) primarily involves resizing a single field (+/- temporal filtering) . If you use it on progressive content, you lose ~1/2 the resolution of a full progressive frame.

Deinterlacing is literally any process that converts interlaced content into progressive content. The specific form of deinterlacing that you're talking about is called bob-deinterlacing. It got that name from the dumb, double-rate, spatial-only filters, which would cause horizontal edges to bob up and down (as seen in the second gif that I posted above).

You keep talking about this project like I have declared: "I am going to create the best 23.976 fps encode of DS9 because it MUST BE IN 23.976.

No, I keep talking about this project like you're willing to pay any price and introduce any new problem to get rid of judder, and I just pointed out that decimating to 23.976 is the least ideal solution you've experimented with.

poisondeathray

16th May 2020, 19:11

Deinterlacing is literally any process that converts interlaced content into progressive content. The specific form of deinterlacing that you're talking about is called bob-deinterlacing. It got that name from the dumb, double-rate, spatial-only filters, which would cause horizontal edges to bob up and down (as seen in the second gif that I posted above).

Not necessarily spatial only; deinterlacing can use temporal algorithms (e.g. QTGMC does)

Interlaced content means each field represents a different moment in time. 59.94 different moments in time represented

The problem is you're mixing up Inverse telecine or pulldown removal with Deinterlacing. These are different things. IVTC is NOT a form of deinterlacing as you said, because it's used when underlying content is progressive.

So if you want to use your own definition, you are wrong ("Deinterlacing is literally any process that converts interlaced content into progressive content")

Since >90% of this is progressive content, you should not deinterlace as you suggested

Katie Boundary

16th May 2020, 19:15

Not necessarily spatial only; deinterlacing can use temporal algorithms (e.g. QTGMC does)

Bobbing got its name from what the the spatial-only versions did. This does NOT imply that all bob-deinterlacing is spatial-only, nor did I say that it did.

Interlaced content means each field represents a different moment in time. 59.94 different moments in time represented

No, interlaced content means that the content has interlacing in it. How that interlacing got there is irrelevant. If a VOB file is encoded as 24 fps with soft pulldown, and I index it in DGindex with "honor pulldown flags", bam, it's interlaced now.

The problem is you're mixing up Inverse telecine or pulldown removal with Deinterlacing. These are different things.

I didn't get them mixed up. I know they're different things. But one is a subset of the other.

So if you want to use your own definition, you are wrong

I'm sorry but your definitions are wrong. Deinterlacing is any process that converts interlaced content to progressive. Bobbing is a form of deinterlacing. Field-matching (which is the first step of IVTC) is also deinterlacing. Blur(1.0) is a form of deinterlacing.

poisondeathray

16th May 2020, 19:22

Bobbing got its name from what the the spatial-only versions did. This does NOt imply that all bob-deinterlacing is spatial-only.

I never said it was. I included temporal and gave an example

Deinterlacing is any process that converts interlaced content to progressive.

That's one definition

Bobbing is a form of deinterlacing.

Yes

Field-matching (which is the first step of IVTC) is also deinterlacing.

No. These are different by definition

Field matching , as the name suggests is for progressive content. 2 fields from the same moment in time

Interlaced content has only 1/2 the spatial resolution in motion. Single fields.

Blur(1.0) is a form of deinterlacing.

It can be

I didn't get them mixed up. I know they're different things. But one is a subset of the other.

No . You got them mixed up.

poisondeathray

16th May 2020, 19:27

No, interlaced content means that the content has interlacing in it.

Well that's an illuminating statement . Circular definition ? :D

How that interlacing got there is irrelevant. If a VOB file is encoded as 24 fps with soft pulldown, and I index it in DGindex with "honor pulldown flags", bam, it's interlaced now.

Of course it's relevant

24FPS with repeat field flags honored will show combing. It's not interlaced content. The underlying CONTENT does not change. It's reorganized as fields, that's all. The content is still progressive, it did not change

"interlacing" as a term - it's is a terrible description of what you see. You mean "combing"

Katie Boundary

16th May 2020, 19:31

https://en.wikipedia.org/wiki/Deinterlacing

Game over. I win.

poisondeathray

16th May 2020, 19:50

https://en.wikipedia.org/wiki/Deinterlacing

Game over. I win.

LOL What are you? like 13 ?

If you use that definition, you're wrong again

The film content is progressive. The underlying content does not change when you add pulldown or telecine . i.e you're not starting with interlaced content .

Interlaced content means 59.94 different moments in time represented (or even and odd scan lines come from different moments in time) . Do you have that ? No. You only have 23.976. It's progressive content.

Katie Boundary

16th May 2020, 20:22

You are literally arguing with wikipedia at this point. If you want to do that, I can't stop you, but you'll get very little support from anyone else.

poisondeathray

16th May 2020, 20:51

You are literally arguing with wikipedia at this point. If you want to do that, I can't stop you, but you'll get very little support from anyone else.

Because wikipedia is the ultimate reference and has no errors :D

I think that definition is not very useful, because it lumps everything under the umbrella of "deinterlacing"

Ok, put it another way - is it better to be more specific, more clear in communication, or vague ?

Since IVTC is a very specific subset if you use that definition, then why not use it when you mean it ?

You could call it "video processing" that's a pretty big umbrella

johnmeyer

16th May 2020, 21:27

IVTC is a form of deinterlacing.That is simply not true (i.e., wrong). IVTC involves the removal of redundant fields and, if the video is viewed directly without re-encoding, IVTC involves zero loss and produces zero artifacts.

By contrast, deinterlacing always involves degradation of the video because you must manufacture fields that were not there in the original. You do this either through duplication, blending, motion estimation, or some other technique.

I am bothering to post this in what has become a thread that is increasingly filled with strange statements because, unfortunately, a lot of people new to dealing with video make the mistake of conflating IVTC and deinterlacing and will often use a deinterlacer on telecined footage and then wonder why they get such horrible results.

So, deinterlacing and IVTC are two completely different things, and you cannot use the tool for one of them to solve the other one's problems. They are orthogonal.

One thing that was correctly stated in this thread by one of the doom9.org experts is that you must do IVTC before applying any temporal filter.

That is 100% correct, and should be obvious.

Why is it obvious? Because if you have a filter that looks at adjacent frames (or, sometimes, a bunch of nearby frames), the total lack of any change whatsoever between some nearby frames or fields, but not others, will completely blow up the algorithms.

As a corollary -- one I found out first-hand when I started encoding VCDs and SVCDs 20+ years ago -- is that encoders have the same problem as temporal filters when you try to encode telecined material without first doing IVTC. In fact, if you try to encode telecined footage, you will need integer multiple larger bitrates to get the same quality. I still remember spending half a day trying to encode the Elton John music video "I'm Still Standing" onto a VCD back in the late 90s. It was shot on film and the capture I made off satellite was telecined. I encoded that telecined video to SVCD and all I could see was "mosquito noise." It was awful. After lots of research and dozens of encodes, I discovered the IVTC built into TMPGEnc. I used it, and the results were a hundred times better (they still look good, even by today's SD standards).

JoelHruska

17th May 2020, 00:21

The whole reason I began using QTGMC in the first place -- the entire one -- is because I could call QTGMC, throw the entire file at it (with zero trimming) and what was output as a result was both substantially denoised and had all of the interlacing / 3:2 pulldown artifacts resolved. (I also got really nice horizontal AA, which is a nice feature).

Now, people here including PDR and H_H have pointed out that there's a substantial negative associated with QTGMC (well, more than one, but one big one in particular): It's destructive, and running a destructive filter over content that doesn't require a destructive filter is less-than-ideal.

The reason I have been attracted to using QTGMC is because, at least with DS9, it has done a remarkable job of fixing what I don't like visually, while introducing relatively few problems that catch my personal eye. Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing?

(I'm going to use trim and H_H's proposed method of treating CGI footage on Sacrifice of Angels, but that's a bigger project and I've got a move coming up, so probably not happening right away. I'm completely new to this kind of video editing, so.. slow going). In the meantime, I'm still exploring global approaches.

johnmeyer

17th May 2020, 01:32

videoh

17th May 2020, 01:34

Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing? If, if, if. Do you want to do things correctly and optimally, or not? Happy with half-assed?

And what in heaven's name is an "artifact of 3:2 pulldown"?

I'm completely new to this Got it.

johnmeyer

17th May 2020, 01:51

And what in heaven's name is an "artifact of 3:2 pulldown".24fps video or film exhibits "judder" to the human eye when the camera pans horizontally. The artifact is entirely made up in the brain and actually does not exist (it is a failure of our "persistence of vision").

Here is some 12 fps hand-held, hand-cranked silent film from 1929 that I transferred and which I left at the original frame rate. Since it is half the normal sound film rate, the judder is really bad. Watch the vertical edges of the buildings in the background and the pipe on the roof to understand what judder looks like:

1928 Backyard Kids Football in Oak Park, Illinois (https://www.youtube.com/watch?v=1YekFxBYKNM)

Repeating fields (or, less commonly, frames) so the material can be shown on 29.97 television simply makes this judder artifact even more pronounced.

It would actually be interesting to take a 24p scene with a horizontal pan and show it on a display natively, and then show the same thing on the same display with 3:2 pulldown added. I've never done that, but I'm pretty sure it would show the "artifact of 3:2 pulldown" and that artifact would simply look like slightly worse judder.

JoelHruska

17th May 2020, 02:16

PoisonDeathRay,

I tried several methods of interpolating up to 119.88 fps and then decimating back to 23.976. The best method I found was to use DaVinci.

The smoothest motion I have produced thus far was expanding to 119.88 fps using optical flow, then decimating back down with "ChangeFPS(24000,1001) in AviSynth. That produced better motion smoothing in 23.976 fps than the following methods:

1). Using ChangeFPS in AviSynth to shift to 119.88 fps and then using my QTGMC repair method.

2). Using QTGMC to interpolate additional frames into the original video (it may have been a dumb idea, but I still tried it) and then trying different combinations of SelectOdd and SelectEven. I ran an entire set of tests where I played around with various methods of increasing and decreasing frame rates to muck around with fixing VFR issues.

There are two ways to push DS9 to 119.88 fps using DaVinci Resolve Studio, and they produce different effects.

A). You can use Nearest Neighbor and 119.88 fps output settings. This will duplicate the frames in the DVD source file and results in the vastly expanded file size. Each frame is repeated 5x or 4x. It takes 4 days to upscale 1x DS9 episode if we go this route. Not much fun.

B). You can use Optical Flow and 119.88 fps output settings. This will result in hypersmoothed motion.

If you choose Option #B and then use "ChangeFPS(24000,1001) in AviSynth, however, the final output is remarkably good. Please don't think that when I say it's "Remarkably good" that I mean you won't find major differences if you inspect frame to frame. I'm sure you will. But the overall judder and motion are remarkably reduced considering I'm interpolating 80% of the frames that AviSynth then used to generate its 23.976fps output. It's closer to the kind of quality I'm trying to achieve than anything else I've found. That doesn't mean I don't think there's better. It's just gotten me the closest.

Btw:

The reason I use the following QTGMC preset is because I found the specific Input Type sequence of 2 followed by 3 to be most helpful in terms of repairing the content I wanted to repair. Other preset tests (1,1, 1,2, 1, 3, 2,1, 2, 2, 3,1, 3,2, 3,3) did not yield the same level of improvement.

But the rest of my QTGMC script? QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.2, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")

Other than the Input Types and the Sharpness, I chose the other presets hoping for maximum quality relative to original content. If there are different QTGMC presets that I should use, I'll try them.

videoh

17th May 2020, 02:26

24fps video or film exhibits "judder" You're calling judder an artifact? OK, fine, then you avoid this "artifact" by performing IVTC.

manono

17th May 2020, 02:28

24fps video or film exhibits "judder" to the human eye when the camera pans horizontally.
Sorry, but I chuckled when reading this. You do know, don't you, that the man you're instructing is responsible for both DGIndex and DGPulldown, and knows as much or more about the subject than any man alive?

Is that what JoelHruska means by 3:2 artifacts? The judder? If so, he won't lose it with QTGMC.

Do you have any evidence that silent video is meant to be played at 12fps? Was it written on the film reel or did it have accompanying notes? To me the motion looks way too slow and it should be 16fps or more.

johnmeyer

17th May 2020, 02:35

You're calling judder an artifact? OK, fine, then you avoid this "artifact" by performing IVTC.NOT TRUE! Look at the film I linked to. That artifact of the vertical lines breaking & then healing as the camera pans horizontally is called judder.

However, I completely agree that you are in safe territory by saying that judder is only caused by telecine, because any simple Google search will turn up lots of sites which say exactly that. However, as my film sample shows, that is not true. Put another way, if you display any 24p, 25p, or even 30p (somewhat rare) material at its native rate, you will see judder, with or without added pulldown (i.e., telecine).

Telecine just makes it worse.

[edit]As is often the case, feeding the correct search parameters to Google will give you the correct answers. Use these search terms and see what you get:

film 24p horizontal panning artifact (https://www.google.com/search?&sxsrf=ALeKk03EGmij3JhAUMC-A_DGslgbv6MMfg%3A1589679410401&ei=MpXAXpaEGI7S-gSU-L2ADg&q=film+24p+horizontal+panning+artifact&oq=film+24p+horizontal+panning+artifact&gs_lcp=CgZwc3ktYWIQAzIHCCEQChCgATIHCCEQChCgAToECAAQRzoFCCEQqwI6BQghEKABOgQIIRAKOggIIRAWEB0QHlCtkQNY1LcDYLW4A2gAcAF4AIABlQGIAdYTkgEEMTkuN5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwiW5cCk4bnpAhUOqZ4KHRR8D-AQ4dUDCAs&uact=5)

First hit links to this page:

https://www.red.com/red-101/camera-panning-speed

which says this:
Being able to control panning is important because moving too quickly can cause unpleasant visual artifacts. Objects or backgrounds may appear to flash across the screen in discrete jumps, for example, whenever the on-screen displacement is too great compared to the duration between frames. This is commonly referred to as strobing or "judder," and has happened since the early days of film.As I'm sure you know the Red Camera is an ultra-high end product used by pros, and things posted on this site are very professional and very credible.

JoelHruska

17th May 2020, 03:15

If, if, if. Do you want to do things correctly and optimally, or not? Happy with half-assed?

And what in heaven's name is an "artifact of 3:2 pulldown"?

Got it.

I have said from the beginning what I am creating. Namely: A tutorial on a method for upscaling and reprocessing DS9 that anyone with DVDs can follow. A tutorial that does not require individually hand-coded optimizations for each and every episode and relies on free software as much as possible. A "best-fit" method that still improves the results you'll get from throwing a basic MKV of the show into Topaz VEAI and watching the result.

That's the project I started off doing. Even if I decide to extend the amount of work I put into it later, that's the project I'm going to complete first. That's the project I committed to finishing for my readers (and not incidentally, my boss).

I have a deliverable I'm expected to complete on this project. So I'm working to complete it.

And what in heaven's name is an "artifact of 3:2 pulldown"?

I should not have called it an artifact. I have not known exactly what to call the short sequences of 3:2 pulldown and interlaced footage sometimes injected into the show. I know that at least *some* of the show footage was completed interlaced and apparently transferred to video in that format.

The strange thing of DS9 is that you can be watching a progressive CGI clip that suddenly has interlaced or 3:2 frames in it, but literally only 1-5 frames in a 90 second clip. You can DS9 and drop it into an upscaler without deinterlacing it at all, and what you get (except for the credits) will actually look really damn good, on the whole... but you'll get periodic horror-show frames, since the upscaler really loves to emphasize what an interlaced or 3:2 pulldown frame looks like. So it's those handful of bad frames that have to be fixed. So far, QTGMC is the filter I've found that's capable of fixing them, but that doesn't mean there aren't better solutions.

I have been calling these occasional problem frames "artifacts" to myself as shorthand. I know that's not the proper term.

zapp7

17th May 2020, 08:27

I have said from the beginning what I am creating. Namely: A tutorial on a method for upscaling and reprocessing DS9 that anyone with DVDs can follow. A tutorial that does not require individually hand-coded optimizations for each and every episode

Why is this such a big concern? If you're making avs script with trim on a per-episode basis, those only need to be created once and then can be shared with others. Same goes for override files. It's more work up front, for sure, but not really an impediment to creating scripts that other people can download and run to make their upscaled versions. Chir and I already have override files for the first 13 episodes.

JoelHruska

17th May 2020, 17:48

It's more work up front, for sure, but not really an impediment to creating scripts that other people can download and run to make their upscaled versions. Chir and I already have override files for the first 13 episodes.

Several parts to this answer.

1). I have been concerned about the amount of time it takes to create each episode's scripting. My assumption -- at least as a person who is literally learning about this process from this thread for the first time -- is that it will take me considerably longer than it might take the other experts in the thread.

2). AviSynth and the various associated tools and front ends are not simple. The more details people have to modify in script files (for example, to point to their own paths), the higher the intrinsic difficulty.

I realize that very few people will ever perform this kind of work, but the reason I started this project is because I was angry at the quality of the show on Netflix and Amazon Prime. I wanted people to be able to see it at something more fairly representative of the actual quality. So I'm hoping to create a project that is as approachable as possible, with as few dependencies and snags as possible. It may be possible to write sophisticated scripts or processing front-ends that get around this problem, but I do not know how to do it.

hello_hello

17th May 2020, 18:12

Time to reverse my position.....

Given poisondeathray posted a link to an interlaced section of an episode, I thought I'd give VFR encoding a try. I took the sample we've been playing with previously and appended the interlaced sample to the end of it. Then I ran an analysis pass with TIVTC to create the metrics files and had a look at the output. TFM breezed through the problematic credits at the start without requiring a tweak. I did have to set micmatching=0 to prevent a glitch in the interlaced section. It's smoother through the fades between shots than the methods I used previously. VFR straight out of Avisynth is probably the way to go.

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing). When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better. New samples below.

The script for the analysis pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Output="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=4, Hybrid=2, Output="D:\TDecimateMetrics.txt")

I added trims to the script after creating the timecodes to apply filtering to the various sections. There's also a version included without any filtering. TFM just doing it's thing.

Encoding script without filters:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Input="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimateMetrics.txt", tfmIn="D:\TFMMetrics.txt", mkvout="D:\TimeCodes.txt")

Resize8(640,480)

Encoding script with filters:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Input="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimateMetrics.txt", tfmIn="D:\TFMMetrics.txt", mkvout="D:\TimeCodes.txt")

__film = last
__t0 = __film.trim(0, 106).MCDegrainSharp()
__t1 = __film.trim(107, 590)
__t2 = __film.trim(591, 2978).QTGMC(InputType=1, Preset="very slow", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
__t3 = __film.trim(2979, 5556).QTGMC(InputType=1, EzDenoise=1)
__t4 = __film.trim(5557, 5979).QTGMC(InputType=1, Preset="very slow", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
__t0 ++ __t1 ++ __t2 ++ __t3 ++ __t4

Resize8(640,480)

# Tried QTGMC(InputType=1, EzDenoise=1) to see how it compares, but I still
# think I prefer MCDegrainSharp for this one even though QTGMC removes more noise

VFR Encodes, Take 2.zip (https://ufile.io/gsnvcw8r) (81.1 MB)

JoelHruska,
VFR isn't that hard to do, and this may be as close to a one size fits all method as it gets, and for the samples I've created at least, it looks the best. Avisynth outputs the average frame rate and the timecodes are used to make it variable at the encoding stage. You create the scripts for the analysis pass, run them, and TIVTC creates the metrics files. When I'm doing that sort of thing I use MeGUI, as it has an analysis pass mode and it's easy to add a bunch of jobs to the job queue. The trick is to never open the analysis scripts after running them, otherwise the metrics files are over-written and you have to run them again. Then it's just a matter of adding the encoding scripts to the job queue. If you specify unique names for the files for each episode, you can develop a system to do it pretty efficiently. I added Trims to one of the encodes above to apply the same filtering as before. It's easy to do, but it's not vital. You do have to decode with repeat flags being honoured though or it won't work.
Adding the timecodes to the x264 command line is done like this:

--tcfile-in "D:\Timecodes.txt"

PS. ChangeFPS(24000,1001) isn't the best way to decimate a 120fps source. It doesn't discriminate.
TDecimate(mode=7, rate = 24.0/1.001) tries to make sure the decimated frames are always duplicates.

hello_hello

17th May 2020, 18:19

NOT TRUE! Look at the film I linked to. That artifact of the vertical lines breaking & then healing as the camera pans horizontally is called judder.

Nope. That's called "tearing" and has nothing to do with telecine or telecine judder.

https://en.wikipedia.org/wiki/Screen_tearing

If it is tearing, you're possibly the only one seeing it, but your brain isn't making it up.

A player should reverse telecine for a progressive display just as it'd bob de-interlace (assuming hard-telecine), so you should end up with the same result as if it were progressive, with whole frames displaying for 2 refreshes and some for 3 on a 60Hz display. That's what causes the 24p/60Hz "judder".

Personally I'm not that sensitive to it, plus even when the frame rate is a multiple of the refresh rate, at much the same panning speed, objects can still "jitter" across the screen.

Film strobing or judder is something else again. I think that one relates to motion blur, or a lack of it, at certain low speeds of motion, so your brain can distinguish the discrete "steps" as objects move across the screen, but that's also probably about the same speed of motion where 24p/60Hz judder is noticeable.

Stereodude

18th May 2020, 01:56

24fps video or film exhibits "judder" to the human eye when the camera pans horizontally. The artifact is entirely made up in the brain and actually does not exist (it is a failure of our "persistence of vision").

Here is some 12 fps hand-held, hand-cranked silent film from 1929 that I transferred and which I left at the original frame rate. Since it is half the normal sound film rate, the judder is really bad. Watch the vertical edges of the buildings in the background and the pipe on the roof to understand what judder looks like:

1928 Backyard Kids Football in Oak Park, Illinois (https://www.youtube.com/watch?v=1YekFxBYKNM)

Repeating fields (or, less commonly, frames) so the material can be shown on 29.97 television simply makes this judder artifact even more pronounced.

It would actually be interesting to take a 24p scene with a horizontal pan and show it on a display natively, and then show the same thing on the same display with 3:2 pulldown added. I've never done that, but I'm pretty sure it would show the "artifact of 3:2 pulldown" and that artifact would simply look like slightly worse judder.
There's 3:2 cadence judder and then there's bad 24p content. The two are being incorrectly conflated. 24p can have very aggressive pans that look fine if the shots have an appropriate amount of motion blur in them. There's a reason for the 180 degree shutter angle rule. 24p will never look right when displayed at 60Hz with a 3:2 cadence. Thankfully decent modern TVs don't do that anymore. They display 24p at 120Hz with a 5:5 cadence.

Katie Boundary

18th May 2020, 02:47

Because wikipedia is the ultimate reference and has no errors :D

There are certain predictable weaknesses in Wikipedia. For example, pages on very obscure or specific subjects tend to be seen and edited by fewer people, and the people who do edit them tend to be obsessed fanboys who want to use Wikipedia as an indiscriminate data dump or a place to publish their original thoughts on subjects, so they can have quality-control issues. Other articles on politically charged topics tend to lean left because Wikipedia's notability and verifiability guidelines make it heavily dependent on what the mainstream media choose to report on, and the MSM are notoriously left-leaning. This latter problem is exacerbated by the fact that the few right-wing media sources that exist tend to be more open about their bias and less enthusiastic about fact-checking, while most of the left-wing ones do a very good job of pretending to be objective.

However, the article on deinterlacing is not one of these edge cases.

By contrast, deinterlacing always involves degradation of the video

Wrong. Bob-deinterlacing involves degradation of the video.

The reason I have been attracted to using QTGMC is because, at least with DS9, it has done a remarkable job of fixing what I don't like visually, while introducing relatively few problems that catch my personal eye. Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing?

There are lots of better filters. But most of them are focused on doing one specific thing, so you'd need to use all of them and in the right order, whereas QTGMC is a bunch of filters from a bunch of different people all tied together in a big knot.

For pure deinterlacing, you've already seen my custom method with NNEDI3, TFM, and Interleave. But this is purely for deinterlacing. It doesn't do any kind of cleanup or denoising.

For reducing general spatial noise, the best filter I've seen is TNLmeans, made by Tritical, the same author who gave us TFM. I can't give any advice on AA, as I've never much cared about it.

My biggest complaint with QTGMC is that it warps the frames in a half-assed attempt at performing a shitty form of motion interpolation. This is literally spending more CPU cycles to get a wrong result on purpose, AND it gets in the way of decimating the film sections back down to 24 fps.

Sorry, but I chuckled when reading this. You do know, don't you, that the man you're instructing is responsible for both DGIndex and DGPulldown, and knows as much or more about the subject than any man alive?

Videoh is Donald Graft?

Now that you mention it, that does feel like something I might have been told once before and then forgotten.

the upscaler really loves to emphasize what an interlaced or 3:2 pulldown frame looks like.... I have been calling these occasional problem frames "artifacts" to myself as shorthand. I know that's not the proper term.

It sounds like you're talking about ordinary interlaced or "combed" frames. Those aren't artifacts, but they really don't like being resized.

Katie Boundary

18th May 2020, 03:07

By the way, is there a way for us to send entire 1-2 GB VOB files to each other? I'd like to see the source footage myself but I can't buy my own copies yet because my Coronabucks haven't arrived in the mail.

huhn

18th May 2020, 07:17

there are without any doubt ways to use the internet to send files.
the practice is samples only for a good reason.

manono

18th May 2020, 08:45

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing).
Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example, you do this:

tdeintted = QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

That's from some included manual or other for TIVTC.

manono

18th May 2020, 08:53

By the way, is there a way for us to send entire 1-2 GB VOB files to each other?
If you use gmail at all, or other Google products, you should have access to your own Google Drive. I forget how large the files you can up/download. 5 GB, maybe? I hardly use it and I'm sure others know more about it.

hello_hello

18th May 2020, 09:12

JoelHruska,
Katie doesn't use QTGMC, and I suspect her latest advice comes without the realisation it has a progressive mode. QTGMC bob de-interlaces with nnedi3 by default, the same method she uses to create an interlaced clip, although oddly enough, after declaring QTGMC to be horrible, she's posting in the QTGMC thread looking for advice on how to get it to work again (https://forum.doom9.org/showthread.php?p=1912436#post1912436).

After bob de-interlacing with nnedi3, QTGMC applies various filters to reduce shimmering, denoise and sharpen etc. A process it gives you a fair amount of control over. Katie's method apparently involves de-interlacing, then maybe KLMeans for denoising, ignoring alaising, and running a bunch of unnamed filters in the correct order to produce a similar result. I'd like someone to tell me what motion-interpolation QTGMC does exactly, aside from interpolating the missing scanlines with nnedi3, which doesn't really qualify as the sort of motion interpolation people like johnmeyer and Katie accuse QTGMC of doing.

I get what Katie is trying to do with her method, so in that respect I'm not sure why there's an IVTC vs de-interlacing argument going on about it, although it appears to be a hard way to produce a "not as good" a result as TFM can do on it's own in VFR mode, assuming a VFR output is desired.

TFM is a field matcher and de-interlacer. For hybrid sources it field matches the telecined parts and de-interlaces the interlaced parts. It has a clip2 argument for taking pixels from a de-interlaced version of the clip to replace combed pixels instead of de-interlacing itself. The IVTC field matching process can sometimes result in combing when there's no perfect match (that could probably be caused by encoding artefacts amongst other things), in which case the newly created progressive frames need to be de-interlaced too. So for Katie's method every frame is bobbed to full height by nnedi3, TFM does it's thing and where there's combing it takes the pixels from the nnedi3 clip, the idea being that most of a progressive frame remains untouched and only the combed pixels are repaired, or TFM can be configured to use the entire frame from the nnedi3 clip when it detects combing, which would be the only time it could be accused of "bob deinterlacing" a telecined/progressive source.

In Katie's case she's using two instances of TFM, while forcing them to use specific matches, so in theory each TFM instance produces a 29.97fps clip where the telecined parts are field matched, combing is repaired, but without decimating the duplicate frames that result, and the interlaced parts are (bob?) de-interlaced. The two TFM instances combine to produce a 59.94fps clip. In a perfect world you'd have film sections where the progressive frames repeat in patterns of two and three duplicate frames, and the interlaced sections are bob de-interlaced for 59.94fps.

I guess the problem is, you can't just throw the same Katie method at every clip and expect a great result. If TFM is taking pixels to fix combing from a de-interlaced clip that has problems of it's own you could replace the combed sections with shimmering or additional aliasing, and I suspect even if the de-interlaced clip is clean, taking pixels from a second clip can still cause shimmering and aliasing if they don't match the rest of the frame exactly, so for the best result it pays to experiment. Even if the end result is perfect though, you've still got to decimate all the repeated frames in the film sections for a VFR, unless you want a 59.94fps clip, and given Katie consistently refers to doing just that, I'd be keen to learn how she does it.

I suspect the biggest obstacle to IVTCing and de-interlacing DS9 with TFM, is it's default de-interlacing doesn't work well with the CGI and causes lots of aliasing. Until I experimented with VFR encoding though, I'd concentrated on taking the combed pixels from a de-interlaced clip and hadn't considered how TFM's de-interlacing looks. After changing it's de-interlacing method, TFM's own de-interlacing looks quite good, and I suspect Katie's method might look better without the de-interlaced clip if different TFM de-interlacing was used. I only tried PP=5 for an encode, but I did briefly try simple blend de-interlacing (pp=2), and for the CGI sections that seemed to work well too.

Anyway, for DS9, based on my VFR examples, I wouldn't bother with any of that. Running an analysis pass, then letting TFM do it's thing using it's own de-interlacing (just not the default method) and outputting timecodes so the film sections are 23.976fps and the interlaced sections are 29.97fps has produced by far the best result of anything I've tried so far, without the need for a second deinterlaced clip and good enough not to require additional filtering to clean it up. Have a look at the samples I uploaded yesterday if you haven't already.

hello_hello

18th May 2020, 09:24

Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example, you do this:

tdeintted = QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

That's from some included manual or other for TIVTC.

That's what I've been doing for the samples I've uploaded, aside from the VFR encodes, although I suspect this might work better as it'd ensure the frames being de-interlaced by QTGMC are the same frames TFM wants to repair.

tdeintted = tfm(pp=1).QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

But even then, because the source clip contains lots of aliasing, depending how closely the de-interlaced clip matches the clip TFM wants to repair, replacing combed pixels with QTGMC pixels could possibly cause shimmering that wasn't there before.

For DS9, if you change TFM's own de-interlacing it looks like the result will be better. I haven't played with every imaginable combination of settings, but for the VFR encodes I uploaded yesterday, switching to pp=5 and not using a QTGMC deinterlaced clip has produced the best result so far, compared to the rest of the samples I've uploaded, and I played around quite a bit with TFM's settings for those encodes.
The VFR encodes are clean enough for repairing the CGI sections with QTGMC to be fairly optional (there's a tad less shimmering than when the source is being IVTC'd by my video card), and even though the CGI section is telecined, VFR rate appears to allow TFM to treat the fades from one shot to the next as purely interlaced (where hard telecine is overlaid on hard telecine with a different field order, I think), and those transitions are quite smooth. I guess my video card does the same, as the source looks smooth through those transitions too, without any combing.
The "problem" telecined CGI section is roughly from frame 570 to frame 3000, after decimation.

# timecode format v1
Assume 29.970030
# TDecimate v1.0.3.1 by tritical
# Mode 5 - Auto-generated mkv timecodes file
0,971,23.976024
992,1831,23.976024
1837,1840,23.976024
1846,2585,23.976024
2591,2998,23.976024

For a VFR encode, if you use a de-interlaced clip for TFM you have to use it for the analysis pass too (otherwise TDecimate complains the CRC no longer matches the analysis clip), so if it's a QTGMC de-interlaced clip, it'd slow down a normally fast analysis pass quite considerably.

zapp7

18th May 2020, 09:49

hello_hello

18th May 2020, 10:19

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample (https://ln2.sync.com/dl/5dd47a7d0/tanjt92v-e52sztfm-t9k5qzud-2tgukgp5)

If they used the same shot, or a similar one, in another episode that's much better quality, you could always swap them out. :)

videoh

18th May 2020, 16:21

There's 3:2 cadence judder and then there's bad 24p content. The two are being incorrectly conflated.
Quite right, Stereodude. Also, tearing and pulldown are being conflated. Actually, pure 3:2 pulldown can be easily removed so it is rather strange to call it an artifact.

Katie Boundary

18th May 2020, 17:46

Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example...

OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.

If you use gmail at all, or other Google products, you should have access to your own Google Drive. I forget how large the files you can up/download. 5 GB, maybe? I hardly use it and I'm sure others know more about it.

I do not use gmail because Google is evil.

There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?

TNLmeans or KNLmeans. Pure motherf***ing magic.

Should I re-send my friend request that you ignored? I feel so bad about Boris kicking you off my web site. You can come back any time and I will personally protect you. Your knight in shining armor!

Well the only reason I was on your website in the first place was to ask that ONE question about 48 -> 44.1 khz conversion and what the differences were between the quality settings. It's not like I was going to stick around one way or another. I will, however, continue to hawk DGindex as the #1 best way to get video content from a VOB file into AVIsynth :)

Joel, meet the legendary Donald Graft, also known as Neuron2. Back in the old days, he created he original Decomb plugin, which was the preferred deinterlacing tool used in the older versions of READFAG, written by the equally legendary Justin Emerson, also known as ErMaC. He also forked DVD2AVI (another program endorsed by early versions of READFAG) to create DGindex. Decomb has since been made obsolete by Tritical's TIVTC, but DGindex is still in use.

videoh

18th May 2020, 18:29

A little aside on tritical's stuff. It is heavily based on the Decomb design. He made it when I went AWOL on further development of Decomb due to my father's illness. tritical did a fantastic job on picking up the torch. I was actually the first person to develop IVTC based on field-matching plus decimation, although it wasn't my original idea. A poster at Avery Lee's VirtualDub forum first suggested field-matching plus decimation, but he was not a developer. I wish I could remember his name; sadly the forum is now off-line and I can't go look it up. If anybody remembers, please let me know.

Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.

The good old days!

JoelHruska

18th May 2020, 19:24

Hidden dependency: Probably of little interest to anyone except myself (since it's part of one of my own workflows), but in the name of sharing information:

My repair scripts that utilize QTGMC will not run against output from DaVinci Resolve Studio. Once the output has passed through DaVinci Resolve Studio and been converted to 119.88 fps, QTGMC's progressive repair mode will fail to engage (at least, when used in Staxrip). Instead, the application will hang. It will continue to do this, even if the framerate on the DRS output is decimated back to 23.976 fps using ChangeFPS(24000,1001). I am now attempting it on footage that used TDecimate instead of ChangeFPS and will report if it works.

You can still engage QTGMC in a standard preset mode, i.e., QTGMC("Medium") I am not sure if it is something specific about the flags I use or just the attempt to engage the progressive repair mode, but it doesn't work.

Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.

However, my Step #3 cannot be meaningfully applied in this workflow.

Not necessarily a problem or anything, just logging that it occurs.

JoelHruska

18th May 2020, 19:28

Zapp7,

Crapstation is a problem I haven't solved yet, either. Like you, I've been very unhappy to discover how bad the early footage is.

videoh

18th May 2020, 20:41

Probably of little interest to anyone except myself Correct, nobody cares.

hello_hello

18th May 2020, 20:41

OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.

Maybe if Katie wasn't ignoring so many people, she'd be aware of the full conversations. The statement was "TFM's default de-interlacing sucks balls for the CGI sections. When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better."

TFM is capable of six different types of deinterlacing.

Still, what do I know, so I ran another VFR encode using cthresh=2, mthresh=2 and Yadif as the de-interlaced clip.
Aside from the fact you've only got to look at the timecodes to see the low cthresh and mthresh settings cause TFM to think there's much more combing, and therefore to declare multiple slices of the film sections are interlaced instead of telecined, replacing the pixels detected as combed with Yadif pixels causes the same problems as TFMs default de-interlacing.
I could tell when TFM was going into interlaced mode for the Katie method, if for no other reason, because the stars would blink out.

cthresh=2, mthresh=2, Yadif de-interlacing
https://i.postimg.cc/rdWBzCns/1-katie.jpg (https://postimg.cc/rdWBzCns)

TFM pp=5
https://i.postimg.cc/5YVZzYT3/1-tfm.jpg (https://postimg.cc/5YVZzYT3)

cthresh=2, mthresh=2, Yadif de-interlacing
https://i.postimg.cc/CBs9sMVV/2-katie.jpg (https://postimg.cc/CBs9sMVV)

TFM pp=5
https://i.postimg.cc/30rVSMK0/2-tfm.jpg (https://postimg.cc/30rVSMK0)

manono,
I may have debunked my theory that this:
DeintClip = TFM(pp=1).Yadif()
Should be better than this:
DeintClip = Yadif()

When I compared the two using Katie's de-interlacing, it sometimes looked better, but other times it didn't, then for the last frame on a scene change where the next shot is mainly black space, I found this. I'm not sure I fully understand why, although it's an example of how much of a progressive frame can be unnecessarily de-interlaced with low cthresh and mthresh settings.

https://i.postimg.cc/XrTMVDgt/TFMThen-Deint.jpg (https://postimg.cc/XrTMVDgt)

Katie Boundary

18th May 2020, 20:57

The good old days!

Haha yes. When Smartripper was new, monitors were fullscreen, ATI still made "All-in-Wonder" cards, and we all got our anime music videos from Kazaa and Morpheus because Youtube didn't exist yet. I wasn't into video editing back then, but my mentor (Fuzzy Chickens) was, so I have been instructed in the Old Ways :)

Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.

Mpeg2dec3.dll is credited to "MarcFD, Nic, trbarry, Sh0dan and others", but was based on mpeg2dec2, which is credited to Mr. Barry alone. Mpeg2dec2 was in turn based on mpeg2dec, credited to "Dividee and others"

Katie Boundary

18th May 2020, 21:07

hello_hello

18th May 2020, 21:07

Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.

Wow, and that's easier than adding a few Trims to a script?
Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?
The whole idea of 120fps is it's a multiple of both 24 and 60, so even though there's lots of repeated frames the interlaced sections still play at their original speed, as do the film sections.

Thinking about it, TDecmiate(mode=7, rate = 23.976) mightn't be the best tool for the job. The output will have a constant frame rate, so to achieve 23.976fps you might end up with the interlaced and film sections effectively playing at the wrong speeds. If the Avisynth output was 23.976fps though, it should be fine.
Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?

After trying some VFR encodes myself, using TFM for a VFR encode from Avisynth seems like the best method anyway, so I don't know why you wouldn't want to try it.

hello_hello

18th May 2020, 21:17

h_h, there's something very wrong with your settings or your script. That looks straight YADIFed, with no TFM at all (or like TFM is set to one of its "dumb" deinterlacing modes, which results in the same thing).

There was nothing wrong with my settings. I ran a VFR encode using the settings you recommended for TFM.

Crop(8,0,-8,0, Align=true)
DeintClip = Yadif()
TFM(Input="TFMMetrics.txt", clip2=DeintClip, cthresh=2, mthresh=2, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)

Not every frame looks so bad, just the ones where TFM was taking a lot of pixels from Yadif.

The "good" screenshots used the following script:

Crop(8,0,-8,0, Align=true)
TFM(Input="TFMMetrics.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)

Katie Boundary

18th May 2020, 21:25

It shouldn't be taking ANY pixels from YADIF except in the parts that are interlaced, in frames that are interlaced. I've never seen results like that from my own scripts.

hello_hello

18th May 2020, 21:39

Your cthresh and mthresh settings are so low it thinks large chinks are interlaced, although I assume you actually mean combed.

I don't need to get into a debate about it. It's pointless anyway. I can still remember when you argued the evils of IVTC with TFM, or deinterlacing with NNEDI3 and Yadif.

I've run quite a few test encodes using the dual TFM method you love, a heavily modified version of it using two QTGMC de-interlaced clips to make it not suck, and encodes using a single instance of TFM with QTGMC de-interlacing that was virtually as good (for when a constant 23.976 frame rate was required), but after trying VFR encoding when poisondeathray linked to a sample of a purely interlaced CGI section, the above VFR output using PP=5 has easily produced the best result.
I've linked to sample encodes in quite a few posts if you want to look for yourself, assuming you can play MKVs like the rest of us now, but please don't just tell me I'm wrong or I've used the wrong settings if you think you can do better, because it's easy to upload a sample of your own to prove it.

There's links for the sample source files I used earlier in the thread somewhere.
Edit: I think this was the first one:
https://forum.doom9.org/showthread.php?p=1906994#post1906994
Second one here:
https://forum.doom9.org/showthread.php?p=1912067#post1912067

I joined them like this for encoding:

A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

VFR samples here:
https://forum.doom9.org/showthread.php?p=1912388#post1912388
CFR sample here:
https://forum.doom9.org/showthread.php?p=1911841#post1911841

Katie Boundary

18th May 2020, 22:08

I never argued that those were "evil". Lies like that are why you're on my ignore list.

There is another argument to be made in favor of pp=5, however. My current deinterlacing algorithm heavily favors field-accuracy, the representation of the original fields and frames as accurately as possible in the newly deinterlaced frames, and the minimization of blended frames and residual interlacing. Why? because the obsessive-compulsive demons who whisper in my ear demanded it. It can't fully eliminate visible interlacing without a high false positive rate, which it conceals pretty well with temporally-aware bobbers (Yadifmod2). But there are types of content where that's not the right tradeoff to make, and blending is better than an autistically literal representation of the original content. The most obvious is the aforementioned retrograde field behavior, but fades, credit animations, and a few other effects are harmed less by blending than the rest of the frame is by bobbing. There are also times when the last field of one clip and the first field of the following clip will be interlaced together and encoded as a single frame, which causes their chroma planes to merge; in such cases, each field benefits from being matched and blended with the nearest field from its own clip rather than bobbed. This doesn't mean that cthresh and mthresh should be set higher, however. You can't do that without unacceptable amounts of visible interlacing getting through.

JoelHruska

18th May 2020, 22:30

Wow, and that's easier than adding a few Trims to a script?

Testbeds do 100% of the work. I set up the encode, walk away, come back 4-8 hours (or 4-8 days) later and check the output. The Trim method is something I need to learn from scratch, using tools I have not found easy to work with. I've been experimenting with the code you've posted and trying to figure out how this different approach to editing works, while continuing some evaluation on my own previously-developed workflow.

Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?

To see what the outcome would be. Exactly the same reason that I have two encodes running downstairs since last night, converting the 8-day Emissary encode I did -- one with ChangeFPS(24000,1001) and one with the TDecimate command you suggested, to see which handles the content better, and how smooth the output is.

I wanted to compare the outputs against what native 23.976 fps video looked like, versus video run through progressive repair mode in QTGMC, etc, etc.

Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?

Well, keep in mind I reversed the workflow in this manner to test the impact of preserving detail for longer. I wanted to compare the visual impact of running QTGMC later in the workflow, which meant keeping the workflow identical except for running QTGMC later in the process. Unfortunately, this proved impossible, because my script won't execute after running DaVinci Studio Resolve.

EDIT: Using TDecimate instead of ChangeFPS doesn't change whether or not my repair script will run after DaVinci RS. It still will not. 2

I will not be retaining this workflow. I'm not spending 4 days of upscaling per episode.

But as for why I'd try it in the first place? Because I am new at this, and do not know what will work and what will not, and have been willing to burn CPU cycles on things I was pretty sure wouldn't work to see what they could teach me. As one simple example: I noticed using QTGMC in default mode would create an interpolated frame in-between each "regular" frame. After I noticed this, I ran multiple tests to look at the difference between SelectOdd and SelectEven. At one point, I double-ran QTGMC and then tried "SelectOdd,SelectOdd," "SelectOdd,SelectEven", "SelectEven, SelectEven", etc. Why? Because I wanted to understand what kind of output would be produced.

All of them were garbage, but some of them were more garbage than others.

"Test it and see what it looks like," has been my guiding philosophy on this endeavor.

wonkey_monkey

18th May 2020, 23:04

Correct, nobody cares.

Why didn't you just not say anything?

hello_hello

18th May 2020, 23:07

I never argued that those were "evil". Lies like that are why you're on my ignore list.

I'm on your ignore list because you're a child.

Obviously I'm not on it now though, so you're lying about that, but given you're claiming I'm on it, you couldn't have read my posts, which means you couldn't have replied, and therefore I don't have to bother reading it.

I said you "argued the evils". How many times have you accused others of not being able to understand plain English?

So, with almost no noticeable difference between AVIsynth's built-in Bob() filter and an actual proper field match, why should I believe that there's a huge difference between Bob() and more computationally intensive, easier-to-screw-up bob-deinterlacers like Yadif and NNEDI?

hello_hello

18th May 2020, 23:33

I wanted to compare the outputs against what native 23.976 fps video looked like, versus video run through progressive repair mode in QTGMC, etc, etc.

Keep in mind temporal filters need to see motion. If you up the frame rate there's going to be repeated frames, and those frames will be identical, so any noise will become static and noise removal won't work as well. That sort of thing. Even if they're interpolated frames, I think you'd be better of running any filtering first.

manono

18th May 2020, 23:38

Once the output has passed through DaVinci Resolve Studio and been converted to 119.88 fps, QTGMC's progressive repair mode will fail to engage (at least, when used in Staxrip).
All you use DaVinci for is to interpolate to 119.88fps? You can do that in AviSynth. I actually like that method of getting interlaced 29.97fps to 23.976 with a minimum of stuttering. But, I also agree that getting back to 23.976fps using ChangeFPS isn't optimal.

And I would also probably suggest the interpolate route only with the video portions and IVTC the film portions. But if you're looking for a set-it-and-forget-it method where you don't have to go through the episodes looking for the GCI parts, I suppose that'll do.

videoh

19th May 2020, 00:04

Why didn't you just not say anything? Why didn't you mind your own business?

videoh

19th May 2020, 00:09

"Test it and see what it looks like," has been my guiding philosophy on this endeavor. You'll get much better results, and much faster, when starting from valid theoretical considerations, rather than groping in the dark. But hey, even a blind pig sometimes finds a truffle.

JoelHruska

19th May 2020, 00:50

You'll get much better results, and much faster, when starting from valid theoretical considerations, rather than groping in the dark. But hey, even a blind pig sometimes finds a truffle.

Indeed. But when one does not know which theoretical considerations are valid and which are not, groping in the dark -- and a little use of the Mark I Eyeball -- will teach you things regardless. The fact that I had never seen what an interpolated frame looked like until I saw one as a result of my first QTGMC run didn't prevent me from recognizing what I was looking at.

Some of the work I have done has been for no other purpose than to show me things about how video behaved when manipulated. I have experimented with accelerating and decelerating frame rates. I ran Handbrake rips against MakeMKV rips so I could see the subtle difference in how they handled color. I learned about pixel aspect ratios from noticing differences in output between MakeMKV and Handbrake. When I wanted to see the impact of incorrect deinterlacing filters on content, I ran deliberately-incorrect deinterlacing filters and looked at the results.

JoelHruska

19th May 2020, 02:06

Time to reverse my position.....

Given poisondeathray posted a link to an interlaced section of an episode, I thought I'd give VFR encoding a try. I took the sample we've been playing with previously and appended the interlaced sample to the end of it. Then I ran an analysis pass with TIVTC to create the metrics files and had a look at the output. TFM breezed through the problematic credits at the start without requiring a tweak. I did have to set micmatching=0 to prevent a glitch in the interlaced section. It's smoother through the fades between shots than the methods I used previously. VFR straight out of Avisynth is probably the way to go.

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing). When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better. New samples below.

The script for the analysis pass:

I added trims to the script after creating the timecodes to apply filtering to the various sections. There's also a version included without any filtering. TFM just doing it's thing.

Encoding script without filters:

Encoding script with filters:

VFR Encodes, Take 2.zip (https://ufile.io/gsnvcw8r) (81.1 MB)

JoelHruska,
VFR isn't that hard to do, and this may be as close to a one size fits all method as it gets, and for the samples I've created at least, it looks the best. Avisynth outputs the average frame rate and the timecodes are used to make it variable at the encoding stage. You create the scripts for the analysis pass, run them, and TIVTC creates the metrics files. When I'm doing that sort of thing I use MeGUI, as it has an analysis pass mode and it's easy to add a bunch of jobs to the job queue. The trick is to never open the analysis scripts after running them, otherwise the metrics files are over-written and you have to run them again. Then it's just a matter of adding the encoding scripts to the job queue. If you specify unique names for the files for each episode, you can develop a system to do it pretty efficiently. I added Trims to one of the encodes above to apply the same filtering as before. It's easy to do, but it's not vital. You do have to decode with repeat flags being honoured though or it won't work.
Adding the timecodes to the x264 command line is done like this:

--tcfile-in "D:\Timecodes.txt"

PS. ChangeFPS(24000,1001) isn't the best way to decimate a 120fps source. It doesn't discriminate.
TDecimate(mode=7, rate = 24.0/1.001) tries to make sure the decimated frames are always duplicates.

Just an update on this. I had no problem getting the first part of the script working and successfully creating the analytics for TDecimate and TFMN.

I wasn't able to get the second part of the script to work, however. I could not find a way to load the D2V file to get the actual frame count off the two VOB files it is comprised of. When I attempted to load the individual VOBs in VDub2, they failed to load properly and displayed nothing like the actual frame count.

The MKV file output created by the analytics passes must not have the same number of frames as the VOB file it is based on. Setting the maximum number of frames in the clip to the same number of frames as the MKV file produced an Error 5 & 6 about how every frame must be accounted for (I do not recall the exact phrasing, but the implication was that the final frame number was incorrect).

So. If I cannot get the proper frame count from loading the VOB files in Vdub2, and I cannot use the MKV file output from the analysis pass, how do I know what the frame count is from the D2V file?

Also: Uncertain if your reference to timecodes is a reference to timecodes I'm supposed to have extracted from somewhere, or a reference to timecodes I'm extracting here for use in a separate encode step.

When I use DVDDecrypter to create VOB files and then input them into DGIndex to create a D2V file, I'm not creating a separate timecodes.txt file at any point -- at least not yet. Is this a step I need to be taking to make this editing method work?

Katie Boundary

19th May 2020, 02:06

Obviously I'm not on it now though

No, you're still on it, but I clicked on "view post" out of curiosity.

TFM() at default settings still delivers less desirable results than trbarry's uncomb(), and finding settings that delivered acceptable results took an unreasonable amount of time. Finding a practical application for its few advantages over uncomb() took even longer. Getting NNEDI3 to work at all can be torture. YADIF was somewhere between the two: it's tricky to get working, but not as much as NNEDI3, and while it doesn't have a lot of settings that need tweaking, it still took forever to find an application where its sole advantage over bob(b=0) really mattered.

None of these are "evils" that I was arguing. They're facts that you and several other people on the forum ignored while you criticized me for using tools that were guaranteed to work and to deliver acceptable results.

The biggest problem in my TFM+Interleave method is that it relies on TFM, and TFM seemingly doesn't look for interlacing, but for horizontal edges. There's a range of cthresh and mthresh settings where it'll incorrectly flag horizontal edges as interlacing while letting real interlacing through. If there was a filter that basically did what TFM does, but actually looks for interlacing instead of horizontal edges, then I could set cthresh and mthresh (or their equivalents) much higher, and there wouldn't be so much unnecessary bobbing of progressive content.

When I wanted to see the impact of incorrect deinterlacing filters on content, I ran deliberately-incorrect deinterlacing filters and looked at the results.

Well, deinterlacing filters don't get much more deliberately incorrect than QTGMC :D

Groucho2004

19th May 2020, 02:15

No, you're still on it, but I clicked on "view post" out of curiosity.I believe it was asked before - How old are you?

Katie Boundary

19th May 2020, 02:23

I believe it was asked before - How old are you?

Eight and a half.

videoh

19th May 2020, 02:53

how do I know what the frame count is from the D2V file? DGIndexNV puts the counts of coded and playback frames at the bottom of the DGI file (equivalent of D2V file). If you have an nVidia card, DGDecNV could help a lot.

For DGIndex, you can enable the info log and then the count of coded frames (also numbers of repeated fields/frames) will be in the log file.

Katie Boundary

19th May 2020, 03:05

Don, do you know why TFM loves to incorrectly flag horizontal edges as interlacing while it ignores real interlacing?

Katie Boundary

19th May 2020, 03:47

Well, there can be real detail at the same vertical spatial frequency as interlacing.

That's extremely unlikely, at least over areas larger than a few pixels

people say you don't give samples, but for your protector?

You are not my protector. You are not my knight in shining armor. You're someone who was an asshole to me 3 years ago for no reason, and whose current behavior I find extremely suspicious but not annoying or objectionable.

I'll see what I can do about uploading an Andromeda clip.

It's an interesting theoretical problem. Maybe we could bring the powerful Katie intellect to the problem. Think about it, dear, how would you distinguish between the two?

I can think of several approaches, none of which I think anyone here would bother to turn into a plugin. But for shits and giggles, here's one approach: Take any stack of pixels, 1 pixel wide by 5 pixels tall (same as TFM). Find the average luma value of pixels a, c, and e. Call it X. Find the average luma value of pixels b and d. Call it Y. Does the difference between x and Y exceed the differences between a and c, between c and e, between a and e, AND between b and d? Then flag the area as interlaced.

That approach might work better than TFM. Might not. I'd love to see its failure modes and tweak it accordingly.

videoh

19th May 2020, 03:50

Seems you lack a sense of humor. I won't trouble you again.

Katie Boundary

19th May 2020, 04:13

https://www.sendspace.com/file/urf1xo
https://www.sendspace.com/file/0iv2h6

Go ahead and try to find a combination of settings that will deinterlace the text in the first clip...

https://i.imgur.com/fxbqiyQ.png

...and the area in the red circle without wrecking the area in the yellow-green circle:

https://i.imgur.com/Wd4Bpxk.png

Seems you lack a sense of humor. I won't trouble you again.

Or perhaps you just weren't funny.

wonkey_monkey

19th May 2020, 07:14

Why didn't you mind your own business?

That's pretty much the gist of my question. You could have not said anything at all instead of actually going out of your way to be offensive.

Katie Boundary

19th May 2020, 09:01

Did one of Don's posts get deleted? I'd like to state for the record that I did not report it.

Anyway, I think we're on way too many tangents right now. We should focus on the retrograde field behavior question. Joel, load one of the VOB files into DGindex, use the "[" and "]" keys to mark the beginning and end of the clip you want to share, and select "File -> Save project and demux video". That'll create both a d2v file and an m2v file. Upload the m2v file to a file-hosting site like Sendspace and then post a link to it here so we can all run our own tests and see what you're dealing with and how to fix it, instead of telling you what tests to run and waiting for you to describe the results. When people here talk about providing a "sample", this is what they mean.

hello_hello

19th May 2020, 10:19

Well, deinterlacing filters don't get much more deliberately incorrect than QTGMC :D

Maybe if you didn't keep ignoring posts, by now you'd know QTGMC de-interlaces with NNEDI3 by default. The required plugins list:

MaskTools2, MVTools2, nnedi3, RgTools, Zs_RF_Shared.avsi

Everything from there attempts to repair problems. Temporal smoothing, denoising, sharpening etc. The things you've implied you'd do yourself after de-interlacing, using unnamed filters in the correct order.

Without your fingers in your ears you'd probably know QTGMC has a lossless mode. The original scanlines are output pixel for pixel. It's rarely the best mode though, because it faithfully reproduces artefacts and causes shimmering unless the source is very clean. There's a semi lossless mode that outputs the original scan lines before the final temporal smooth.

If you didn't ignore people who don't agree with you, you'd probably know QTGMC has source-match modes that attempt to match the output to the source without being lossless, and if you read the info on the wiki you'd know QTGMC's arguments give you a great amount of control over what it does.

If you looked at the optional plugins list you'd see it includes KNLMeansCL as a denoising option. Didn't you recommend using an NL-means denoising algorithm after de-interlacing yourself?

Of course QTGMC is deliberately "incorrect" as you put it, because it's defaults generally output a repaired de-interlaced clip, not a purely de-interlaced one. If you want the latter, then just keep using NNEDI3 or Yadif and apply your unnamed filters in the correct order afterwards.

hello_hello

19th May 2020, 10:21

Upload the m2v file to a file-hosting site like Sendspace and then post a link to it here so we can all run our own tests and see what you're dealing with and how to fix it, instead of telling you what tests to run and waiting for you to describe the results. When people here talk about providing a "sample", this is what they mean.

I linked to two samples for you in a previous post. They're the one's he's using. We're still waiting to see your results.

hello_hello

19th May 2020, 11:24

I wasn't able to get the second part of the script to work, however. I could not find a way to load the D2V file to get the actual frame count off the two VOB files it is comprised of. When I attempted to load the individual VOBs in VDub2, they failed to load properly and displayed nothing like the actual frame count.

The MKV file output created by the analytics passes must not have the same number of frames as the VOB file it is based on. Setting the maximum number of frames in the clip to the same number of frames as the MKV file produced an Error 5 & 6 about how every frame must be accounted for (I do not recall the exact phrasing, but the implication was that the final frame number was incorrect).

So. If I cannot get the proper frame count from loading the VOB files in Vdub2, and I cannot use the MKV file output from the analysis pass, how do I know what the frame count is from the D2V file?

I'm not sure what you're doing exactly, but you wouldn't open the vob files in VD or another GUI, you'd open the script. The script should open the vob files with mpeg2source. Once you've created the first pass script and run it, which I assume you've done by opening a script that opens the vob files with mpeg2source, you'd open the second pass script.

As I normally have MeGUI running, to combine the samples, I open the vob/m2v files with it, and get it to index them with DGIndex. MeGUI then creates a script to open each which I save. They look like this:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

and

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\space battle.d2v")

From there I copy/paste and manually create a script with Notepad, adding the other stuff to it.
My method it is to make the first pass and second pass scripts a single script, as it helps to keep things organised when you're working in batches. To switch from 1st pass mode to 2nd pass mode, I comment out the 1st pass lines and uncomment the rest. ie analysis pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B
Crop(8,0,-8,0, Align=true)

TFM(Output="D:\S01E01TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=4, Hybrid=2, Output="D:\S01E01TDecimate.txt")

# TFM(Input="D:\S01E01TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=5, Hybrid=2, Input="D:\S01E01TDecimate.txt", tfmIn="D:\S01E01TFM.txt", mkvout="D:\S01E01Timecodes.txt")

# Resize8(640,480)

Encoding pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B
Crop(8,0,-8,0, Align=true)

# TFM(Output="D:\S01E01TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=4, Hybrid=2, Output="D:\S01E01TDecimate.txt")

TFM(Input="D:\S01E01TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="D:\S01E01TDecimate.txt", tfmIn="D:\S01E01TFM.txt", mkvout="D:\S01E01TimeCodes.txt")

Resize8(640,480)

So for encoding a bunch of episodes I create the appropriate scripts for each, add them all to MeGUI's job queue and let it run all the analysis passes. Obviously when encoding in batches, the metrics files need to be appropriately named for each script, as I did for the example above.
Once that's done I edit the scripts for the encoding pass. The Timecodes file is automatically created by TDecimate when the encoding script is opened. Giving it a unique name for each episode is optional. If you don't it'll be over-written each time a script runs. If you do, you need to modify the x264 command line for each job you add to the queue (it doesn't matter if the timecodes file doesn't actually exists at that stage). If you don't want to give it a unique name, you can keep the same command line for each encode. ie
--tcfile-in "D:\TimeCodes.txt".

If you want to process the AVS output first, rather than encode it, or especially if you don't use the timecodes when encoding (I think it's better to do it that way though), you can add the timecodes when muxing the output with MKVToolNixGUI for a VFR that way, so you'd want to give the timecodes files unique names. I generally get the x264 encoder to write the output directly to an MKV. If it's writing a raw h264 stream, you'd probably have to use the timecodes when muxing too, even if you add them to the x264 command line for encoding.

After the analysis pass has run, when you open the encoding script, the timecodes file is automatically generated by TDecimate and the script output you see is the script you'll be encoding, IVTC'd, de-interlaced and decimated, with new frame numbers to match. The only difference is the frame rate, as it's being decoded at the average rate, but what you see is what's encoded. I use MeGUI's preview, but opening the script in VD should still display the frame numbers as it normally does.

If I want to add different filtering to different frame ranges, the final step for me would be to open the encoding scripts with MeGUI's AVS Cutter and use it to find the frame numbers and add the Trims to the script.
Even if I don't want to apply different filtering to different frame ranges, I still might want to exclude certain sections from filtering, so I'd add trims the same way. The combined Trims must include every frame, otherwise frames that are skipped won't be encoded when they're spliced together.

https://i.postimg.cc/c60FHkcP/AVS-Cutter.jpg (https://postimg.cc/c60FHkcP)

Once that's done, I add the script to MeGUI's job queue for encoding, but that's because I don't use other programs to process the Avisynth output first. Generally if you want to do something, there's a way to do it with Avisynth.

PS. Be careful not to open the script after the analysis pass until you've edited it to make it an encoding pass script. If you do, the metrics files are over-written and you'll have to run the analysis pass again. I still forget now and then, especially when running test encodes, and it's annoying.

Katie Boundary

19th May 2020, 12:26

Maybe if you didn't keep ignoring posts, by now you'd know QTGMC de-interlaces with NNEDI3 by default....

Without your fingers in your ears you'd probably know QTGMC has a lossless mode.

I've known both of those things for years. Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching, and can even re-introduce the interlacing that we're supposed to be getting rid of. I re-tested it very recently just to be sure, and the results are not much different from what plain unmodified NNEDI3 would have given me, and worse than what my own method produces.

Did you really think I wouldn't experiment with something like this at one point or another?

mpeg2source("212.d2v")

A=qtgmc(sourcematch=3,lossless=1).selecteven()
B=qtgmc(sourcematch=3,lossless=1).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

Interleave(C,D)

Well, I did. Years ago. But hey, now that it's here for everyone to see, maybe someone will take it and modify it and find a way to make it produce better results than my standard approach.

If you didn't ignore people who don't agree with you

I don't ignore people who disagree with me. I ignore arrogant jackholes who repeatedly prove that they can't be reasoned with and love talking out their asses, as you're doing now.

Of course QTGMC is deliberately "incorrect" as you put it, because it's defaults generally output a repaired de-interlaced clip, not a purely de-interlaced one.

It's deliberately incorrect in much worse ways than that.

hello_hello

19th May 2020, 13:28

I've known both of those things for years. Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching, and can even re-introduce the interlacing that we're supposed to be getting rid of. I re-tested it very recently just to be sure, and the results are not much different from what plain unmodified NNEDI3 would have given me, and worse than what my own method produces.

What?? So QTGMC is a bad de-interlacer now because it doesn't field match too? Does NNEDI3 or Yadif field match? Ridiculous.
Of course lossless mode can can re-introduce problems you're wanting to get rid of.

Did you really think I wouldn't experiment with something like this at one point or another?

mpeg2source("212.d2v")

A=qtgmc(sourcematch=3,lossless=1).selecteven()
B=qtgmc(sourcematch=3,lossless=1).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

Interleave(C,D)

I'm not going to spend time testing that method again, because I've linked to samples earlier in the thread where I used a modified version of your "method" with good results (and your original method with NNEDI3 de-interlacing that produced horrible results), but aside from not needing to de-interlace twice, and the field order for TFM being the wrong way around for the times I've tried your method:

DeintClip=qtgmc(sourcematch=3,lossless=1)
A=DeintClip.selecteven()
B=DeintClip.selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

You could use pp=2 for TFM, which would cause it to take the entire frame from DeintClip when combing is detected, not just bits of it that mightn't match the rest of the frame as such and cause problems that weren't there before, and then it'd be TFM's fault if it wasn't always right, not QTGMC's, and you could use far less aggressive combing detection settings, so it's not likely to take the whole DeintClip frame unnecessarily when IVTCing film sections, or you could adjust other settings to suit the clip instead of expecting the same method to produce the best result each time, and you could keep in mind QTGMC was designed to be a quality de-interlacer of interlaced video, without the knowledge that Katie would once again re-invent the wheel.

Well, I did. Years ago. But hey, now that it's here for everyone to see, maybe someone will take it and modify it and find a way to make it produce better results than my standard approach.

If I wasn't sure you were serious, I'd probably find that funny.
It doesn't make QTGMC a bad de-interlacer simply because it doesn't always work with your method, it just means your should use something else if the result isn't good.

I don't ignore people who disagree with me. I ignore arrogant jackholes who repeatedly prove that they can't be reasoned with and love talking out their asses, as you're doing now.

Your long standing poor attitude when asking for help certainly doesn't help. How many people are on your ignore list now? Obviously not enough to make you wonder if maybe you're the problem and not everyone else, and even though nobody else has felt the need to request forum features that allow them to effectively become a moderator of their own threads so they can't be disagreed with.
When did I make your ignore list? Was it while trying to explain that interlaced video is a thing and not 60p video with half the scanlines removed, or while trying to explain how field matching works?
Back then you had a quite a mental method, and anyone disagreeing was a troll or talking out of their arse. Now, after re-inventing the wheel again, something that doesn't work well with your current method is naturally labelled bad.

It's deliberately incorrect in much worse ways than that.

For example? Don't just throw out some unsubstantiated nonsense and expect me to buy it. You're the expert there apparently, so be specific.

hello_hello

19th May 2020, 14:14

Go ahead and try to find a combination of settings that will deinterlace the text in the first clip...

There's no combing. What's the problem?

For the second clip, why don't you try this:
TFM(pp=1).QTGMC(FPSDivisor=2)
Or this:
TFM(pp=1).NNEDI3()
Or this:
TFM(pp=1).Yadif()

TFM would be field matching without de-interlacing, and the whole frame is then de-interlaced.
I concluded TFM wasn't missing anything when the problem didn't go away, and was therefore baked into the the fields, but maybe I'm jumping to the wrong conclusion.

You can see by looking at the "light around the letter "n" that NNEDI3, and therefore QTGMC, have dropped half the fields to create a progressive frame, whereas TFM must be using both fields to create the new frame, and it seems logical to me the result of replacing just the combed pixels with pixels from a de-interlaced clip will sometimes look different according to how the de-interlaced clip is created.
If you switch between the field matched screenshot and the QTGMC and NNEDI3 screenshots, the QTGMC version is not as close a match to the field matched frame as NNEID3, as nothing has been "warped" by NNEDI3 on it's own. It doesn't make QTGMC's processing bad, because there's far less aliasing, it just means replacing combed pixels with pixels from a QTGMC de-interlaced clip mightn't always look the best.
If you look at the shape of the very light areas in the image though, you'll see NNEDI3 on it's own distorted some of them or reduced detail, so in that respect the QTGMC version is better.
As stand-alone interlaced frames though, the QTGMC version looks by far the best to me.

Telecined frame
https://i.postimg.cc/McrLpSFm/telecinded-frame.jpg (https://postimg.cc/McrLpSFm)

TFM(PP=1) Field matched, no de-interlacing
https://i.postimg.cc/G96DR5hw/filed-matched.jpg (https://postimg.cc/G96DR5hw)

TFM(pp=5)
https://i.postimg.cc/KKYVbzJY/TFM-pp-5.jpg (https://postimg.cc/KKYVbzJY)

TFM(pp=1).NNEDI3()
https://i.postimg.cc/1Vz8Sbf6/TFM-pp-1-NNEDI3.jpg (https://postimg.cc/1Vz8Sbf6)

TFM(pp=1).QTGMC(FPSDivisor=2)
https://i.postimg.cc/WFFHDyRL/TFM-pp-1-QTGMC-FPSDivisor-2.jpg (https://postimg.cc/WFFHDyRL)

poisondeathray

19th May 2020, 17:59

There's no combing. What's the problem?

KTB's clip1 does have combing, you can brighten it up if you can't see it with something like levels (raise the black level or boost gamma). You can brute force it with vinverse2, it does not really harm the text .

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

(Is this Hercules in space? I never got into this one either)

poisondeathray

19th May 2020, 18:14

There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample (https://ln2.sync.com/dl/5dd47a7d0/tanjt92v-e52sztfm-t9k5qzud-2tgukgp5)

Another option - some fans actually replace space scenes with their own CG footage . Search youtube there are many examples. You can do it in blender or other 3D programs. There are quite a few free models and textures from Star Trek universe (and many other sci fi shows)

It will stick out like a sore thumb, because even the low quality models and textures will look better , so you have to dumb it down a bit if you want to match quality with other scenes , especially with live actors and soft footage

Getting it to match exactly to match is a bit difficult , because the models might be slightly different, and the lighting will be difficult to mach exactly 100% . But in terms of framing, camera path, that's entirely "do-able"

hello_hello

19th May 2020, 18:54

KTB's clip1 does have combing, you can brighten it up if you can't see it with something like levels (raise the black level or boost gamma). You can brute force it with vinverse2, it does not really harm the text .

I've changed the levels, looked at it sideways, run it fullscreen, stepped through frames, and I'm stuffed if I can see anything in the text that needs de-interlacing. Are we both talking about "212 clip1.demuxed.m2v", or am I blind?

https://i.postimg.cc/Mv7vQLTJ/212-clip1-demuxed-m2v.png (https://postimg.cc/Mv7vQLTJ)

hello_hello

19th May 2020, 19:07

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

(Is this Hercules in space? I never got into this one either)

That's one I couldn't get TIVTC to run through smoothly using VFR encoding, although I didn't mess around too much.
I "think" my video card can do it, but if I start playing the clip from the beginning it totally misses IVTCing the first couple of frames of the shot of the ship and there's a judder in the motion, and the text part goes by before my brain can lock into the motion again, but I think it's pretty smooth after the first few frames.

I don't know anything about that show. I've never watched it.

poisondeathray

19th May 2020, 19:34

I've changed the levels, looked at it sideways, run it fullscreen, stepped through frames, and I'm stuffed if I can see anything in the text that needs de-interlacing. Are we both talking about "212 clip1.demuxed.m2v", or am I blind?

Look at the fade (in/out) parts

It's similar to text fades in DS9 (or even scene fades)

poisondeathray

19th May 2020, 19:41

That's one I couldn't get TIVTC to run through smoothly using VFR encoding, although I didn't mess around too much.

The BG is 23.976, but the text overlay is 29.97 - same deal as DS9

If you use no filters at all, and step through, you can see the text is pretty clean @ 29.97, except for the 1st few frames for the text wipe in - because the underlying BD has telecine and the overlay is partially transparent at that point . If you just filter the text and composite it back over, it's not too bad at 23.976. The wipe is not as smooth as compared to 29.97, but the BG is proper 23.976. Jerky ship is more noticable than jerky text wipe

Alternatively, you can completely retime the text at 23.976, but that's quite more work (might be able to do some of it with optical flow, but you'd need to clean it up manually)

hello_hello

19th May 2020, 19:51

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

Aliasing aside, there was enough image left after using TFM's own arguments for excluding sections of the frame from the field matching to allow it to decimate the CGI section to a CFR of 23.976fps. A pain to have to treat that section differently, although if it's the same intro for every episode I guess it's not too tragic.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate()
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2.mkv (https://ufile.io/awptek3s)

poisondeathray

19th May 2020, 20:04

Aliasing aside, there was enough image left after using TFM's own arguments for excluding sections of the frame from the field matching to allow it to decimate the CGI section to a CFR of 23.976fps. A pain to have to treat that section differently, although if it's the same intro for every episode I guess it's not too tragic.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate()
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2.mkv (https://ufile.io/awptek3s)

The problem with this - is a sort of "strobing" effect; every few frames is "blurred" slightly when the deinterlacer is applied

For the people doing the upscaling, jumping through hoops - I would go the extra mile to get it right, especially if it's some series that supposedly deserves better treatment . Many of the intros are recycled and the same, just the guest stars might be different

hello_hello

19th May 2020, 20:12

Look at the fade (in/out) parts

It's similar to text fades in DS9 (or even scene fades)

If it's what I'm seeing now, I'd never have noticed, or cared if I did, but is it combing as such, because if it was you could pick the right scanlines and interpolate the rest, but doing that looks like this:

nnedi3(field=-1)

https://i.postimg.cc/sB3tfw6c/clip1.png (https://postimg.cc/sB3tfw6c)

Edit: Here's a confession. I thought Katie had uploaded the wrong screenshot. I still use a Trinitron CRT for most things that aren't video, and don't have it set very bright, so when I looked at Katie's screenshot all I saw was black. Looking it it on the LCD I can see there be text now. :)

poisondeathray

19th May 2020, 20:32

If it's what I'm seeing now, I'd never have noticed, or cared if I did, but is it combing as such, because if it was you could pick the right scanlines and interpolate the rest, but doing that looks like this:

nnedi3(field=-1)

You could argue that nnedi3 produces "zebra-ed" result, where there are dark/bright bands on frame 331 . If you use something like vinverse2(sstr=0.5), you won't get those bands.

Also, nnedi3 "damages" the other frames more. If you look on a non fade frame , the horizontal line of "A" in "spiral" is obscured, and text less clear overall . (Although you could apply filters selectively only to specific frames)

clip1 is much "easier" because the BG is black

SaurusX

19th May 2020, 20:39

This is a case of too many cooks in the kitchen.

hello_hello

19th May 2020, 20:42

You could argue that nnedi3 produces "zebra-ed" result, where there are dark/bright bands on frame 331 . If you use something like vinverse2(sstr=0.5), you won't get those bands.

Also, nnedi3 "damages" the other frames more. If you look on a non fade frame , the horizontal line of "A" in "spiral" is obscured, and text less clear overall . (Although you could apply filters selectively only to specific frames)

clip1 is much "easier" because the BG is black

I agree, but I was just speculating if it's technically "combing" as such.

hello_hello

19th May 2020, 20:45

The problem with this - is a sort of "strobing" effect; every few frames is "blurred" slightly when the deinterlacer is applied

For the people doing the upscaling, jumping through hoops - I would go the extra mile to get it right, especially if it's some series that supposedly deserves better treatment . Many of the intros are recycled and the same, just the guest stars might be different

I'd also agree with that, but if it was me, just encoding the episodes to watch myself, I'd probably give it the QTGMC treatment to even it out a bit and call it a day. :)

After-all, it's only a few seconds of an opening credit. I don't think even my OCD extends beyond this.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate().QTGMC(InputType=1, Preset="very slow")
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2b.mkv (https://ufile.io/lhsbtybk)

hello_hello

19th May 2020, 21:00

By the way, how would you go about masking out the text. I assume it'd have to be done frame by frame? It's not something I've thought about doing before.

Katie Boundary

19th May 2020, 21:15

My latest private message from Joel indicates that he might have been duplicating interlaced frames before attempting any kind of deinterlacing. If that's the case, then I think we've just identified a huge source of his problems.

What?? So QTGMC is a bad de-interlacer now because it doesn't field match too?

Not what I said.

It doesn't make QTGMC a bad de-interlacer simply because it doesn't always work with your method, it just means your should use something else if the result isn't good.

No, what it means is that its results aren't noticeably better than NNEDI3 alone, and worse than my script... which is exactly what I already said.

Your long standing poor attitude when asking for help certainly doesn't help.

I don't have a poor attitude when asking for help. I have a poor attitude when dealing with people who can't read.

How many people are on your ignore list now?

Three.

There's no combing. What's the problem?

Oh, no wonder. It turns out that you can't even see. That explains a lot.

(Is this Hercules in space? I never got into this one either)

I don't know because I never saw Hercules but I might be able to explain it in Star Trek terms: Imagine Captain Kirk getting frozen in time for 300 years and waking up to find that his crew is dead, the Federation has disintegrated, the whole galaxy has been overrun by Khan's genetically engineered supermen and a race of giant, technologically advanced, man-eating tribbles, and it's up to him and a group of low-budget actors from Vancouver to put civilization back together.

The first two seasons were pretty good.

hello_hello

19th May 2020, 21:27

Not what I said.

It's exactly what you said. How do you manage to deny what's there in back and white?

Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching

No, what it means is that its results aren't noticeably better than NNEDI3 alone, and worse than my script.

If you don't think the QTGMC results are noticeably better than NNEDI3 alone, there's no point discussing it any further, because it's obviously far better than NNEDI3 alone for de-interlacing an interlaced source. If it wasn't, everyone would simply de-interlace with NNEDI3.
Worse than your script?? At what, de-interlacing a purely interlaced source as it's designed to do? Comedy Gold!!

I don't have a poor attitude when asking for help. I have a poor attitude when dealing with people who can't read.

The consensus here suggests otherwise.

Oh, no wonder. It turns out that you can't even see. That explains a lot.

That's been explained, read the rest of the posts and move on.

Katie Boundary

19th May 2020, 21:57

It's exactly what you said.

Wrong. Thank you for once again confirming everything I've said about your lack of reading skills.

The consensus here suggests otherwise.

There is no consensus. Thank you for once again confirming everything I've said about your lack of reading skills.

poisondeathray

19th May 2020, 22:05

By the way, how would you go about masking out the text. I assume it'd have to be done frame by frame? It's not something I've thought about doing before.

Not frame by frame, there is keyframe "interpolation" in most editors, you just do every nth frame and it fills the rest in, and you might have to fine tune a bit

It's called "rotoscoping" or essentially moving masks . It's a basic "101" skill set for programs like after effects, blender, natron, fusion, resolve, etc... most video editors. Mocha and silhouette are dedicated rotoscoping tools, they have some additional tricks that can help .

You don't need to be very precise for this or ds9, just a rough shape. (But you can be subpixel precise if you needed to)

The top of the ship in this one, or some of the DS9 shots , you're going to use pretty heavy filter stacks with, QTGMC in progressive mode, some AA filters, you don't want to "damage" the other parts, so you use masks. Especially important for the upscaling people, those strong AA filters reduce what little detail was there in the first place

Stereodude

19th May 2020, 22:17

My latest private message from Joel...
There's a key part of his problem. He's probably getting questionable advice.

The rest of us discuss ideas on how to process these episodes in the open so others can offer improvement and alternate methods, not hide them in the dark.

Katie Boundary

19th May 2020, 23:13

There's a key part of his problem. He's probably getting questionable advice.

The rest of us discuss ideas on how to process these episodes in the open so others can offer improvement and alternate methods, not hide them in the dark.

Our private messages are specifically about the retrograde field behavior issue, since you're all apparently unwilling to discuss it here.

poisondeathray

19th May 2020, 23:59

Our private messages are specifically about the retrograde field behavior issue, since you're all apparently unwilling to discuss it here.

I specifically asked for a sample of this

And for an "interlaced content" sample . (Not text, not fades)

hello_hello

20th May 2020, 04:05

Wrong. Thank you for once again confirming everything I've said about your lack of reading skills.

You can keep accusing me of a lack of reading skills but it only makes you look more childish.
You implied.... no actually stated... that QTGMC's lossless mode doesn't field match. Of course it was a criticism.

Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching

There is no consensus. Thank you for once again confirming everything I've said about your lack of reading skills.

You flogged the "poor reading skills" line to death a long time ago.
It doesn't take any reading skills to know that in almost every thread where you ask a question, someone complains about your attitude.

You've not mentioned any of the screenshots or samples I uploaded. Why not discuss them instead of just bickering? Or at least discuss the topic in addition to bickering. Where's your encodes showing how your method does it better? I supplied links for the source files that others supplied, and you provided links to your own source samples. Show us how they look using your method and explain why it's better instead of just pontificating.

manono

20th May 2020, 04:14

Geez but I'm getting sick of this. I'll happily close this thread and any other where the male testosterone level gets too high. For the last time, keep it civil. Don't gimme any of the "But she started it" crap, either. Some of you people behave like children.

Katie Boundary

20th May 2020, 04:57

I specifically asked for a sample of this

And for an "interlaced content" sample . (Not text, not fades)

Retrograde field behavior has nothing to do with text or fades.

poisondeathray

20th May 2020, 05:07

Retrograde field behavior has nothing to do with text or fades.

I never said it did.

"And an interlaced content" sample was requested. It's a separate topic. I suggested someone post both earlier. People might see "combing" on text or fades and identify it as "interlaced", but I'm not interested in text or fades - that has already been established. I'm asking if there is actual interlaced content other than text or fades - that affects how you might approach VFR

Katie Boundary

20th May 2020, 05:15

OOh, gotcha. Well I already posted samples of retrograde field behavior a long time ago in the two other threads dedicated to that exact topic. I'm not sure what would be gained from posting them a third time, since the consensus hasn't changed: You have to either throw half the fields away or blend. Or manually correct the field order one field at a time with dozens of trim commands, if a sane field order even exists in the first place.

I'm pretty sure you already know what interlacing looks like, so I'm not sure why you'd need a sample of that either.

EDIT: if you mean that you specifically want samples of these things from DS9, then I'm the wrong person to ask, since I don't have DS9 at home.

poisondeathray

20th May 2020, 05:21

OOh, gotcha. Well I already posted samples of retrograde field behavior a long time ago in the two other threads dedicated to that exact topic. I'm not sure what would be gained from posting them a third time, since the consensus hasn't changed: You have to either throw half the fields away or blend.

But does it exist here in DS9? Did you confirm it ?

People use various terms all the time, but sometimes some people describe things incorrectly.

Or some people used ffms2 in this thread . I've seen that cause field misorder - is that the cause ? Or something else in the script ? Or some other program ?

I'm pretty sure you already know what interlacing looks like, so I'm not sure why you'd need a sample of that either.

Same as above . Did you verify? Any content other than text or fades ?

3:2 telecine can look like "interlacing" too in layman's terms on some frames . But it's not interlaced content.

Katie Boundary

20th May 2020, 05:25

My Star Trek DVD collection is limited to the first ten movies and Voyager. It's up to Joel to provide DS9 clips. But if h_h wants to lose a bet with me about his/her incorrect and easily disprovable claims about the threads that I start, then I promise that DS9 will be the first thing I buy with his/her money.

hello_hello

20th May 2020, 05:33

Katie Boundary

20th May 2020, 09:42

This thread stopped being just about DS9 a looooong time ago.

Swede

20th May 2020, 10:26

This thread stopped being just about DS9 a looooong time ago.

Yes it has. AND the second anyone posts anything BUT you will be stricken, hard, and the thread will be closed. Be careful now! All of you!

Katie Boundary

20th May 2020, 15:15

I think I understand what PDR means when he says "actual interlaced content other than text or fades". You mean 60 hz content where the WHOLE image is changing every 1/60th of a second, not where parts of it change every 1/30th of a second and other parts change every 1/24th of a second, right? I agree, finding out how much of that content exists in DS9 has huge ramifications, for reasons explained in post #257

EDIT: I am such an idiot. There's no need to specify field parity for TFM at all. You can just let line C use whatever field AVIsynth says is first, and let line D use complementparity():

Y=nnedi3(field=-2)

A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(clip2=A)
D=complementparity().Tfm(clip2=B)
Interleave(C,D)

(unnecessary parameters removed for readability)

Then it'll work on both TFF and BFF content without needing manual adjustment.

JoelHruska

21st May 2020, 03:26

Ok, so, a few things:

1). I messaged KB about what she calls "retrograde field behavior" because when I first attempted to run various TIVTC scripts against DS9, one of the problems all of these scripts introduced into various *specific* video sequences was frames moving backwards. This problem is actually what pushed me away from TIVTC and towards QTGMC in the first place. The reason I messaged her privately, ironically, is precisely because I was trying to avoid drama here. I followed KB's instructions for looking for this behavior. I did not see it.

I am not here to argue with anyone. I am not here to take sides or cause fights.

2). I am not going to keep anything in the dark. The entire stated purpose of my project is to create a tutorial.

3). H_H, I saw your response to me and I'll respond back to you tomorrow. Had a major work project for the last couple days.

I appreciate people being helpful with this project. I know I have been slow to test some of these workflow ideas -- I've been hit with other projects and I'm preparing to move -- but I intend to spend some time with it this weekend.

JoelHruska

21st May 2020, 04:00

One update to the above. I have done what some of you have recommended: Specifically, I analyzed the file with DGIndex, created a D2V file, and then used StaxRip to rip the VOB to an MKV. The resulting file is in 29.97 fps, not 23.976 fps -- but this injects 3:2 pulldown where none previously existed.

Do I really want to convert the episode into 29.97 fps?

manono

21st May 2020, 04:52

...and then used StaxRip to rip the VOB to an MKV
Bad idea. After decrypting the DVD to the hard drive, you make the D2V file for the episode and then use MPEG2Source on it. That's thoroughly explained in the docs accompanying the DGMPGDec package. Forget repackaging to an MKV. I don't know whose toes I stepped on with that statement.

The resulting file is in 29.97 fps, not 23.976 fps -- but this injects 3:2 pulldown where none previously existed.

All NTSC DVDs output 29.97fps. If it's really progressive 23.976fps, you IVTC it back to 23.976fps. So, the latter part of your statement is incorrect. The only time you should get 23.976fps out of DGIndex is by setting Field Operation to Force Film and you do that only if it's 100% film. Some say 95% film is good enough, but not I. You can open the D2V and at the bottom it'll tell you the percentage of film.

Stereodude

21st May 2020, 05:06

Do I really want to convert the episode into 29.97 fps?
It already is i60 (when the soft pulldown flag is honored). You don't want an input file that changes framerates. That's how you have audio sync issues. Because you have a mix of i60 encoded content and p24 content with a soft pulldown you need to apply the soft pulldown and work from there. The flags=5 argument in TFM will undo the soft pulldown perfectly without relying on visually matching fields.

Katie Boundary

21st May 2020, 07:16

1). I messaged KB about what she calls "retrograde field behavior" because when I first attempted to run various TIVTC scripts against DS9, one of the problems all of these scripts introduced into various *specific* video sequences was frames moving backwards. This problem is actually what pushed me away from TIVTC and towards QTGMC in the first place. The reason I messaged her privately, ironically, is precisely because I was trying to avoid drama here. I followed KB's instructions for looking for this behavior. I did not see it.

See, this is why I kept hounding everyone here about figuring out what was going on with the field order. If using "honor pulldown flags" in DGindex and then bob() in AVIsynth, and doing absolutely nothing else, shows absolutely no stuttering/RFB, but some other workflow does, then the other workflow has a huge screw-up somewhere. So we need to know: are you blindly duplicating frames to get your frame rate up to 48 fps before applying any kind of deinterlacing? What exactly are you doing that produced this effect?

BTW, the term "retrograde field behavior" isn't mine. I got it from John Meyer.

Do I really want to convert the episode into 29.97 fps?

No, you want to convert it to 59.94 :)

manono

21st May 2020, 07:30

Because you have a mix of i60 encoded content and p24 content with a soft pulldown you need to apply the soft pulldown and work from there.
Oh, there's soft pulldown in these? If there have been samples posted, I haven't looked at them and don't intend to go back through this thread to find them. Then, yes, avoid field-matching if possible by using the D2V argument to read the soft pulldown from the D2V file. It'll go something like this:

TFM(D2V="Video.d2v",Flags=5)

I've always used the default 4, but if you prefer 5 for these...

hello_hello

21st May 2020, 10:30

The section poisondeathray linked to looks interlaced. I originally thought it was, but it's progressive with the fields out of alignment (if that's the correct description), however the overlaid shooting is interlaced, either because it was overlaid with the fields out of phase with the CGI, or it is interlaced, or telecined, or something.
https://forum.videohelp.com/threads/396652-Deinterlacing-Deep-Space-Nine-NTSC#post2578796

Katie's method without YadifMod, given TFM's taking pixels from YadifMod, which is in turn taking them from nnedi3 anyway, as best as I can tell. I also disabled micmatching to fix the jitter towards the end.
Y=nnedi3(field=-2)
A=Y.selecteven()
B=Y.selectodd()
C=Tfm(clip2=A,micmatching=0)
D=complementparity().Tfm(clip2=B,micmatching=0)
Interleave(C,D)
space battle Katie.mkv (https://ufile.io/a930oo8r)

Katie's method unmodified
Y=nnedi3(field=-2)
A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(clip2=A)
D=complementparity().Tfm(clip2=B)
Interleave(C,D)
space battle katie with yadif.mkv (https://ufile.io/fmukhe2i)

VFR
# TFM(Output="D:\TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=4, Hybrid=2, Output="D:\TDecimate.txt")
TFM(Input="D:\TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimate.txt", tfmIn="D:\TFM.txt", mkvout="D:\Timecodes.txt")
space battle VFR.mkv (https://ufile.io/uyrm10ih)

VFR with a QTGMC de-interlaced clip
DeintClip=QTGMC().SelectEven()
# TFM(Output="D:\TFM.txt", PP=5, Clip2=DeintClip, micmatching=0)
# TDecimate(Mode=4, Hybrid=2, Output="D:\TDecimate.txt")
TFM(Input="D:\TFM.txt", PP=5, Clip2=DeintClip, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimate.txt", tfmIn="D:\TFM.txt", mkvout="D:\Timecodes.txt")
space battle VFR QTGMC.mkv (https://ufile.io/z0x8spkh)

VFR followed by QTGMC in progressive mode
# TFM(Output="D:\TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=4, Hybrid=2, Output="D:\TDecimate.txt")
TFM(Input="D:\TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimate.txt", tfmIn="D:\TFM.txt", mkvout="D:\Timecodes.txt")
QTGMC(InputType=1, Preset="Very Slow")
space battle VFR QTGMC P.mkv (https://ufile.io/l2pkeu4j)

I followed each encode with some cropping and resizing, because I felt like looking at the correct aspect ratio, plus it'll be resized at some stage anyway.
CropResize(0,0, 6,2,-4,-2, InDAR=15.0/11.0, ResizeWO=true, Resizer="Resize8")

I suspect, but haven't seen a sample of it, the "retrograde field" problem could be fixed by using TFM in a somewhat standard manner with micmatching disabled. It might just be picking the wrong match on occasion.
If there are purely interlaced sections, and the sample above may suggest otherwise, I guess Katie's method should bob them to 59.94fps, but I'm not sure that's always a good idea because it might make them stand out even more from the film sections, and more of a pain than it's worth if they're few and far between, but either way, have we learned how Katie goes about decimating all those duplicate frames after outputting everything at 59.94fps? I'm sure she's mentioned doing it, but not how she does it. It'd be tedious to do manually, especially when it comes to creating matching timecodes for a VFR.

For the VFR encodes, It's hard to avoid seeing "quality pumping" for the frames where the de-interlacing kicks in to fix the combing in the "shooting". In that respect, I think the VFR encode with QTGMC de-interlacing looks the best, but it does cause some aliasing (not as much as nnedi3 de-interlacing on it's own though). The other problem is you have to use the de-interlaced clip for the analysis pass, and if it's a QTGMC clip that'll make the analysis pass much slower.

nnedi3 deinterlaced
https://i.postimg.cc/PNHbf5Px/nnedi3-deinterlaced.png (https://postimg.cc/PNHbf5Px)

qtgmc deinterlaced
https://i.postimg.cc/5678m5pY/qtgmc-deinterlaced.png (https://postimg.cc/5678m5pY)

tfm deinterlaced, pp=5
https://i.postimg.cc/cgyRnshF/tfm-deinterlaced-pp-5.png (https://postimg.cc/cgyRnshF)

Stereodude

21st May 2020, 15:30

Oh, there's soft pulldown in these? If there have been samples posted, I haven't looked at them and don't intend to go back through this thread to find them. Then, yes, avoid field-matching if possible by using the D2V argument to read the soft pulldown from the D2V file. It'll go something like this:

TFM(D2V="Video.d2v",Flags=5)

I've always used the default 4, but if you prefer 5 for these...
A value of 4 for flags doesn't address the primary issue I saw in the few episodes I tested. The problem is that TFM sees combing where there is none and deinterlaces large areas of the frames despite it being progressive with no real combing. 5 puts an end to that. I didn't seen any examples of combing (except in the occasional scene change) which 5 still checks and addresses.

Katie Boundary

21st May 2020, 17:37

*facepalm*

h_h, if you're going to use something that looks superficially similar to my approach, at least use something that was actually designed to be used, not something that was posted only for demonstrative purposes. It is insulting and borderline slanderous to refer to a script that I would never use or approve of as "Katie's method".

And in case you're wondering, Yadifmod2 provides temporal smoothing, which NNEDI3 does not. The whole idea behind yadifmod2 is to add temporal interpolation to bobbers that don't have it. The effect of this is to eliminate flicker along object edges during interlaced fades.

hello_hello

21st May 2020, 19:26

h_h, if you're going to use something that looks superficially similar to my approach, at least use something that was actually designed to be used, not something that was posted only for demonstrative purposes. It is insulting and borderline slanderous to refer to a script that I would never use or approve of as "Katie's method".

I didn't use something superficially similar. There's a sample encode using the exact method you posted. I thought you'd be happy I showed it works. I was going to add micmatching=0 to it, to fix a glitch, but then of course I'd be accused of changing it and still calling it your method.

Why not post an encode from a script that was actually deigned to be used instead of pushing some generic idea? We've heard the talk. What would you change that'd make it look all that different anyway? Was TFM taking too much from the nnedi3 clip? To little? It was hardly an unfair comparison, as I didn't use different TFM settings for the VFR samples, aside from micmatching=0. Tell me what to change for your method and I'll try it for a VFR encode too.

I've tried setting cthresh and mthresh to 2 as you've suggested previously, and linked to samples of it in previous posts.

I asked the question again, but you've chosen to ignore it again. You've referred to decimating a 59.94fps output for a VFR. How do you go about it?

And in case you're wondering, Yadifmod2 provides temporal smoothing, which NNEDI3 does not. The whole idea behind yadifmod2 is to add temporal interpolation to bobbers that don't have it. The effect of this is to eliminate flicker along object edges during interlaced fades.

Yep, I've wondered about that numerous times without a response. Now you've finally supplied one, so I've looked, and I can see a picture de-interlaced via YadifMod slightly smooths the aliasing caused by taking pixels from a nnedi3 clip. I hadn't noticed the difference before because there's still aliasing and I wasn't aware of a reason to look for it.
If I supply TFM with a de-interlaced clip that needs temporal smoothing, I'll probably do it that way from now on. I might even try it with a QTGMC de-interlaced clip to see how much difference it makes.

hello_hello

21st May 2020, 20:11

NNEDI3 retains more detail than QTGMC when TFM is de-interlacing where it shouldn't, but you pay for it with aliasing.

Interestingly, with QTGMC de-interlacing, Yadif seems like it might help retain detail when TFM is de-interlacing where it shouldn't, but you pay for it with aliasing.

I used YadifMod2 out of habit, but the result should be the same.

DClip1 = NNEDI3(field=-2).SelectEven() # or QTGMC()
DClip2 = NNEDI3(field=-2) # or QTGMC()
DClip3 = yadifmod2(mode=1,edeint=DClip2).SelectEven()
TFM(clip2=DClip1) # or TFM(clip2=DClip3)

http://www.framecompare.com/screenshotcomparison/77D67NNX

http://www.framecompare.com/screenshotcomparison/99JMFNNU

lansing

21st May 2020, 22:22

Use http://www.framecompare.com when doing comparison, it's a lot easier to show the differences

hello_hello

21st May 2020, 22:51

I wasn't aware of that site. I'll try it next time.

Edit: I tried it last time.

JoelHruska

22nd May 2020, 02:34

Bad idea. After decrypting the DVD to the hard drive, you make the D2V file for the episode and then use MPEG2Source on it. That's thoroughly explained in the docs accompanying the DGMPGDec package. Forget repackaging to an MKV. I don't know whose toes I stepped on with that statement.

I don't think I've been clear.

If you want to use MPEG2Source in StaxRip, you load a D2V file that you've created in DGIndex... which is what I did. I use StaxRip as a front-end for running AviSynth. I used MKV for output because StaxRip automatically selects it. I may not have a particular sense of which file type to output in, but that also means I couldn't care less (except, perhaps, for retaining workflow compatibility) whether output is in MKV or not. So what ought it be in?

All NTSC DVDs output 29.97fps. If it's really progressive 23.976fps, you IVTC it back to 23.976fps. So, the latter part of your statement is incorrect. The only time you should get 23.976fps out of DGIndex is by setting Field Operation to Force Film and you do that only if it's 100% film. Some say 95% film is good enough, but not I. You can open the D2V and at the bottom it'll tell you the percentage of film.

Alright. Sacrifice of Angels is 86% film. That would increase if you excised the credits. It would not excise enough to bring the number above 95%, I don't think.

Alright. So, at this point in the process, I have a D2V file for Sacrifice of Angels. What I do not have is an actual video file I can get a frame count on. Attempting to load the VOB files into Vdub2 produces very strange results with only a few seconds or minutes of footage available from a 1GB file. There's no way to load a D2V into Vdub2. What app do you use for the frame-by-frame checking to see where TIVTC needs to be applied?

manono

22nd May 2020, 03:01

What app do you use for the frame-by-frame checking to see where TIVTC needs to be applied?
MPEG2Source in an AviSynth script file (an .avs). I then open it in Virtual Dub (or you can use VDub2). Again, read the docs included with DGMPGDec. Although it has a good reputation, you don't need StaxRip to do what you're asking. Then you use TIVTC as explained by Stereodude and me earlier.

I don't think I've been clear.
You provided an MKV. Inside that MKV was an M2V. But you can't create a good D2V file when it's inside an MKV. Anyone intending to help and intending to use MPEG2Source has to first extract that M2V from the MKV. It wastes time.

If 86% film, then that 86% is treated as if you used Force Film in DGIndex (giving you a progressive 23.976 video for those portions) and the rest undergoes a full IVTC (field-matching and decimation) with the whole thing winding up as progressive 23.976fps.

...frame-by-frame checking to see where TIVTC needs to be applied? You don't need to do any checking. With TIVTC set up properly, it can do the job all by itself. If there's any real interlaced video or any progressive 29.97fps portions, that'll throw a monkey wrench into the whole procedure.

JoelHruska

22nd May 2020, 03:06

Also, I've just noticed something really important.

My QTGMC repair script creates much worse-looking output if run against a VOB than if run against MakeMKV. Proof:

Look at the left-hand side of the Jem'Hadar fighter on the far left. The left-hand side of this image is 29.97 VOB with my QTGMC repair script applied. The right-hand side image is 23.976 fps output created with MakeMKV and processed with the same QTGMC script.

I did all of my tuning on that repair script using MakeMKV, and you can tell.

First frame comparison:

https://i.imgur.com/ndXgnKq.png

Second frame comparison:

https://i.imgur.com/E7v5bfT.png

The left-hand image is duller and markedly less sharp.

Here, the difference is downright egregious:

https://i.imgur.com/y1KW4so.png

Note: In a stunning display of competence on my behalf, the left-hand file names above are incorrect. If you've used Staxrip, you know you can change your script settings and just hit "Encode" again without specifically changing the output file name. I have sometimes forgotten to actually change out the file names. In this case, I know what the output actually is, because I ran the encode on the left today and have documented the settings on the right. The left-hand images are not unprocessed, despite the fact that they say they are.

This is after running the exact same QTGMC script against both videos.

I now understand why H_H and some others have cautioned against the use of QTGMC. When my repair script is used against a VOB-ripped file at 29.97 fps, the end result is rather awful looking. The MakeMKV output processed at 23.976 fps is *substantially* better than the VOB.

All of the CGI scenes in the episode look substantially worse when processed with QTGMC in 29.97 fps than when processed by QTGMC in 23.976 fps.

manono

22nd May 2020, 03:13

My QTGMC repair script creates much worse-looking output if run against a VOB than if run against MakeMKV.
Nonsense.

Apples to oranges, don't you think? Maybe you should IVTC the episode down to 23.976fps first? And using the procedure I described? I didn't bother looking at the pics.

JoelHruska

22nd May 2020, 03:22

MPEG2Source in an AviSynth script file (an .avs). I then open it in Virtual Dub (or you can use VDub2). Again, read the docs included with DGMPGDec. Although it has a good reputation, you don't need StaxRip to do what you're asking. Then you use TIVTC as explained by Stereodude and me earlier.

Alright. I'll specifically go digging around there.

You provided an MKV. Inside that MKV was an M2V. But you can't create a good D2V file when it's inside an MKV. Anyone intending to help and intending to use MPEG2Source has to first extract that M2V from the MKV. It wastes time.

Nope. Definitely have not been clear.

Let me break down exactly what I'm doing:

1). Rip the DVD in DVD Decrypter. This creates the following directory and populates it with VOB files: C:\DS9S6D2\VIDEO_TS
2). Load the VOB file into DGIndex. Confirm that DGIndex is set to honor pulldown flags.
3). Run the project through DGIndex. This creates a D2V file as output.
4). Load the D2V file into StaxRip. This opens the project file using MPEG2Source. Screenshot:

https://i.imgur.com/xD7Enho.png

The output of this process is the MKV file.

JoelHruska

22nd May 2020, 03:25

Nonsense.

Apples to oranges, don't you think? Maybe you should IVTC the episode down to 23.976fps first? And using the procedure I described? I didn't bother looking at the pics.

If you didn't bother looking at the images, on what basis are you declaring this nonsense? I've been fine-tuning that QTGMC output on 23.976 fps source for three months. I promise you, I'm qualified to determine when it looks different. The reason I'm bothering to post about it is because H_H was comparing my output earlier when I first started posting, and he provided feedback based on scripts he ran. But if he ran my QTGMC script against 29.97 content then he didn't see the output I saw when I opted to use the script in the first place.

Given that the entire point of being here is to get advice on the best practices, methods, and comparative results of different encode methods, it would seem important that we all be on the same page when attempting to evaluate different material.

Since you declare my post "Nonsense" followed by an acknowledgement that 23.976 and 29.97 are two different frame rates and thereby likely to produce two different results, I have no idea how to read your comment. It may not be relevant to you that the output you see off my script is substantially different than what I see, but having spent months experimenting with QTGMC to arrive at the results I had arrived at, I assure you, it's relevant as heck to me.

poisondeathray

22nd May 2020, 04:06

I now understand why H_H and some others have cautioned against the use of QTGMC. When my repair script is used against a VOB-ripped file at 29.97 fps, the end result is rather awful looking. The MakeMKV output processed at 23.976 fps is *substantially* better than the VOB.

This shows you don't understand or didn't read. By your own admission you said you are new to this.... so pay attention

When you IVTC the VOB, and then apply your repair script - you will get same results if that was a 23.976p section as makemkv at 23.976p and then applying the repair script. Try it and prove it to yourself

Again, you might want to use that repair script along with QTGMC in progressive mode on those 5% problem sections. But on the 95% + sections, no way, you kill too much detail

JoelHruska

22nd May 2020, 04:13

Manono,

I have created an AVSI file per your instructions, using the DGDecode manual for syntax advice. The AVSI is very simple:

LoadPlugin("C:\Path\DGDecode.dll")
MPEG2Source("C:\Path\VTS_2_1.d2v")

I load the AVSI into Vdub2. Vdub2 displays the following screen:

https://i.imgur.com/TzTQGPQ.png

When I loaded the script the very first time, the default frame rate was 15 fps. I adjusted it to 29.97 fps and it has stayed that way thereafter. However, upon clicking "Ok," nothing further happens. No video loads. Vdub2 displays the following:

https://i.imgur.com/5OlHn7H.png

I have tested VirtualDub2_44282 and _44237. I have experimented with changing the various options in the Input Format, as well as frame order. Nothing has mattered. I experimented with adding "info=1" to see if this would display video information when the load fails in Vdub2. It did not. I created multiple D2V files using DGIndex in case one of them had been corrupted. Neither file worked.

Any idea what the problem is?

JoelHruska

22nd May 2020, 04:17

This shows you don't understand or didn't read.

The former. I have no interest in wasting time, either mine or yours. I have spent hours reading the posts in this specific thread and continue to return to the earlier posts to see the entire flow of conversations. I knew that you were advocating working in VOB -- I saw and understood that. What I did not understand was that my QTGMC script would produce different output on a 29.97 VOB compared to a 23.976 MKV. In my own defense, I *thought* I had checked this possibility. I have seen my script run against 24.66 fps and 23.976 fps, as well as 119.88 fps. I've seen it run against a CFR file from Handbrake as well as a VFR file. I would not have been surprised to see it perform slightly differently, but the output on 29.97 is far worse than anything I saw at 23.976.

Clearly I did not understand and I apologize for that.

poisondeathray

22nd May 2020, 04:19

The file extension should be ".avs" only , not ".avsi" - the latter are auto importing avisynth scripts

In the vdub2 open file dialog box, it should say AVIFile/AVISynth input driver, not raw video input driver

manono

22nd May 2020, 04:20

Nope. Definitely have not been clear.

Damn, I've been confusing 2 separate threads. A different fellow uploaded a sample from a VOB packaged as an MKV. I haven't even seen any samples from you. I apologize.

Now, pay attention to what pdr wrote. Some (hello_hello, among others) sometimes use QTGMC specifically for its cleaning properties. That's fine. It seems to me, though, that if you want to do some 'cleaning' (spatial and/or temporal denoising) you use filters designed specifically for that. As pdr mentions, that cleaning QTGMC does by default also removes a little bit of detail. And, since you're upscaling to Hi-Def, you want all the initial detail you can get.

Edit: Now I see you've seen the light. :)

JoelHruska

22nd May 2020, 04:43

PDR,

Thank you. This worked -- once I found a DLL that can supposedly function to decode MPEG2Source with a 64-bit DLL:

http://avisynth.nl/index.php/MPEG2DecPlus

Can I rely on MPEG2DecPlus64.dll to provide equivalent MPEG2 decoding to DGDecode? I would prefer to use an x64 DLL if at all possible. If I need to specifically drop back to 32-bit for everything to make DGDecode work and there's no functional x64 equivalent, I can bring up a new system tomorrow.

poisondeathray

22nd May 2020, 04:53

Can I rely on MPEG2DecPlus64.dll to provide equivalent MPEG2 decoding to DGDecode?

Yes, that is the correct version and works fine

It's the other x64 version by joshyd (DGDecode_3-19-2010) that has problems

JoelHruska

22nd May 2020, 05:00

I avoided that one on the basis of age. The current 32-bit DGDecode.dll is from 2011. Seemed better to look for a later source than the earlier DLL based on earlier code.

Katie Boundary

22nd May 2020, 06:01

I will be running DS9 tests soon, but for now, I'm still using Andromeda as my guinea pig for various deinterlacing methods. Just to be ABSOLUTELY sure about the relative merits of NNEDI3+Yadifmod2 versus QTGMC, I put the following script together:

mpeg2source("212.d2v")

A=qtgmc(sourcematch=3,lossless=1).selecteven()
B=qtgmc(sourcematch=3,lossless=1).selectodd()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=1,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=1,MI=40,blockx=8,mthresh=2,clip2=B,micmatching=0)

E=Interleave(C,D)

Y=nnedi3(field=-2,nsize=3,nns=4,qual=2,etype=1)

L=yadifmod2(mode=3,edeint=Y).selecteven()
M=yadifmod2(mode=3,edeint=Y).selectodd()
N=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=1,MI=40,blockx=8,mthresh=2,clip2=L,micmatching=0)
O=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=1,MI=40,blockx=8,mthresh=2,clip2=M,micmatching=0)
P=Interleave(N,O)

stackvertical(E,P)

(I kept the manually specified field arguments instead of using the complementparity approach because I know the field parity already)

This was the result:

https://i.imgur.com/OA212K4.png

Remember, that's "lossless" QTGMC at the top, and Yadifmod2+NNEDI3 at the bottom. HuRr DuRr MuH aLiAsInG.

I didn't use something superficially similar. There's a sample encode using the exact method you posted.

No, you posted encodes using a lobotomized script that I provided for the purpose of demonstrating where Complementparity() would go. HUGE difference.

I thought you'd be happy I showed it works. I was going to add micmatching=0 to it, to fix a glitch, but then of course I'd be accused of changing it and still calling it your method.

All of my ACTUAL methods use micmatching=0 and about a dozen other settings that I cut out of that particular bit of code because, as I stated, the complementparity demonstration code was NOT meant for actual use.

So what ought it be in?

For compatibility: AVI
For variable frame rate: MP4

MP4 is an official industry standard recognized by the International Organization for Standardization as ISO/IEC 14496-14:2003, whereas MKV is developed and supported by a couple of hackers in France in their spare time, so MP4 should offer slightly better compatibility than MKV. However, this is purely speculation, and I don't know of any software that works with MP4 but not MKV.

What app do you use for the frame-by-frame checking to see where TIVTC needs to be applied?

Just open an AVIsynth script in Virtualdub?

Not Virtualdubmod or Virtualdub2 or TheVirtualdubStrikesBack or MyOwnVirtualDubWithBlackjackAndHookers...

If 86% film, then that 86% is treated as if you used Force Film in DGIndex (giving you a progressive 23.976 video for those portions) and the rest undergoes a full IVTC (field-matching and decimation) with the whole thing winding up as progressive 23.976fps... If there's any real interlaced video or any progressive 29.97fps portions, that'll throw a monkey wrench into the whole procedure.

Ummm... are you assuming that the episode was encoded at 86% soft pulldown and 14% hard pulldown, just for shits and giggles? Because the DVD authors wanted to screw with us? I mean, that's not a totally insane assumption, and I've seen it before in Farscape and a few episodes of Captain Planet. It just happens to not be true in this particular case. The "real interlaced video or any progressive 29.97fps portions" make up almost the entirety of the other 14%.

Attempting to convert the whole episode to 23.976 is a very bad idea.

Let me break down exactly what I'm doing:

1). Rip the DVD in DVD Decrypter. This creates the following directory and populates it with VOB files: C:\DS9S6D2\VIDEO_TS
2). Load the VOB file into DGIndex. Confirm that DGIndex is set to honor pulldown flags.
3). Run the project through DGIndex. This creates a D2V file as output.
4). Load the D2V file into StaxRip. This opens the project file using MPEG2Source.

All right, cool. This is a pretty sane and simple workflow that we can replicate for the purpose of diagnosing any errors you encounter.

LoadPlugin("C:\Path\DGDecode.dll")
MPEG2Source("C:\Path\VTS_2_1.d2v")

Why are you manually loading an autoload plugin?

Can I rely on MPEG2DecPlus64.dll to provide equivalent MPEG2 decoding to DGDecode? I would prefer to use an x64 DLL if at all possible. If I need to specifically drop back to 32-bit for everything to make DGDecode work and there's no functional x64 equivalent, I can bring up a new system tomorrow.

Why are we now screwing around with 64-bit anything?

manono

22nd May 2020, 06:38

Ummm... are you assuming that the episode was encoded at 86% soft pulldown and 14% hard pulldown, just for shits and giggles?
As I've not seen so much as a sample, maybe it's just wishful thinking. However, I have seen combinations of hard and soft pulldown like that much more often than I've seen soft pulldown mixed with video at similar percentages.

Attempting to convert the whole episode to 23.976 is a very bad idea.
Then you make the case for bobbing it and calling it a day. That would be a helluva lot easier, that's for sure. Either that or doing a VFR encode, something for which I, personally, have no use.

Why are you manually loading an autoload plugin?
I do that myself. For one thing, it keeps a screwed up plugins folder from also screwing up your scripts, things such as multiple and conflicting versions of the same filter. Like you, I also thought it screwy that he was trying to do everything in 64-bit.

Katie Boundary

22nd May 2020, 07:36

UPDATE: I have independently confirmed the existence of at least one instance of retrograde field behavior: when the "Star Trek - Deep Space Nine" logo first fades in near the beginning of the opening credits. In addition, I'm having a hard time finding a combination of blockx and MI settings that will stop TFM from incorrectly flagging large amounts of progressive content as interlaced while still catching real interlaced content. The penalty for this is high - it causes the stars to blink in and out during space shots. I may switch to recommending blending instead of interpolation. I'll keep you all updated.

EDIT: for the sake of fairness, I tested QTGMC's performance as well. One of these images is my traditional TFM+Yadifmod2+NNEDI3 method. One of them is a variation of my method, using TFM+"Lossless" QTGMC. One of them is a variation of my method with TFM set to blend instead of bobbing/interpolation. Can you guess which is which?

https://i.imgur.com/oII2fLy.png

Finally, I recommend adding d2v="601.d2v",flags=1 to the TFM settings in both the C and D lines. This will prevent TFM from checking for interlacing during any soft pulldown, which cuts the false positive rate dramatically. The scene that benefits from this the most, by far, is the close-up of the comet's tail.

EDIT #2: Get ready because this next script is going to REALLY blow your damn minds...

Mpeg2source("601.d2v")

A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="601.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="601.d2v",flags=1,micmatching=0)

interleave(C,D)

Film content is, as usual, properly field-matched and then left the goddamn hell alone. Orphaned fields get blended with their nearest matches, then vertically sharpened to undo the resulting softness. This prevents the image from suddenly getting blurrier and sharper when transitioning between film content and interlaced content.

It will, of course, still spoil any true 60 hz content by creating ghosting and cutting the effective frame rate, but I think that's a fair price to pay here.

hello_hello

22nd May 2020, 09:15

I will be running DS9 tests soon, but for now, I'm still using Andromeda as my guinea pig for various deinterlacing methods. Just to be ABSOLUTELY sure about the relative merits of NNEDI3+Yadifmod2 versus QTGMC, I put the following script together:
(I kept the manually specified field arguments instead of using the complementparity approach because I know the field parity already)

This was the result:

It seems you found a frame where yadif/nnedi3 looks better while using ridiculously low threshold settings and sourcermatch etc for QTGMC. Well done. Here it is again, same TFM settings, no sourcematch or lossless for QTGMC.
Similar aliasing, and a tad sharper than Yadif/nnedi3.
Most of the time when QTGMC looks less sharp it'll be because it's removed noise, as it did for the example I posted earlier.

https://i.postimg.cc/hzjrcp2n/QTGMC.png (https://postimg.cc/hzjrcp2n)

No, you posted encodes using a lobotomized script that I provided for the purpose of demonstrating where Complementparity() would go. HUGE difference.

So I'll ask again. What difference would it have made using a non lobotomised version? Would TFM have taken more pixels from nnedi3 or less? Where's the sample of how you'd do it to show it's improved?

All of my ACTUAL methods use micmatching=0 and about a dozen other settings that I cut out of that particular bit of code because, as I stated, the complementparity demonstration code was NOT meant for actual use.

And I already said micmatching=0 would have fixed a glitch towards the end. That's it.
Lets be fair though and check out the huge difference you can expect to see with Katie's TFM settings vs the defaults, with the exception of micmatching=0 both times. I'll confess I've only checked the same frame again. I'm sure you'll be able to post screenshots where your settings are a vast improvement, won't you?

Katie TFM, Yadif/nnedi3, vs Default TFM, Yadif/nnedi3.
http://www.framecompare.com/image-compare/screenshotcomparison/77DPLNNX
I assume there's a difference, as the png files are slightly different sizes.

Are you not answering my question as to how you decimate for a VFR due to a lack of reading skills or because you're ignoring it?

All of my ACTUAL methods use micmatching=0 and about a dozen other settings that I cut out of that particular bit of code because, as I stated, the complementparity demonstration code was NOT meant for actual use.

So I'll ask again. What difference would it have made. Shouldn't you check a non lobotomised version is going to look better before complaining? Where's your sample encode?

manono

22nd May 2020, 09:18

Finally, I recommend adding d2v="601.d2v",flags=1 to the TFM settings in both the C and D lines. This will prevent TFM from checking for interlacing during any soft pulldown, which cuts the false positive rate dramatically. The scene that benefits from this the most, by far, is the close-up of the comet's tail.
Stereodude said something similar a couple of pages ago, except he likes Flags=5:

The problem is that TFM sees combing where there is none and deinterlaces large areas of the frames despite it being progressive with no real combing. 5 puts an end to that. I didn't seen any examples of combing (except in the occasional scene change) which 5 still checks and addresses.

hello_hello

22nd May 2020, 10:30

UPDATE: I have independently confirmed the existence of at least one instance of retrograde field behavior: when the "Star Trek - Deep Space Nine" logo first fades in near the beginning of the opening credits.

I can't see it. It looks fine to me if I change the field order.

N=Tfm(field=0,mode=0,micmatching=0)
O=Tfm(field=1,mode=0,micmatching=0)
Interleave(N,O)

I've no idea how you make your field order work at all for the DS9 sample.

Film content is, as usual, properly field-matched and then left the goddamn hell alone. Orphaned fields get blended with their nearest matches, then vertically sharpened to undo the resulting softness. This prevents the image from suddenly getting blurrier and sharper when transitioning between film content and interlaced content.

The comet tail is noticeably mushy. The CGI shot just before it switches to film has horrible quality pumping. The quality pumping seems worse in general. The sharpening makes the aliasing more noticeable.
I'm looking at an encode using your exact settings, including the field order. I don't think switching the field order to stop the frames bouncing back and forth would change that. Edit: I checked and it didn't.

I assume the transitions between shots have been de-interlaced by TFM and treated as video, so the transitions need to be kept at 29.97fps to stay smooth for a VFR. How do you do about picking them out if you decimate?

Katie Boundary

22nd May 2020, 10:31

All right, time to feed the troll one last time.

Are you not answering my question as to how you decimate for a VFR due to a lack of reading skills or because you're ignoring it?

I'm ignoring it because I don't give a tenth of a rat's ass about VFR, and have been advocating against VFR from the very beginning.

So I'll ask again. What difference would it have made.

The difference is that one is my method and the other isn't.

And now, back to serious discussion:

Stereodude said something similar a couple of pages ago, except he likes Flags=5

Flags=1 is even stricter than 5, because 5 will look for scene changes and then perform deinterlacing around those, whereas 1 will not look for interlacing in film content ever. If there are any orphaned fields around scene changes, then there will also be an interruption in the pulldown pattern, and TFM will start checking for interlacing again anyway. I'm therefore not sure what 5 would accomplish other than getting false positives around scene changes.

hello_hello

22nd May 2020, 11:23

All right, time to feed the troll one last time.

Ah.... a Katie classic. When the discussion doesn't go her way, out come the troll accusations. Why not participate in a discussion like an adult?

I'm ignoring it because I don't give a tenth of a rat's ass about VFR, and have been advocating against VFR from the very beginning.

You've advised JoelHruska to output MP4 for a VFR. Where's the rest of the workflow? I'm sure you've even mentioned doing it yourself.

The difference is that one is my method and the other isn't.

So lets ignore the screenshot showing the lack of difference shall we? And lets avoid explaining any specific differences while you're at it.

And now, back to serious discussion:

Are you leaving the thread, or planning to finally upload a sample encode?

videoh

22nd May 2020, 11:56

If I need to specifically drop back to 32-bit for everything to make DGDecode work and there's no functional x64 equivalent, I can bring up a new system tomorrow. Don't you know that you can run 32-bit stuff on a 64-bit system? Or did I misunderstand you?

hello_hello

22nd May 2020, 12:25

I do that myself. For one thing, it keeps a screwed up plugins folder from also screwing up your scripts, things such as multiple and conflicting versions of the same filter. Like you, I also thought it screwy that he was trying to do everything in 64-bit.

Not to mention that most GUI's have their own collection of plugins and load them automatically in scripts so they don't require Avisynth to be installed. The newer StaxRip comes in a 64 bit flavour only, with Avisynth and plugins to match, as far as I know, so I assume that's why he needs 64 bit plugins.

There's a 64 bit version of DGDecode here (http://avisynth.nl/index.php/AviSynth%2B_x64_plugins).
I know nothing about MPEG2DecPlus. I don't know which one StaxRip uses.
I guess videoh would know more about the different flavours.

Then you make the case for bobbing it and calling it a day. That would be a helluva lot easier, that's for sure. Either that or doing a VFR encode, something for which I, personally, have no use.

What's the disadvantage of VFR?
Avisynth outputs the average frame rate when you use TFM for the process, so you can still apply filtering to the output using a program that only supports a CFR if need be. It doesn't become VFR until the final encoding/muxing stage.

videoh

22nd May 2020, 12:36

manono didn't say there were disadvantages, only that he has no personal use for it.

hello_hello

22nd May 2020, 12:51

manono didn't say there were disadvantages, only that he has no personal use for it.

His comment implied a bobbed CFR output suits him better, and therefore logically offers some advantage, even if it's only specific for his needs, so I found myself wondering what that might be.
Of course he may do something entirely different again, such as always converting to a 23.976 or 29.97 CFR, but as his comment implied otherwise, I thought I'd ask.

PS In case you missed the edit of my previous post, can you provide any insight regarding the different flavours of 64 bit DGDecode?

huhn

22nd May 2020, 13:26

Boulder

22nd May 2020, 13:50

NTSC VFR is a problem if you playback content using Kodi.

Katie Boundary

22nd May 2020, 14:49

Don't you know that you can run 32-bit stuff on a 64-bit system? Or did I misunderstand you?

Joel writes for ExtremeTech about such topics as microwaves with built-in Wi-fi and smart toilets with personal butthole recognition (https://www.extremetech.com/extreme/308979-actual-internet-of-shit-smart-toilet) so I assume he's aware that 64-bit versions of Windows have a 32-bit application compatibility layer.

if you bob it you should do 60 fps.
if you have a problem with 3:2 judder in the source then you are totally doomed with 30 fps because it's not even possible to get a 3:2 judder in 30 progressive with 24p source you get something far worse. in fields is 4:2:2:2.

I've never found either to be noticeable, but it may be moot. If Joel goes with my blend approach, then there won't be any true 60 hz content left, and he might be able to safely decimate down to 48 without losing any unique frames.

hello_hello

22nd May 2020, 15:04

if you bob it you should do 60 fps.
if you have a problem with 3:2 judder in the source then you are totally doomed with 30 fps because it's not even possible to get a 3:2 judder in 30 progressive with 24p source you get something far worse.
in fields is 4:2:2:2.

Doesn't that depend how it was telecined?

ie a 3:3:2:2 pattern vs 3:2:3:2

https://en.wikipedia.org/wiki/Telecine#2:3_pulldown

This results in a 3:2 pattern using a progressive source.

AssumeFrameBased
SeparateFields
SelectEvery(8, 0,1, 2,3,2, 5,4, 7,6,7)
Weave
Yadif(mode=1)

videoh

22nd May 2020, 15:09

can you provide any insight regarding the different flavours of 64 bit DGDecode? I haven't looked at any of them or tried them until now. I just now played around with the one here: https://github.com/Asd-g/MPEG2DecPlus (version 8). It seems to work very fine and I am impressed by the amount of work put in to create it. Bravo Chikuzen and others!

huhn

22nd May 2020, 15:09

hello_hello

22nd May 2020, 15:18

I've never found either to be noticeable, but it may be moot. If Joel goes with my blend approach, then there won't be any true 60 hz content left, and he might be able to safely decimate down to 48 without losing any unique frames.

I suspect he'll see it looks horrible. And probably wonder why the picture jitters back and forth.

PP=5 isn't blend de-interlacing as such.

Some of the CGI sections need to be treated as video to make them smooth, during fades between shots, or probably during some text overlays, even though they're technically hard telecine. That's exactly what your method is designed to achieve isn't it?
You can't bob them to 59.94fps and decimate to 48fps. Wouldn't that be much the same as de-interlacing them to 29.97 fps and decimating to 23.976?

Have you tried decimating your "blend approach" to 48fps to see how it looks?

hello_hello

22nd May 2020, 15:22

doesn't matter if the end product is progressive.

these 3:3:2:2 stuff is in field don't forget in progressive we don't have fields so they have to be a power of 2.

3:2 judder in 720p60 is a used term but it is meant in progressive frames.

I don't follow. Did you see the example in my previous posts after I edited it?

I agree bobbing telecine to 60fps will generally produce a pattern of 3 and 2 repeated frames, I'm just not sure how often it'd result in 4:2:2:2. Unless I've misunderstood what you meant.

huhn

22nd May 2020, 16:07

i'm talking about 24p in 30p not in 60p.

hello_hello

22nd May 2020, 16:08

Ah.....

JoelHruska

22nd May 2020, 17:53

Katie,

I don't know of any software that works with MP4 but not MKV.

At this point in time, DaVinci Resolve / Resolve Studio by BlackMagic will not import MKV files. It will import MP4 files (and M4V files) but not MKV. It also will not work with MPEG-2 in any format or container. It's surprisingly limited in terms of format support.

JoelHruska

22nd May 2020, 18:03

Don't you know that you can run 32-bit stuff on a 64-bit system? Or did I misunderstand you?

If you can explain to me how to prevent "Cannot load a 32-bit DLL in a 64-bit version of AviSynth" errors, I'm all ears. The reason I look for 64-bit DLLs is because StaxRip installs the 64-bit version of AviSynth+ and requires them. Attempting to load 32-bit DLLs into 64-bit AviSynth produces the following error. As I said earlier in the thread, If I need to bring up a specific 32-bit workflow without StaxRip, I'll do it -- but I'm going to use a different system for it rather than unwinding the workflow I've been implementing here.

More generally, yes, I am familiar with WOW64.

huhn

22nd May 2020, 18:22

by installing avisynth 32 bit it doesn't effect your current work flow it even has it's own plugin folder for obvious reasons..

JoelHruska

22nd May 2020, 18:29

Manono,

I'm happy to provide clips, but I'm not sure there's a point. I've been using QTGMC to fix a 23.976 fps version of the show. The issues that I would want help with -- the issues I've already spent three months trying to repair -- are all in that version of the show. I have spent zero time with the 29.97 fps footage. I've spent hundreds of hours with the 23.976 fps footage in Sacrifice of Angels (the only episode of the show I've worked on to any significant degree besides Emissary). I am aware of all of the locations in SoA where there is evidence of 3:2 pulldown or single interlaced frames when the content is played back in a VFR MKV created via MakeMKV and/or when it's processed through Handbrake, but the content encoded at 29.97 fps is alien to me and I have attempted no method of processing the footage in the first place.

Effectively, this means starting over from scratch. I need to examine how QTGMC now treats 29.97 fps. I need to learn how TIVTC treats 29.97 fps. I need to learn how to use TIVTC in the first place, because I currently don't know at all. And I figure I'll start by running various scripts people have recommended here over the past 19 pages and see what all of the output looks like.

I haven't decided if I am starting over from scratch, but I won't be able to make that decision until I've wrangled what video output looks like here. I'm moving this upcoming week and prepping for it will take most of my time between then and now, but I'll get some tests running. Right now, I have no clips to show that would represent particular problem areas. I don't know enough about what doesn't work yet to have problem areas.

JoelHruska

22nd May 2020, 18:30

by installing avisynth 32 bit it doesn't effect your current work flow it even has it's own plugin folder for obvious reasons..

I suppose I could install AviSynth+'s 32-bit version. Can't install AviSynth and AviSynth+ on the same system, however.

videoh

22nd May 2020, 18:31

by installing avisynth 32 bit it doesn't effect your current work flow it even has it's own plugin folder for obvious reasons.. Yes, of course, as huhn perspicaciously points out, you can have both 32-bit and 64-bit Avisynth installed. Not necessarily suggesting you go to 32-bits, just responding to your initial suggestion that you would need to bring up a 32-bit system.

I suppose I could install AviSynth+'s 32-bit version. Can't install AviSynth and AviSynth+ on the same system, however. Respectfully, that doesn't seem to make sense.

JoelHruska

22nd May 2020, 18:59

Videoh,

If you attempt to install StaxRip with Avisynth installed, you will receive an error message telling you that you cannot have AviSynth and AviSynth+ installed on the same machine. Perhaps you can, and the software merely tells people not to do this to limit problems. Since one of the most common problems people have with AviSynth is keeping 32-bit and 64-bit plugins straight (based on messages I've read), and since I was using 32-bit AviSynth and StaxRip installs 64-bit AviSynth+, I swapped from one to another. You'll have to forgive me for believing a literal "You cannot and should not attempt to configure both of these programs on a system simultaneously" warning.

You admitted that you are a noob, so clips allow us to duplicate/verify your noobish claims. All clear?

Right. And had you only continued reading past that single sentence, you would have read my explanation for why I do not have clips to share. This type of response contains no useful information whatsoever, ignores the explanation I already wrote, and then attempts to mock me by emphasizing how new I am. As conversational tactics go, it's kind of a dick thing to do, and I don't know what kind of performance or rise you think you're going to get out of me by being a jerk.

What I wrote amounts to: "I have no claims to make and no pain points identified, because I need to reevaluate all my footage."

What you wrote amounts to: "I will ignore everything you actually said in your response so I can demean you by emphasizing a point you've already acknowledged."

What did that get you, exactly? Do you feel better? Do you feel bigger?

I'm not playing this game with you. And I'm not impressed.

huhn

22nd May 2020, 19:33

If you attempt to install StaxRip with Avisynth installed, you will receive an error message telling you that you cannot have AviSynth and AviSynth+ installed on the same machine.

excuse me?

Katie Boundary

22nd May 2020, 19:43

At this point in time, DaVinci Resolve / Resolve Studio by BlackMagic will not import MKV files. It will import MP4 files (and M4V files) but not MKV.

Well there ya go! :D

It also will not work with MPEG-2 in any format or container.

MPEG-2 support is a bit weird in general because I think some parts of it are still under patent, whereas MPEG-1 and MPEG-4 are "open". That's why, for example, XviD (which is MPEG-4) is a thing that you can use for free, and Tsunami can export to MPEG-1 for free, but Tsunami only gives you a 30-day trial for MPEG-2 encoding, after which you have to pay to keep using it. But I could be wrong on that.

StaxRip installs the 64-bit version of AviSynth+

Oh wow, okay, yes, that could definitely be a problem. And yeah, 32-bit AVIsynth requires 32-bit plugins and 64-bit AVIsynth requires 64-bit plugins. That's the main reason why I stick with 32-bt AVIsynth. All the 64-bit plugins have 32-bit versions, but there are tons of 32-bit plugins without 64-bit ports.

This is why I use plain old original 32-bit AVIsynth, not "AVIsynth 2: Electric Boogaloo", or "The AVIsynth Strikes Back", or "My Own AVIsynth with Blackjack and Hookers"...

I've been using QTGMC to fix a 23.976 fps version of the show. The issues that I would want help with -- the issues I've already spent three months trying to repair -- are all in that version of the show. I have spent zero time with the 29.97 fps footage. I've spent hundreds of hours with the 23.976 fps footage

The 23.976 fps footage doesn't need to be "fixed". It's already perfect. The only thing that QTGMC can do to it is screw it up. If you want to play around with a version of the episode where the 23.976 content is perfectly preserved and needs no processing or even restoration whatsoever, and you don't give a damn what happens to the 29.97 footage, then just run DGindex in Forced Film mode, load the d2v file into AVIsynth, and... and nothing. That's it. You're done. There are also several methods for perfectly restoring it after it's been through Honor Pulldown Flags, but none of those methods involve QTGMC.

I need to learn how TIVTC treats 29.97 fps.

That depends entirely on what's actually in those 29.97 frames. If it's hard pulldown, then TIVTC (or really any kind of IVTC) will fix it right back up. If it's some mixed-framerate crap or true 60 hz content, then IVTC will do very horrible things to it that you don't want to see.

I need to learn how to use TIVTC in the first place

It's pretty easy.

tfm().tdecimate()

If you want to save a few CPU cycles and reduce your chances of inexplicably and obviously wrong behavior, then it's:

tfm(mode=0,pp=0,mchroma=false,micmatching=0).tdecimate()

(and those should REALLY be tfm's default settings)

had you only continued reading past that single sentence... This type of response contains no useful information whatsoever, ignores the explanation I already wrote, and then attempts to mock me by emphasizing how new I am... it's kind of a dick thing to do, and I don't know what kind of performance or rise you think you're going to get out of me by being a jerk.

What did that get you, exactly? Do you feel better? Do you feel bigger?

I'm not playing this game with you.

Be careful. If you continue to point out the dickish and/or unhelpful behavior of certain people on this forum, they'll accuse you of having a "bad attitude" every time they try to "help" you :)

Let me know when you get a chance to try my latest blend/converttorgb/convolution/converttoyv12 approach. I think you'll be blown away by the image quality, even if the frame rate goes to hell in what used to be the 60 hz parts.

hello_hello

22nd May 2020, 19:48

I'm happy to provide clips, but I'm not sure there's a point. I've been using QTGMC to fix a 23.976 fps version of the show. The issues that I would want help with -- the issues I've already spent three months trying to repair -- are all in that version of the show. I have spent zero time with the 29.97 fps footage. I've spent hundreds of hours with the 23.976 fps footage in Sacrifice of Angels (the only episode of the show I've worked on to any significant degree besides Emissary). I am aware of all of the locations in SoA where there is evidence of 3:2 pulldown or single interlaced frames when the content is played back in a VFR MKV created via MakeMKV and/or when it's processed through Handbrake, but the content encoded at 29.97 fps is alien to me and I have attempted no method of processing the footage in the first place.

It's not that complicated.
TFM() both field matches film and de-interlaces video (29.97fps).
The film sections end up 29.97fps with one frame in a group of five being a duplicate, and the video is de-interlaced to 29.97fps. "Video" in this context means interlaced, as each field is a unique moment in time. You can't just combine them for a progressive frame.

In the case of the DS9 sample we've been playing with, there's hard telecined CGI that fades from one shot to the next, but the telecine pattern overlaps. Due to the pattern of combed frames changing, TFM can detect the overlaps is interlaced instead of telecine and de-interlace them rather than field match (de-interlacing means it combines the frames while removing the combing). That's a good thing as makes the transitions smooth, potentially at the expense of a little blurring due to the de-interlacing, but what was technically telecined film becomes progressive film with tiny sections of 29.97fps progressive during the fades between shots.

TDecimate is then mostly to used convert the 29.97fps TFM output to 23.976 by removing the duplicate frames in the film sections. For the video sections it removes frames using frame blending to keep the motion smooth. That's Hybid=1.
Hybrid=3 does it the other way around and removes the duplicate frames from the film sections, then adds frames using frame blending to make the motion in the film sections look smooth at 29.97fps and the "video" frames are untouched.
Without a Hybrid mode, TDecimate assumes everything is film and removes one in five frames, making any 29.97fps video sections "choppy".

VFR mode decimates the film sections to 23.976fps while keeping the video sections at 29.97fps. It's VFR but effectively a combination of two constant frame rates. That's similar to what Handbrake does for a VFR output.

Katie's method works by combining two instances of TFM matching different fields, restoring the original frames in the telecined sections (field matching), duplicating them for 59.94fps, and deinterlacing the video sections to 59.94fps. The transition sections during the fades in the CGI probably end up as 59.94fps progressive. The film sections have frames that repeat in a pattern of two frames, then three frames etc. Any "video" interlaced sections probably have unique frames at 59.94fps progressive.

That's how the original video should display on a 60Hz display. The 23.976fps film sections are field matched and decimated (for hard telecine), with the film frames displaying for two screen refreshes, then three, then two etc. The video frames display for a single refresh. The end result, on a 60Hz display, should look virtually identical to a VFR encode. On a display with a different refresh rate, not so much, and Katie's yet to explain how to take the 59.94fps output and decimate it to a VFR combination of 23.976fps and 59.94fps, assuming that's the end goal, athough she's mentioned it's an option several times.

Effectively, this means starting over from scratch. I need to examine how QTGMC now treats 29.97 fps. I need to learn how TIVTC treats 29.97 fps. I need to learn how to use TIVTC in the first place, because I currently don't know at all. And I figure I'll start by running various scripts people have recommended here over the past 19 pages and see what all of the output looks like.

QTGMC on it's own, de-interlaces the 29.97fps interlaced video sections by expanding each field to a full frame, making it progressive at 59.94fps. It's not that simple, but that's the upshot. QTGMC(FPSDivisor=2) does the same, but only outputs every second frame for 29.97fps instead of 59.94fps.

It's yet to be shown any purely interlaced content exists in DS9 anyway.

For the film sections, on it's own QTGMC does exactly the same thing, expanding the telecined fields to full height and you end up with the same pattern of repeated frames (2:3). The fades in the CGI sections would be treated the same way and therefore look quite smooth.
For film though, it's a bit like taking a progressive frame, separating the fields and then expanding them to full height, doubling the frame rate, and making a duplicate for each original frame as a result. It's better to keep the original progressive frames at the lower frame rate, but for combinations of telecined film and video, simply bobbing the whole lot to 59.94fps is an option. In the case of DS9 though, which appears to be mostly telecined, it's probably better to use TFM to restore the original progressive frames. You then have to decide how to deal with any sections that are de-interlaced to 29.97fps in order to keep motion smooth. VFR, decimation using frame blending, or increasing the whole lot to 59.94fps by duplicating frames etc (effectively Katie's method).

The arguing over TFM's settings mainly come down to how to configure it, so it detects the film sections and field matches them, while de-interlacing any combed sections that can't be properly field matched, ensuring it makes the correct decisions as to what it should do, and how it should de-interlace (or fix any combing) when it needs to. If you increase the likelihood it'll detect combing it can de-interlace a progressive film frame unnecessarily, or decide a section of telecined film must be video, when really it's not. If it's combing detection isn't strong enough, it can leave combing unrepaired. The default settings work quite well for the DS9 sample. I just added micmatching=0 (read the help file) and changed it's de-interlacing method to one that works better with the CGI.

If you run QTGMC as a deinterlacer over the lot, bobbing it all to 59.94fps, there's not all that much to argue about.

Specifying the D2V file for TFM allows it to use the repeat flags in the source to treat soft telecine as film. The progressive frames are reconstructed according to the repeat flags in the source (I think). Specifying a D2V file also comes with new options to argue over, such as what TFM should do if it does detect combing in the soft telecined film sections, and whether it should even check etc.

huhn

22nd May 2020, 19:57

The film sections end up 29.97fps with one frame in a group of five being a duplicate, and the video is de-interlaced to 29.97fps.

well it sounds dumb but if you don't want the best result this can still be improved by running a hardware frame adaptive deinterlacer over the file instead of TFM. nvidia and intel do field matching when deint is used and you should end up with the far superior 3:2 judder (which i think is bad but what ever).

the question is how buggy are they these days. i know nvidia places the chroma wrong by half a subpixel and AMD uses bilinear bob now but no clue about intel.

in the past all 3 where really good at this job.

i did a fast test.
nvidia DXVA2 is still field matching fine and produces what i'm talking about but CUVID is not field matching for some reason...
intel quick sync field matches but destroys the levels. both didn't age very well but they didn't reach AMD levels yet.
on OLD nvidia card with windows 7 combined with very old driver could do the trick but that's more then i'm willing to test...

manono

22nd May 2020, 20:36

manono didn't say there were disadvantages, only that he has no personal use for it.
And KB put it a little more colorfully:
I don't give a tenth of a rat's ass about VFR, and have been advocating against VFR from the very beginning.
These days I don't see these mixed film/video television series' as I work on old films with constant framerates.

scharfis_brain

22nd May 2020, 20:48

If I might chime in here:

I'd personally would skip VFR / 120Hz, just for these reasons:

1) compatibility: a 59.94p video file (despite codec/container) will work on almost any given hardware out there.

2) smart TVs: newish TVs are capable to detect 3:2 pulldown, 2:2 pulldown and video almost instantaneously. Thus no 3:2 judder will occur. Additional frame interpolation might be done for the 3:2 parts by the TV on the fly, if the user has set it up.

3) retention of the original frame sequence/cadence: no messing around with frame timings. No false judder being introduced. Just sticking to field-matching and proper deinterlacing.

hello_hello

22nd May 2020, 21:01

My video card (Nvidia) doesn't display the short Andromeda sample Katie uploaded correctly. It takes a few frames to pick up the telecine pattern (probably because it fades to white between shots) and they display combed. From there it displays the telecined CGI with the overlaid text quite smoothly.
I think for that one, TFM required a bit of coaxing to get through the section with overlaid text smoothly, probably using settings I wouldn't use for the rest of it, but I'd have to check.

JoelHruska

22nd May 2020, 21:07

If you attempt to install StaxRip with Avisynth installed, you will receive an error message telling you that you cannot have AviSynth and AviSynth+ installed on the same machine.

excuse me?

I'm assuming you are objecting to the ambiguity in what I said regarding "installing" StaxRip. StaxRip, of course, does not have an installer. But StaxRip *does* have a set of filters, scripts, and tools that it ships with, and it ships with AviSynth+. After working with 32-bit AviSynth 2.6.0 for some time, I decided to test StaxRip. StaxRip requires AviSynth+ 64-bit.

https://forum.doom9.org/showthread.php?t=172068&page=3

"Stax76: "StaxRip from now on requires AviSynth+ 64-Bit."

When I ran the AviSynth+ 64-bit installer, it informed me I would have to remove my 32-bit AviSynth 2.6.0 installation due to non-compatibility. The text of the message is not ambiguous. A legacy installation can be preserved in case you wish to downgrade / uninstall later. It explicitly states that this is not equivalent to a side-by-side operational mode.

How to Repro This Warning

1). Download and install AviSynth 2.6.0
2). Download StaxRip 2.0.8-Stable
3). Navigate to the C:\Users\Username\Downloads\StaxRip-x64-2.0.8.0-stable\Apps\Installers folder.
4). Run the AviSynthPlus_3.5.0_20200302.exe file.
5). Observe the following message.

https://i.imgur.com/7olF7K6.png

If you're telling me that this is untrue and that AviSynth+ and AviSynth can co-exist just fine, that's great to know. But that's not what the AviSynth+ installer says.

manono

22nd May 2020, 21:08

Manono,

I'm happy to provide clips, but I'm not sure there's a point. I've been using QTGMC to fix a 23.976 fps version of the show.
This caught my eye before I saw the videoh curt response (but with humor - you took it the wrong way) and KB's more elaborate response.

I think you unwittingly provided the rationale for the need for samples. Other, more experienced eyes and AviSynth users might suggest a different and easier and much faster workflow. On the face of it, your reasons for using QTGMC on these film portions make no sense. But maybe there are other reasons for filtering some soft telecine.

One possible way is to return it all to 23.976fps via the script suggested by Stereodude earlier, and then replace the non-film portions later with something specially created to convert video to film. I'm referring to the idea (your idea?) of interpolating the video CGI portions to 119.88fps followed by decimating it to 23.976fps. Never know till you try. Replacing bad portions of a video with a better version is a simple matter. It's easy to use Trims (http://avisynth.nl/index.php/Trim) for the job. All that's if the series has to be made a constant 23.976fps. Again though, bobbing to 59.94fps is an easy solution. scharfis_brain (whom you don't know but who is greatly respected around here) also seems to be advocating for 59.94fps.

hello_hello

22nd May 2020, 21:08

well it sounds dumb but if you don't want the best result this can still be improved by running a hardware frame adaptive deinterlacer over the file instead of TFM. nvidia and intel do field matching when deint is used and you should end up with the far superior 3:2 judder (which i think is bad but what ever).

TFM outputting one duplicate frame in a group of five would look awful, but that's before TDecimate removes it for 23.976fps to give you the same 3:2 judder on a 60Hz display.

The problem is how to handle hybrid sources when re-encoding. Personally I prefer VFR most of the time.

Not a typical example, but I re-encoded some old Doctor Who NTSC DVDs a while back. The video sections had been converted to interlaced NTSC using one method (I can't remember what it's called) and the film sections converted with field blending. I pulled them apart, de-interlaced the video to 59.94fps with QTGMC, and deblended the film sections to 25fps progressive. It was a fair bit of work, and manually creating a timecodes file was a chore, but the end result was quite good.

Each to his own, but I haven't had any problem playing them yet, although I've probably only used my PC and the TVs built in media player.

JoelHruska

22nd May 2020, 21:33

This caught my eye before I saw the videoh curt response (but with humor - you took it the wrong way)

None of Videoh's responses to me have been intended with humor, as far as I can tell. He began by assuming bad faith when I joined the thread on page 6 and has not appreciably mellowed on that stance thereafter that I can see. I've got no quarrel with him or anyone here.

I am absolutely willing to test any script suggestion that anyone has. When I said I didn't see the point of posting test clips, I was only referring to test clips based on my own ideas. I'm currently out of ideas. :P

Also, Vdub2 question. Does anyone actually use Vdub2 for outputting video after editing, or do you just edit the project and then save the AVS script? If Vdub2 is used to output video, what export settings should I use?

Katie Boundary

22nd May 2020, 21:50

Digging though the old posts in this thread, I stumbled upon the "QIVTC" script by h_h. It had three obvious problems, all of which I've managed to fix: first, it assumed a field parity that would be wrong 99.999% of the time, so I added a modification that Stainless and I have been working on for my own scripts, which automatically detects and uses the correct field parity. Second, after going through all the trouble of using QTGMC's weird shitty pseudo-motion-interpolation to create something that might look like natively 60 fps content if you were stoned, it decimated down to 23.976 fps for no damn reason. I cut that line out. Third, it used QTGMC's "very slow" preset, which has an extra dependency that the other speed settings don't have and which QTGMC's own documentation admits is a waste of CPU cycles for almost no quality improvement. So I restored it to its default "slower" setting. The final QIVTC looks like this:

function QIVTC(clip Video)
{

underpants = GetParity(Video) ? 0 : 1

A = Video.Tfm(mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=underpants,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=underpants,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
QTGMC(InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

If the moderators would like to put that on the wiki and credit it to Katie Boundary, hello_hello, and StainlessS, the three of us would be honored (note: I'm also putting a package together of two other functions that I'd like credited to just me and Stainless when I'm ready to publish it, which should be very soon).

As for my opinion on the results of this corrected version, I have to say... it's a brave and probably clever attempt, but ultimately it fails to do what a deinterlacer is supposed to do: get rid of visible interlacing. This is why I set cthresh and mthresh so low. At this script's settings, lots of interlacing still slips through:

https://i.imgur.com/uKTWxto.png

That might not seem like a big deal at standard definition, but it'll play hell with AI upscalers. It also shares the same flaws as QTGMC: (a) it's slow, (b) it has lots of dependencies, (c) it bobs everything indiscriminately rather than preserving the original progressive frames whenever possible, and (d) it generates worse interpolation than other temporally-aware bobbers, then tries to hide the bad data by replacing all the good data around it (original scanlines) with more bad data. Basically it's just QTGMC with less aliasing but worse detection and removal of interlacing.

Also, since I'm not a QTGMC guru, I don't know exactly how the script is supposed to work, so all support, maintenance, and further improvement concerns should be directed to h_h and Stainless.

And KB put it a little more colorfully:

These days I don't see these mixed film/video television series' as I work on old films with constant framerates.

Paradoxically, newer TV shows also have constant frame rates because the rise of HDTV has made it necessary for TV shows to look good at multiple resolutions, thus putting an end to natively interlaced editing and content. This mixed-framerate, edited-on-tape crap was a fad that lasted less than 20 years, from the mid-'80s to the early 2000s.

If I might chime in here:

I'd personally would skip VFR / 120Hz, just for these reasons:

1) compatibility: a 59.94p video file (despite codec/container) will work on almost any given hardware out there.

2) smart TVs: newish TVs are capable to detect 3:2 pulldown, 2:2 pulldown and video almost instantaneously. Thus no 3:2 judder will occur. Additional frame interpolation might be done for the 3:2 parts by the TV on the fly, if the user has set it up.

3) retention of the original frame sequence/cadence: no messing around with frame timings. No false judder being introduced. Just sticking to field-matching and proper deinterlacing.

You forgot:

4) it looks the same on a 60 hz monitor anyway

I am absolutely willing to test any script suggestion that anyone has.

Mpeg2source("601.d2v")

A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="601.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="601.d2v",flags=1,micmatching=0)

interleave(C,D)

As usual, feel free to tweak the parameters if you think you can improve the interlacing-detection.

hello_hello

22nd May 2020, 21:51

Also, Vdub2 question. Does anyone actually use Vdub2 for outputting video after editing, or do you just edit the project and then save the AVS script? If Vdub2 is used to output video, what export settings should I use?

I've never used VD2 for x264 encoding, but you can open Avisynth scripts with it.

MeGUI is pretty easy, It's script creator comes with a preview so you can modify a script and preview the changes, or create the script yourself via other means, load it into MeGUI's video section and add it to the job queue. The encoder preset used for each job is the preset selected at the time the job is added to the queue. You can create your own.
For x264, you'd generally just pick a quality (CRF value) the slowest speed preset you can stand (I generally use slow or slower) and maybe select a tuning (I generally use the film tuning).
MeGUI also supports Xvid, X265 and a huffyuv lossless output (via it's pre-rending option).

For VD2, using the File/Save menu the selecting x264 as the compression method, the configuration looks like this. Is that what you meant?

https://i.postimg.cc/Whntz4BN/vd1.jpg (https://postimg.cc/Whntz4BN)

Or you can pick a lossless option supported by the program you're want to use to import the output. If you're opening and editing the source video rather than a script, but want to open the edited video in a script, you'd possibly do it that way.

https://i.postimg.cc/bS0rQYdR/vd2.jpg (https://postimg.cc/bS0rQYdR)

scharfis_brain

22nd May 2020, 22:03

You forgot:

4) it looks the same on a 60 hz monitor anyway

Nope: I left it out on purpose. I don't look at 60Hz screens anymore.

Groucho2004

22nd May 2020, 22:15

If you're telling me that this is untrue and that AviSynth+ and AviSynth can co-exist just fine, that's great to know.
Proof:
https://i.postimg.cc/mDgRCrYH/Image2.png

Katie Boundary

22nd May 2020, 22:19

Proof:
(image)

So AVIsynth and AVIsynth+ can coexist but only if one is 32-bit and the other is 64-bit?

Groucho2004

22nd May 2020, 22:21

So AVIsynth and AVIsynth+ can coexist but only if one is 32-bit and the other is 64-bit?Correct.

manono

22nd May 2020, 22:25

For VD2, using the File/Save menu the selecting x264 as the compression method, the configuration looks like this. Is that what you meant?
Yes, that's the impression I got - that he was asking how to encode using VDub2. Thanks for explaining it and with the pictures as well. First set Video for Fast Recompress, followed by Video->Compression where he chooses x264 8-bit. To save resizing he might choose SARs of 4:3 (if 4:3 and not 16:9).

After setting up the video encode the way he likes and if he wants an MKV out of it, go File->Save As->Matroska and let 'er rip.

JoelHruska

22nd May 2020, 22:28

Correct.

Then the AviSynth+ installer gives incorrect information, because I definitely was using 32-bit AviSynth and it still told me it could not be left installed. Noted.

Groucho2004

22nd May 2020, 22:30

Then the AviSynth+ installer gives incorrect information, because I definitely was using 32-bit AviSynth and it still told me it could not be left installed. Noted.
Just use the Universal Installer.

JoelHruska

22nd May 2020, 22:42

Just use the Universal Installer.

Until this moment, I was not aware one existed.

Groucho2004

22nd May 2020, 22:46

Until this moment, I was not aware one existed.You'll find it here (https://forum.doom9.org/showthread.php?t=172124). If you decide to use it I'd recommend the one with AVS+ 3.5.1.

JoelHruska

22nd May 2020, 23:03

Thank you very much. This looks like a nice way to increase flexibility in future.

hello_hello

22nd May 2020, 23:04

Digging though the old posts in this thread, I stumbled upon the "QIVTC" script by h_h. It had three obvious problems, all of which I've managed to fix: first, it assumed a field parity that would be wrong 99.999% of the time, so I added a modification that Stainless and I have been working on for my own scripts, which automatically detects and uses the correct field parity. Second, after going through all the trouble of using QTGMC's weird shitty pseudo-motion-interpolation to create something that might look like natively 60 fps content if you were stoned, it decimated down to 23.976 fps for no damn reason. I cut that line out. Third, it used QTGMC's "very slow" preset, which has an extra dependency that the other speed settings don't have and which QTGMC's own documentation admits is a waste of CPU cycles for almost no quality improvement. So I restored it to its default "slower" setting. The final QIVTC looks like this:

The field parity was correct for the source, unlike the scripts for this sample you've uploaded. Is this a sad attempt to distract everyone from that because you didn't notice?

Where did your "Converting boolean values to integers and other AVSI problems" thread go? Did you delete it?

You can do what you like with the function, but don't give me any credit.

The slower the QTGMC preset in progressive mode, the less likely it'll cause "wobbles" in straight lines and sharp edges. Whether "very slow" makes much difference I don't know. I used it because that's what JoelHruska had been using previously, and the idea was to stabilise the picture as much as possible. Obviously you're the QTGMC expert now, so you'd know though....

It decimated to 23.976 because the output desired at the time was 23.976, and it looked quite smooth at the frame rate.

That might not seem like a big deal at standard definition, but it'll play hell with AI upscalers. It also shares the same flaws as QTGMC: (a) it's slow, (b) it has lots of dependencies, (c) it bobs everything indiscriminately rather than preserving the original progressive frames whenever possible, and (d) it generates worse interpolation than other temporally-aware bobbers, then tries to hide the bad data by replacing all the good data around it (original scanlines) with more bad data. Basically it's just QTGMC with less aliasing but worse detection and removal of interlacing.

Complete nonsense. TFM field matches the film sections as it always would and only replaces the combed pixels by taking them from the de-interlaced clip in exactly the same way every other example of your method does.

Funnily enough, if lower TFM threshold settings caused it to take more pixels from the QTGMC de-interlaced clip you regularly claim not to like, despite all the evidence posted previously showing it can be better (which you've continually ignored) you're saying that would be better?

The final instance of QTGMC isn't de-interlacing, just smoothing the output.

Also, since I'm not a QTGMC guru, I don't know exactly how the script is supposed to work, so all support, maintenance, and further improvement concerns should be directed to h_h and Stainless.

After the long critique on what it's doing, you're now claiming not to know how it's supposed to work?

The idea was to use TFM to field match, but not de-interlace itself, then QTGMC de-interlaces those frames, the logic behind that being the frames being de-interlaced by QTGMC would always be the same frames TFM wants to use to replace the combed pixels later on. Whether it works better or not, I don't know. Other than that it's just another variation of your method, and so far still looks better than any script you've posted.

How about you upload a sample yourself instead of pontificating, so we can compare it to the sample encode I no doubt linked to when I posted the script?

This is why I set cthresh and mthresh so low. At this script's settings, lots of interlacing still slips through:

Nonsense. If anything, in hindsight, the combing detection was too sensitive. That was probably the cause of any picture wobbling, but it was an attempt to get through the fades between shots smoothly at 23.976fps, and I think it did that. I've gone off the idea now though, and I'd do a VFR encode instead.

Here's a script showing the pixels TFM is taking to repair combing.

Video = last
A = Video.Tweak(bright=200)
B = Video.Tweak(bright=200)
C = Video.Tfm(field=0,mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
https://i.postimg.cc/Vr0WPhkC/1.jpg (https://postimg.cc/Vr0WPhkC)

Here's what the frame you posted looks like using the settings from my script (or roughly, I don't know the frame number).
https://i.postimg.cc/XZzkhW4W/2.jpg (https://postimg.cc/XZzkhW4W)

Here's what it looks like using cthresh=2 and mthresh=2.

Video = last
A = Video.Tweak(bright=200)
B = Video.Tweak(bright=200)
C = Video.Tfm(field=0,mode=0, mthresh=2, Slow=2, CThresh=2, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, mthresh=2, Slow=2, CThresh=2, Clip2=B, Micmatching=0)
Interleave(C,D)
https://i.postimg.cc/ZWjc83QC/3.jpg (https://postimg.cc/ZWjc83QC)

The TFM settings from my script.
https://i.postimg.cc/gXbHdDXt/4.jpg (https://postimg.cc/gXbHdDXt)

cthresh=2 and mthresh=2.
https://i.postimg.cc/7C2npc6T/5.jpg (https://postimg.cc/7C2npc6T)

The TFM settings from my script.
https://i.postimg.cc/pmpdXMKy/6.jpg (https://postimg.cc/pmpdXMKy)

cthresh=2 and mthresh=2.
https://i.postimg.cc/N5GMSHM6/7.jpg (https://postimg.cc/N5GMSHM6)

Your most recent TFM settings aren't much different. They don't detect any combing in frame 4500 (the first screenshot I used as an example). Try it for yourself.

A = Tweak(bright=200)
B = Tweak(bright=200)
C=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=1,MI=30,blockx=4,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=1,MI=30,blockx=4,mthresh=2,clip2=B,micmatching=0)
interleave(C,D)
https://i.postimg.cc/sQkVXM92/8.jpg (https://postimg.cc/sQkVXM92)

How about a sample encode instead of posting nonsense?

manono

23rd May 2020, 00:43

Katie Boundary

23rd May 2020, 00:58

I DO leave him alone. Everything I've said in response to him has either been debunking his bullshit claims about me or an impersonal analysis of his claims about video editing. Not once in the history of this forum have I gone on the offensive against him.

Stereodude

23rd May 2020, 01:26

Complete nonsense. TFM field matches the film sections as it always would and only replaces the combed pixels by taking them from the de-interlaced clip in exactly the same way every other example of your method does.

Funnily enough, if lower TFM threshold settings caused it to take more pixels from the QTGMC de-interlaced clip you regularly claim not to like, despite all the evidence posted previously showing it can be better (which you've continually ignored) you're saying that would be better?

The final instance of QTGMC isn't de-interlacing, just smoothing the output.
Careful, that's really not how TFM works. TFM doesn't replace pixels. It replaces entire lines of pixels. The TFM help file is misleading IMHO. I thought it only replaced the small areas of the frame that have detected interlacing. Like it made a mask of the moving part and would leave the underlying image. I'm nearly certain that it replaces the entire line for any line that has has detected interlacing in it. So in content like this DS9 stuff it's basically replacing the entire frame even if you're using PP>=5.

It may even be replacing anything between the top most line and bottom most line that have detected interlacing in them.

Katie Boundary

23rd May 2020, 01:29

Careful, that's really not how TFM works. TFM doesn't replace pixels. It replaces entire lines of pixels. The TFM help file is misleading IMHO. I thought it only replaced the small areas of the frame that have detected interlacing. Like it made a mask of the moving part and would leave the underlying image. I'm nearly certain that it replaces the entire line for any line that has has detected interlacing in it.

:eek:

That would explain a LOT.

EDIT: I'm going through more of the DS9 footage and I'm extremely impressed by how much of this is straight film. The space battles, for example, are rendered at pure film; you don't have ships moving at 24 fps while firing phasers at each other that are animated at 30 fps. This is a huge departure from earlier seasons of Berman-era Star Trek, where you'd see, for example, the Enterprise-D moving at 24 FPS while stars whizzed by at 30. When Weyoun points at a screen showing some animation or another, the animation moves at 24 fps in sync with his hand, like that was an actual functioning display that they had on-set during filming and not something that was green-screened in later (as was done in Andromeda). Even the fades to and from black around commercial breaks are done at film. As far as I can tell, the only orphaned fields in this whole episode are during the opening and possibly closing credits (I haven't checked the closing credits yet), the credit fade-ins and fade-outs right after the opening credits, and around scene changes. A few frames of true 60hz content (i.e. "not text, not fades") do exist during the wormhole opening and closing at the end of the opening credits.

This episode would benefit strongly from a deinterlacing approach that favors preserving the original progressive frames untouched, even if it means reduced quality in the orphaned fields. This doesn't mean that a decimation down to 24 is safe, however. Those scene changes keep screwing up the cadence.

zapp7

23rd May 2020, 02:21

This episode would benefit strongly from a deinterlacing approach that favors preserving the original progressive frames untouched, even if it means reduced quality in the orphaned fields. This doesn't mean that a decimation down to 24 is safe, however. Those scene changes keep screwing up the cadence.

What do you mean by screwing up the cadence?

Katie Boundary

23rd May 2020, 02:37

What do you mean by screwing up the cadence?

Meaning it stops being a 3:2 pattern and any given group of five consecutive frames won't necessarily have exactly three frames that should be decimated and two that should be kept.

zapp7

23rd May 2020, 02:46

Meaning it stops being a 3:2 pattern and any given group of five consecutive frames won't necessarily have exactly three frames that should be decimated and two that should be kept.

I've scrubbed through about 18 episodes in the first season creating TFM override files for each, and while that does happen, it is very rare. When it does happen it's usually the first frame of the next scene that is problematic, and QTGMC handles it pretty well. The effect isn't even noticeable upon playback.

Katie Boundary

23rd May 2020, 02:48

No, it happens with practically every scene change.

zapp7

23rd May 2020, 03:02

We must not be talking about the same thing then.

hello_hello

23rd May 2020, 09:35

Can the two of you please stop? I've already asked hello_hello via PM to just leave you alone. I have no idea why he even persists. Now I ask you to leave him alone. Just ignore each other. Is it so hard? I thought he was on your ignore list. What happened? You just relished the thought of letting have it? Me, I'm easy, but you don't want Swede to make good on his threats.

By the way, both Swede and I can see what you wrote and then thought the better of it just above.

She went back through the thread, copied an old script I posted, then proceeded to post mostly incorrect nonsense about it. Why should I simply leave that uncorrected? Sometime in the future someone might think her nonsense should be taken seriously. If Katie copied something you'd posted and then proceeded to post incorrect assumptions about it, wouldn't you reply to correct her?

What did I write? I edited that post quite a few times to change and add things.
Edit: I've looked through the edits (I forgot you could do that) but I still can't see what you're referring to.

hello_hello

23rd May 2020, 09:48

Careful, that's really not how TFM works. TFM doesn't replace pixels. It replaces entire lines of pixels. The TFM help file is misleading IMHO. I thought it only replaced the small areas of the frame that have detected interlacing. Like it made a mask of the moving part and would leave the underlying image. I'm nearly certain that it replaces the entire line for any line that has has detected interlacing in it. So in content like this DS9 stuff it's basically replacing the entire frame even if you're using PP>=5.

It may even be replacing anything between the top most line and bottom most line that have detected interlacing in them.

If you look at the screenshots in this post (https://forum.doom9.org/showthread.php?p=1913232#post1913232) you'll see I've used a script to show which pixels are being taken from the de-interlaced clip. A simple version would be this:

DeintClip = Tweak(Bright=200)
TFM(Clip2=DeintClip)

Most of the screenshots just show it taking specific pixels, but one of them shows it's replacing the whole frame, somewhat inexplicably to my eye, but the screenshot immediately below that one shows that with slightly different settings, it's only taking specific pixels. I think it may relate to setting the MI argument too low, although I haven't tested that, but the screenshots show it must be replacing just the combed pixels, and probably a small area around them, most of the time.

hello_hello

23rd May 2020, 11:27

What do you mean by screwing up the cadence?

It means that normally for telecined content you can step through it and see a pattern of three clean frames and two combed frames repeating, but during the fades from one shot to the next, they've been overlaid without ensuring the pattern isn't interrupted. They're both still telecined with a pattern of 3 clean and 2 combed frames, but the clean and combed frames don't line up for each shot during the fades.

This is the sort of fade I'm referring to. I've made it brighter so it's easier to see. For the first screenshot, the shot that's fading in doesn't show telecine combing while the shot that's fading out does. A few frames later on it's the opposite.

https://i.postimg.cc/VS1VSQKq/A.png (https://postimg.cc/VS1VSQKq)

https://i.postimg.cc/nCy5JQ7w/B.png (https://postimg.cc/nCy5JQ7w)

Keep in mind though, DVDs were originally intended to be displayed on an interlaced display (CRT), so this sort of thing would display perfectly fine. For a progressive display, and when converting to progressive for encoding, it's a problem. If you simply keep decimating one in five frames for 23.976, one of the shots will probably have "jerky" movement during the fades.
One solution is to de-interlace those sections to remove the combing, and to keep them at 29.97fps. That way they'll look smooth, but the end result would be little sections of 29.97fps progressive amongst the 23.976 detelecined progressive frames, and you have to decide how to deal with it.

A player performing detelecine and deinterlacing on the DVD would probably treat those fades as video (29.97fps) and de-interlace them, effectively outputting a variable frame rate, which would look quite smooth.

Stereodude

23rd May 2020, 15:24

If you look at the screenshots in this post (https://forum.doom9.org/showthread.php?p=1913232#post1913232) you'll see I've used a script to show which pixels are being taken from the de-interlaced clip. A simple version would be this:

DeintClip = Tweak(Bright=200)
TFM(Clip2=DeintClip)

Most of the screenshots just show it taking specific pixels, but one of them shows it's replacing the whole frame, somewhat inexplicably to my eye, but the screenshot immediately below that one shows that with slightly different settings, it's only taking specific pixels. I think it may relate to setting the MI argument too low, although I haven't tested that, but the screenshots show it must be replacing just the combed pixels, and probably a small area around them, most of the time.
Interesting... I saw a number of jello like rolling shutter sort of artifacts that were caused by the deinterlacing of CG portions that were subsequently avoided with flags=5. It looked to be replacing some entire lines. Specifically in S06E01 at around 105 seconds in the Defiant as it flies toward the camera.

scharfis_brain

23rd May 2020, 15:57

hello_hello

23rd May 2020, 19:21

@hello_hello:

from a theoretical standpoint it sould be perfectly possible to retrieve a non-jerky 24p decimation out of two 3:2 cadences blended together.
For sure field matching will be broken. But the decimation of a 50:50 blend of two cadences should be (manually) possible.

I've got to say I'm extremely disappointed at being forced to evaluate my entire view of everything, but you're right.
There's been so much debate about how to handle this source that I've obviously became guilty of over-thinking it, so I went back to basics.

It appears TFM can sail through all the broken cadences on it own, except one. TFM().TDecimate repeats a frame, and I couldn't convince it not to, so I removed it manually.
As it's just the opening sequence, I waited until a fade to black to repeat a frame to make up for it.

As it turns out, disabling MicMatching doesn't even make a difference for the opening sequence as I'd come to believe it would. It does however, help with the progressive section I tacked onto the end of the sample for testing, which by the way is the only thing preventing me from wanting to stick my head in a hole somewhere now, because the existence of at least one progressive 29.97fps section still supports an argument for VFR encoding. :)

I'll upload a sample in a little while. There's still one section I'm not happy with at 23.976fps, so I'm working on it now.

hello_hello

23rd May 2020, 23:51

Here's the new encode of the opening sequence.

Clip "A" does normal field matching for the majority of the CGI and all the film sections.
Clip "B" is me being pedantic about making the movement of the stars smoother after the wormhole (behind the text). It's not smooth even if you bob it, so that's probably as good as it gets.

The first lot of Trims combine the two clips, deleting one repeated frame during a transition in the CGI, and repeating a frame during a fade to black to compensate.
The second lot of Trims are just used for optional filtering on the decimated video.

I couldn't find a happy that allowed TFM to de-interlace the text as it fades in/out without unnecessarily de-interlacing half the picture, so I left it, as I think it's the lesser of two evils. Personally, I don't care about it that much. Whether it's encoded as VFR or 23.976, or 59.94, I'm now of the opinion fussing over making sure TFM caught all the combing was heading in the wrong direction. Based on the following test, TFM isn't de-interlacing a single pixel in the whole sample. At least none that I saw.

DeintClip = Tweak(Bright=200)
TFM(PP=5, MicMatching=0, Clip2=DeintClip)

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

A = TFM(pp=5, MicMatching=0).TDecimate()
B = TFM(pp=5, y0=180, y1=300, MicMatching=0).TDecimate()

A.Trim(0,2118) + \
A.Trim(2120,2643) + \
B.Trim(2644,2799) + \
A.Trim(2799,0)

Smooth = \
"""QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)"""

Trim(0, 58).MCDegrainSharp() + \
Trim(59, 578) + \
Trim(579, 2971).Eval(Smooth) + \
Trim(2972,0).MCDegrainSharp()

Resize8(640, 480)

VTS_02_1_sample 23.976.mkv (https://www.sendspace.com/file/qf83vh)

Katie Boundary

24th May 2020, 00:05

I just realized that, since Joel is putting a GUIDE together and not distributing full video files, there's room for several different AVIsynth scripts and several different target frame rates, with notes on why people might want to select - or not select - each one.

I'd also love to see some examples of what happens when an interlaced frame gets put through AI upscaling.

Oh and I just noticed this article:

If you don’t want to run content at 29.97fps, there’s another option — re-encode it at 23.976fps. Convert the interlaced frames, lower the frame rate, and you’ll end up with imperfect, highly noticeable jerks and jumps during fast-paced space combat scenes — which are exactly the ones we are trying to preserve. This is known as judder. Judder sucks.

WHOA WHOA WHOA, no. That's NOT what judder is. Judder is when p24 content is displayed on a 60 hz device, or when telecined film content is bobbed or field-matched to p60, resulting in half of the frames lasting 1/20th of a second and the other half lasting 1/30th of a second, alternating, instead of each one lasting 1/24th of a second. It's not noticeable to humans, although some people claim it is. When you take higher-framerate (p30, i60) content and decimate it down to 24, that's called a "war crime".

Asmodian

24th May 2020, 03:49

While I do agree with most of your post I take exception to this:
It's not noticeable to humans, although some people claim it is.

I suspect you expected this feedback. ;)

All the things you care about yet you don't believe this is a problem? Humans cannot even see it? :p

Katie Boundary

24th May 2020, 04:02

If you're on drugs that have significantly altered your perception of time in such a way that the world around you seems to be moving in slow motion, like Fry in that episode of Futurama where he drinks 100 cups of coffee in a single day, then you might notice it.

JoelHruska

24th May 2020, 04:25

Per request, here are clips of specific scenes from Sacrifice of Angels from a VOB of the DVD. Clip explanations below. Clips are cut to 10 seconds or less per Manono's advice, unless it made no sense to split the scene at that point. Contiguous versions of two longer scenes (FFE = First Fleet Engagement, SFE = Second Fleet Engagement) are available below. I tried to start each clip with a few frames of padding on each side.

In most of the clips below, there are specific places I'm hoping to achieve better antialiasing results. I used QTGMC partly because it fixed some of the problems I'm going to describe below and would like to duplicate this (or improve upon the initial repair). I'm going to describe some specific problem spots I saw in the 23.976 version of this footage because it's all I have for reference and because I was attempting to repair very specific things in certain cases.

Apparently, I cannot offer a direct download link via SendSpace unless I have a premium account. A link to 13 of the 14 uploaded clips is here:

https://fs01u.sendspace.com/upload?SPEED_LIMIT=0&MAX_FILE_SIZE=314572800&UPLOAD_IDENTIFIER=1679361406.1590287955.42420C06.20.0&DESTINATION_DIR=5

You do not need to download "FirstFleeetEngagement-Complete" if you download "FirstFleetEngagement-Short1" through "Short3." The complete version of "SecondFleetEngagement" is only available on OneDrive below due to size restrictions. Files are abbreviated as "FFE" and "SFE" on OneDrive after I realized the file names were too long for easy display. The 14th file is "SFE-Complete" and it does contain a few scenes on the Defiant that are not embedded in SFE-1 -- SFE-5.

Alternately, if you want individual file downloads, you can use the OneDrive links below:

A link to the entire directory is here.

https://1drv.ms/u/s!AphTLFRW13WMjk7io5jlCJzdRO9I?e=3NcPQ3

Individual file links in-line below. Each individual file link will open a OneDrive link. OneDrive does *not* require you to have a login, and you can download by hitting the button in the upper-left-hand corner.

FirstDefiantFlyby:

https://1drv.ms/v/s!AphTLFRW13WMjlmk6jeSNMAA7rPM?e=UFmhtT

There's aliasing on both the Excelsior-class ships (inside the nacelles) and the Defiant's angle of attack in this scene is hard to fix. QTGMC against 23.976 fps did a very good job of resolving a troublesome flicker on the Defiant's left "shoulder" that gave me a lot of trouble, but I never had a great solution to the in-nacelle aliasing. Would like to nuke that from orbit.

FirstDominionFleet:

https://1drv.ms/v/s!AphTLFRW13WMjlKdB-r9XOdfALn3?e=RgO5tL

At the end of this clip there are three Jem'Hadar scarab fighters visible. All three have strong horizontal aliasing across their bottoms. I'd like to eliminate it as part of properly processing the scene. QTGMC using my repair script against a 23.976 MKV does a very good job of eliminating the horizontal lines.

SecondDefiantFlyby:

https://1drv.ms/v/s!AphTLFRW13WMjleXZXPXV9u0nKcK?e=mzSZ13

This is one where I've had trouble with some scripts causing backwards movement in the small fighters that fly across the screen. The background ships are basically fine no matter what you run against them.

First Fleet Engagement (FFE) 1-3

https://1drv.ms/v/s!AphTLFRW13WMjlTGczLl0T3HFrqU?e=mDKlE4

https://1drv.ms/v/s!AphTLFRW13WMjlHg_lI-zv5l2aGB?e=dfREGR

https://1drv.ms/v/s!AphTLFRW13WMjlYtPRhYSRJlENGV?e=yWZGR5

This scene hasn't been problematic, per se, but it's definitely one of the major CGI battle scenes of the episode, so it's included for those reasons. If you want to download the entire contiguous clip, you can do so here (100MB file).

https://1drv.ms/v/s!AphTLFRW13WMjlpFWKjMN_MpsSD_?e=gmrWr5

Second Fleet Engagement (SFE) 1- 5

https://1drv.ms/v/s!AphTLFRW13WMjljS5j6zP-Zek-we?e=nswJKR

https://1drv.ms/v/s!AphTLFRW13WMjlWMWPE-XeSI3x6M?e=DjUO8I

https://1drv.ms/v/s!AphTLFRW13WMjlMeGQWYCTBv6E3y?e=mhfNsh

https://1drv.ms/v/s!AphTLFRW13WMjlA2IsY_v3iH_ocz?e=YWsefX

https://1drv.ms/v/s!AphTLFRW13WMjk8D0hg-WBq98s-w?e=3ehQve

The second full fleet engagement. Major things I'm hoping to fix include nacelle-aliasing and the very aliased lines across the bottom of the Miranda-class vessel's ventral saucer. (SFE-1). The other scenes of SFE have been pretty solid, but are included here for completeness' sake. The SFE clips 1-5 omit the Defiant scenes that take place in-between the CGI shots. The full file, available here:

SendSpace is limited to 300MB, so if you want the full file, OneDrive is the only place to go.

https://1drv.ms/v/s!AphTLFRW13WMjlvPfszuhSSw7gN7?e=qOAQeq

Includes the Defiant interim scenes as well as the CGI scenes.

Trivia: According to the VFX team that built the show, this was the most-detailed fleet battle they ever shot. They brought in tactical experts specifically to help them design the fight. SFE is the arguable high-water mark for DS9's entire seven-year run, so if there was one CGI scene to absolutely nail, this one would be it.

Sisko-On-Defiant

Finally, I've uploaded a clip that I thought might be useful for seeing how DS9 mixes and matches VFR content together. All frame rate references measured using MPC-HC. The frame rate when the DVD is played back begins at 29.97 fps, but starts dropping immediately. By the time the shot switches to Defiant, the frame rate is 23.976 fps again -- but it starts increasing back to 29.97 fps when the Ferengi Nog appears on-screen. The clip stays at 29.97 (at least, MPC-HC reports 29.97 for several minutes thereafter).

https://1drv.ms/v/s!AphTLFRW13WMjl_bY-hbpvlTrpuU?e=5ETmw2

I have thought that this could be somehow related to the credits, but there's a later shot on DS9 itself that stays at 29.97 fps well after the credits are finished. I can clip that as well if people want to see it, but this seemed good for now.

Katie Boundary

24th May 2020, 04:53

The solution you your aliasing problems is to use a script that reconstructs the original progressive frames and then leaves them the goddamn hell alone.

BTW, I think you say "antialiasing" a lot when you mean "aliasing". Aliasing is bad. Antialiasing is good :)

JoelHruska

24th May 2020, 05:39

Fixed.

JoelHruska

24th May 2020, 05:46

When you take higher-framerate (p30, i60) content and decimate it down to 24, that's called a "war crime".

While it's obvious that you are far from the only person to have this opinion, I would prefer a better phrase when discussing 23.976 fps playback of p30/i60 fps footage than "war crime." Is there a specific term?

Katie Boundary

24th May 2020, 06:15

Kay, looks good :)

I've been running more tests and while I still think that both the interpolation and blend variations of my script gives better results than QTGMC in the vast majority of frames, there are still a few interlaced frames where even a straight blending script with no interlacing-detection (treats all frames and pixels as interlaced), no resharpening, and no other fancy stuff...

A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0)
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0)
interleave(A,B)

...STILL doesn't fully deinterlace:

https://i.imgur.com/BjSDFuh.png

...whereas QTGMC does:

https://i.imgur.com/d4u3JD2.png

Although this doesn't justify switching over to straight QTGMC, it does make me think that the best script for DS9 specifically might be a variation of my method that incorporates QTGMC. And because "lossless" QTGMC preserves the aberrant field order of the title at the beginning of the opening credits, but "normal" QTGMC smooths it out, this hybrid method would need to incorporate the "lossy" version of QTGMC. So, when you get a chance, try...

A=qtgmc().selecteven()
B=qtgmc().selectodd()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="601.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="601.d2v",flags=1,micmatching=0)

interleave(C,D)

A few fields around each scene change will get QTGMCed, but most of the frames in each shot will be perfectly restored and then left alone. For the opening credits, the comet's tail will be preserved perfectly because that part is encoded on the DVD as 24 fps progressive, but the rest of the opening credits will be 100% QTGMCed.

While it's obvious that you are far from the only person to have this opinion, I would prefer a better phrase when discussing 23.976 fps playback of p30/i60 fps footage than "war crime." Is there a specific term?

"looking like unmitigated shit" is another good one.

hello_hello

24th May 2020, 09:42

hello_hello

24th May 2020, 10:18

Damn, I just noticed an oddity in the film section towards the end of the Sisko-on-Defiant sample. It's possibly a field order thing that wouldn't have happened working with the original video, but here's the fixed version.

Sisko-on-Defiant take 2.mkv (https://www.sendspace.com/file/udnpv7)

LWLibavVideoSource("D:\Sisko-on-Defiant.mkv.lwi")
AssumeTFF()
Crop(8,0,-8,0)

TFM(pp=5, micmatching=0).TDecimate()

Trim(0,168).QTGMC(InputType=1, Preset="slower") + \
Trim(169,0).MCDegrainSharp()
Spline36Resize(640,480)

I'm not sure about MCDegrainSharp() for the film sections in every episode. That one seems a bit noisy and it's leaving too much behind for my taste, but anyway....

hello_hello

24th May 2020, 11:19

wonkey_monkey

24th May 2020, 12:17

when telecined film content is bobbed or field-matched to p60, resulting in half of the frames lasting 1/20th of a second and the other half lasting 1/30th of a second, alternating, instead of each one lasting 1/24th of a second. It's not noticeable to humans, although some people claim it is.

It absolutely is noticeable, it just doesn't necessarily jump out at you as a problem (especially to people who grew in NTSC-land) until you view both versions together.

StainlessS

24th May 2020, 12:50

The noise in the film sections is hard to deal with. It looks like the noise removal used previously stabilised what it didn't remove, so it doesn't behave like noise.
That sounds like you mean the noise is same across frames ie temporally same, maybe somebody added fixed pattern grain. [bad idea]

Katie Boundary

24th May 2020, 13:05

I just downloaded and played with Topaz for the first time and I have two things to say about it:

1) It only allows scaling by a single factor, applied to both width and height. It cannot change the aspect ratio. If you want to upscale a fullscreen show (like DS9 or Buffy) from 720x480 to 1600x1200, you're screwed. If you want to upscale a widescreen show (like Andromeda) from 720x480 to 1920x1080, you're screwed. You have to use a conventional resampler like Lanczos at some point to correct the aspect ratio. Or perhaps use a different AI upscaler.

2) Contrary to my expectations, interlaced text, when upscaled with AI, looks absolutely bitchin'. I wish there was a way to make all the text in the show look like this whether it's fading in/out or not. Here's a sample of upscaled interlaced text: https://i.imgur.com/cH4788X.png

JoelHruska

24th May 2020, 15:19

Are there any samples of the original video amongst that lot, or is it all re-encoded?

Every sample was created using the following method.

1). Rip episode with DVD Decrypter
2). Read D2V file into StaxRip to create an MP4 with no other scripting applied. Said MP4 created using quality 12.0, Preset Slow.
3). Clips created in VirtualDub2.

I should have loaded the D2V file directly into VirtualDub2. Truthfully forgot you could do that -- I was pretty tired last night after spending most of the day packing and schlepping. If you guys want to see what that output looks like, I can do that.

JoelHruska

24th May 2020, 15:39

KB,

TVEAI is in pretty rough shape at the moment. The audio pass-through is so bad, you're better off assuming the audio will not work and just planning to do your own remuxing. It encodes in the most basic MP4 profile settings imaginable and there is zero control over output quality in any regard.

You can only do lossless upscaling if you output to PNG or TIFF and I've seen some people say that some content doesn't look as good when pulled apart into individual frames and then stitched back together, though I have not tested that myself. I don't know which content is supposedly affected or what the differences are.

Also, the storage requirements get pretty ugly. It's 4-5MB per frame per PNG and about 16MB per 16-bit TIFF, so you need between 342GB - 1.216TB per episode for upscaling output, and you'd want an SSD for certain over an HDD for rapid file writes (when upscaling) and faster reads (when reassembling). At these data sizes and rates, the gap would be material. I think 16-bit TIFFs may be overkill when upscaling DVD content.

Emissary, Way of the Warrior, and What You Leave Behind would require double this storage requirement if you wanted to do both episodes in a single gulp. So ~644GB - 2.4TB to store the output frames from the entire episode.

I actually liked the 1.1.0 version more in terms of output quality. There's a way to get the older versions if you were interested in comparing the outputs. They are meaningfully different.

Katie Boundary

24th May 2020, 18:21

I hadn't even played with video modes. I was just using their single-image upscaler. For video, I would have certainly wanted to test output options to see if exporting XviD AVI was possible. What you're saying is that you have to export from Topaz as a series of PNG files, and then import that series of images into a program like Vdub to turn them back into a video and export with the settings of your choice?

SaurusX

24th May 2020, 18:26

Katie, you’re in luck. If you want video output from Topaz then you should know that all it exports is xvid. In order to use a modern codec you have to take the sequence of images route.

hello_hello

24th May 2020, 18:55

Every sample was created using the following method.

1). Rip episode with DVD Decrypter
2). Read D2V file into StaxRip to create an MP4 with no other scripting applied. Said MP4 created using quality 12.0, Preset Slow.
3). Clips created in VirtualDub2.

You can split the vob files themselves.
DGindex will output an mpeg video stream, MKVToolNix will split and remux as MKV, TSMuxer with open them and split and remux as TS files.

DGIndex is probably easier as it has a preview, but for samples it's better to work with the source. Not that it matters to me as such, I've played around and re-encoded a few of the h264 samples and I know how I'd handle them.

Did you look at the last few re-encodes I uploaded? I'm just wondering what you thought. I'm not claiming extreme cleverness of anything. I just decimated to 23.976 and ran QTGMC over them. I was just wondering how they compared to whatever method you're using at the moment, that's all.

JoelHruska

24th May 2020, 19:33

Did you look at the last few re-encodes I uploaded?

Haven't checked any yet. Wanted to get the clips posted last night once I understood the DVD source was what people wanted to see. I will look tonight. So far today, the garage has been demanding my attention. ;) (I move on Thurs).

Katie Boundary

24th May 2020, 21:50

Katie, you’re in luck. If you want video output from Topaz then you should know that all it exports is xvid. In order to use a modern codec you have to take the sequence of images route.

But it does so in MP4, not AVI, and it offers no control over bitrate or number of passes or anything like that?

JoelHruska

25th May 2020, 01:14

All,

Updated set of shared files. These clips were exported via DGIndex directly from the DVD VOB.

All of the clip names are identical to what I used in the previous series of posts, except for the addition of ".demuxed" to the file names.

No need for OneDrive this time, and I can't post individual links because I'm not a SendSpace premium member, so here's the link to all 14:

https://www.sendspace.com/filegroup/Y%2B9HP%2BpjhYvgld3iuIz%2F5KHAfjHMjlgefsUUt%2B%2FkHq%2FvD5HuwCXmYhV4VwOIVnnw1%2Fg4Q59rK5s5tWAfQJIYIBN2Vfhgu7iIZU%2FC3z%2F9ympIrpc2%2FSo2GQ

The link below is for the original file descriptions:

https://forum.doom9.org/showthread.php?p=1913411#post1913411

videoh

25th May 2020, 01:23

These clips were exported via DGIndex directly from the DVD VOB.Now you're cooking with gas. ;)

JoelHruska

25th May 2020, 01:36

But it does so in MP4, not AVI, and it offers no control over bitrate or number of passes or anything like that?

You get four options:
*.MP4
*.PNG
*.TIFF (8-bit)
*.TIFF (16-bit)

You can change the file name and you can choose the upscale model. You can choose to enable audio pass-through and you can choose the multiplier or specific output resolution (within certain restrictions, as you've noted). You can also choose to only upscale part of a video rather than the whole thing.

For now, that's literally everything.

I cannot speak to how Artemis-LQ and Artemis-HQ will treat your own source, but my experience with both suggested they are semi-functional atm as far as DS9 is concerned.

https://www.extremetech.com/wp-content/uploads/2020/05/ImproperAIScaling.jpg

That's one of the Artemis models run against a generic upscaled DS9 MKV and against my own work on the right. Pretty sure HQ, in this case.

https://www.extremetech.com/wp-content/uploads/2020/05/Artemis-LQ-Upscaler.jpg

I know this one is Artemis-LQ, specifically. Again, MKV on left, my own output on right.

I'd be very curious to know if you get different results. Most people are sticking with either Gaia-HQ or Gaia-CG.

Also, be advised, Topaz *does* introduce color-shifts on some content. I don't know which or why, but Weyoun's jacket is often color-shifted towards a darker brown.

zapp7

25th May 2020, 04:19

I can't image you'll do much better than smoothing out the baked-in aliasing with QTCMC.

The noise in the film sections is hard to deal with. It looks like the noise removal used previously stabilised what it didn't remove, so it doesn't behave like noise. I think QTGMC would clean it up okay, but it might soften the picture too much. It might be better to just leave it.

Samples 2.zip (https://www.sendspace.com/file/gwxxi0) (7.4 MB)

I can confirm that the noise is difficult to deal with. I tried a bunch of different filters - mcdegrainsharp, temporaldenoise2, QTGMC, TNLMeans - and while the intro looked great, the film sections had too soft a picture for my tastes. The more filters I try, the more I prefer just field matching and decimating and leaving the progressive frames alone. I think the higher picture quality outweighs the benefits of denoising, in this case. Viewing distance is a factor too. The noise is less noticeable when I'm sitting on the couch watching on my 4K TV compared to a couple of feet from my computer monitor.

JoelHruska

25th May 2020, 04:51

H_H,

I've looked at SFE-1 on both Samples and Samples 2. I'll check Sisko in the AM.

They're both gorgeous, but I don't see all that much difference between them. When I watched frame-by-frame, I saw the ripple pattern move somewhat differently on the Miranda-class ship's hull when it fires, but I didn't see much else in terms of difference. Where should I be looking?

hello_hello

25th May 2020, 07:50

Just the resizing. I cropped it but didn't resize to the correct aspect ratio the first time, so I included it again with the second samples.

Looking at the script, for the first one I ran QTGMC over it like this:
QTGMC(InputType=1, Preset="slower")
and for the second one like this:
QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)

That may have changed the ripple pattern a tad. The second one should help retain any motion blur, but it doesn't make a huge difference for the CGI.
Or the ripple pattern may just look a bit different due to the resizing.

hello_hello

25th May 2020, 07:59

That sounds like you mean the noise is same across frames ie temporally same, maybe somebody added fixed pattern grain. [bad idea]

I think it's like the effect you can sometimes get denoising with QTGMC. Because it stabilises the picture, and therefore any remaining noise, if it leaves too much behind and you use a slow speed preset, the effect can be a little like watching video through a fly-screen. Sometimes if I'm leaving noise behind I use a faster speed preset so as not to stabilise it too much.
It doesn't look that bad for the DS9 samples I've seen though, but against a flat background it's fairly static.

I'd be interested to know what was used to filter the noise originally, and/or just how noisy it was. Aside from the static noise left behind, there's a bit of blurring during motion on occasion. That's probably why they didn't remove any more than they did.

JoelHruska

25th May 2020, 14:42

JoelHruska

25th May 2020, 14:53

If anyone is curious to see what Hello_Hello's samples look like after being run through the two functional Topaz AI upscaler models:

https://www.sendspace.com/filegroup/hAmG8UuXAfl3T2VT6e8pntkJJg0KsgtR

I'll test the other two again (Artemis) on the properly deinterlaced clips to see if it functions now, but I'll be a little surprised if they do.

Katie Boundary

25th May 2020, 19:12

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched.

Well, that's because the faces ARE stretched, and so is everything else. These images are meant to be displayed on a device with non-square pixels. You need to resize to 640x480 (or some other resolution with a 4:3 aspect ratio) to get everything back to the right shape.

hello_hello

25th May 2020, 19:27

Pardon my stupidness, but what's the difference between CG and HQ?

I'm only looking at them on a 1808p display, but I ran the upscaled versions at full size (I only see 1080p worth of picture) then upscaled the DVD encode by roughly the same amount with MPC-HC. The CG versions are an improvement for the CGI. It looks better than I expected, because I was half expecting the upscaling to do things to it I'd hate. I wouldn't mind seeing a live action comparison. The HQ versions are over-sharpened for my taste.

1080p screesnshots.zip (https://www.sendspace.com/file/kys70d)

hello_hello

25th May 2020, 19:33

Hello_Hello,

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched. But they're both absolutely gorgeous.

Also, the lighter QTGMC approach still cleans up the aliasing on the lower hull very nicely.

Technically you're better off cropping the DVD to 704x480 (which is an exact 4:3 display aspect ratio) and upscaling directly to 4:3 dimensions, as that way you might squeeze a tiny bit more detail out of it than you get resizing to 640x480 first. Can you specify the upscaled width and height individually?

JoelHruska

25th May 2020, 19:52

Hello_Hello,

Those are the names of the new models they use. I've done a fair bit of comparison between them and I've run full encodes of both on episodes.

HQ typically retains more noise than CG if used on scenes with people in them. CG has stronger default noise filtering and deblocking applied. Gaia-CG is typically criticized for being overly aggressive on noise reduction outside of CG scenes, and for applying a bit too much smoothing. Gaia-HQ is typically criticized for retaining too much noise.

I generally prefer Gaia-CG over HQ for even the character scenes, but the models are still under active training and there's going to be a big update dropping in the next few weeks. Neither is perfect.

zapp7

25th May 2020, 20:26

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

JoelHruska

25th May 2020, 21:52

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

Which episode and where? I'll see if I can duplicate.

zapp7

25th May 2020, 21:54

Which episode and where? I'll see if I can duplicate.

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

JoelHruska

25th May 2020, 22:22

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

I'll rip it and take a look.

SaurusX

25th May 2020, 22:38

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

JoelHruska

25th May 2020, 23:05

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

The impact of Gaia-CG on CG scenes in DS9 is minimal compared to the uplift. It's more noticeable on non-CG content where, yes, there is some visible smoothing out of detail. I ran the denoise / deblock 100% render option on Gaia-CG multiple times on the same episode, just to see what would happen. In the "Sacrifice of Angels" recap of the previous episode, Sisko starts the clip wearing a quilted uniform top that has vertical lines on it -- seams, stitched into the fabric. You can see them clearly.

After a 2x runthrough Gaia-CG, they are noticeably less visible when he changes the angle of his stance. At three passes, they disappear altogether when he's standing at certain angles. Hopefully all of this will be tuneable one day.

zapp7

26th May 2020, 15:41

So my previous comments about noise filters had one caveat: I was using Artemis HQ in Topaz, which I have discovered already applies its own denoising. So in actuality, I was denoising twice and that's probably why the image looked too smooth for my tastes.

I've produced some sample clips using Gaia-HQ and Gaia-CG in an effort to see how the noise filters look with those. The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking. My QTGMC call is QTGMC(InputType=1). These filters are applied before upscaling with Topaz. (I wonder, would there be any theoretical advantage to applying them after upscaling?)

I'm trying to find a filter that I could apply to everything, since this project is big enough without having to edit noise filters scene-by-scene.

I only ran MCDegrainSharp with Gaia-HQ because it's obvious that the result isn't good - it seems to leave a pattern on the character scenes that doesn't look good. I'm also not too fond of the QTGMC result - perhaps someone can suggest better settings as I'm not very familiar with them. I kind of like the TemporalDegrain2 result, but I fear my eye for detail is lacking as the file size for this one is about half what it is without denoising.

TFM and TDecimate were applied before applying the noise filters. Input and output from Topaz was in .png format. These clips were created with ffmpeg libx265, using the slow preset and RF of 20.

The clips can be found here. (https://ln2.sync.com/dl/9dace98c0/8mr6xhrr-dcbbbfpa-4paijz6x-myybx25t)

Forteen88

26th May 2020, 16:20

...The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking.When I use TemporalDegrain2, I prefer the setting: TemporalDegrain2(grainLevel=false)
Because that setting don't denoise as much (I'd like to know how to make TemporalDegrain2 denoise even less sometimes, but I don't know what setting to use for that).

Boulder

26th May 2020, 16:22

A suggestion for x265: set the parameter no-sao. SAO will smooth the image even more, sucking out details on its way.

zapp7

26th May 2020, 17:58

Thanks, I'll implement those suggestions and generate some new clips tonight.

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

Boulder

26th May 2020, 19:01

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

I use debanding (neo_f3kdb) myself in all my encodes as the last item. Using the preset "low", I haven't noticed any ill effects aside from a slight increase in needed bitrate but it should keep those ugly artifacts away.

zapp7

27th May 2020, 00:45

I did a quality comparison and I'm pretty sure that more detail is retained by applying TemporalDegrain2 after upscaling. This makes sense since more detail is available for the input to Topaz.

I made a new clip with TemporalDegrain2(grainLevel=false) applied after upscale and encoded with no-sao. The result is in "S06E06_GaiaHQ_post_TempDeg2.grainLevel0_nosao.mkv" in this folder. (https://ln2.sync.com/dl/9dace98c0/8mr6xhrr-dcbbbfpa-4paijz6x-myybx25t)

I think this result is pretty good. If I zoom in and compare specific frames with the GaiaHQ_NoDenoising clip, I can't see any significant difference in detail but the noise seems significantly reduced. Could use a second opinion to verify my sanity...

JoelHruska

27th May 2020, 02:30

zapp7

27th May 2020, 02:48

It looks to me like you conserve more detail with the -no sao and applying degrain after upscale.

https://i.imgur.com/YXqP44c.png

What does Gaia-CG look like with these settings?

I put the GaiaCG version in the same folder. I believe GaiaCG already has denoising so I'm also making a GaiaCG_NoDenoising_nosao that should be there shortly.

JoelHruska

27th May 2020, 03:27

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

zapp7

27th May 2020, 04:50

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

I agree. What did you think of the MQ model in the previous versions? I read some people liked that one but I don't have it on hand.

Stereodude

27th May 2020, 22:08

Also, be advised, Topaz *does* introduce color-shifts on some content. I don't know which or why, but Weyoun's jacket is often color-shifted towards a darker brown.
Are you feeding it TIFF/PNG files or a video file and how are you making them?

Also, how are you getting the image files back into video?

We discussed this with zapp7 earlier in the thread. If don't right you shouldn't have a color shift (unless Topaz is broken or we don't understand what it's doing).

JoelHruska

27th May 2020, 23:11

Stereodude,

This was based on a video file, not PNG or TIFF output. I started to play with lossless at one point but had underestimated the storage requirements (Emissary and even PNG output don't mix unless you've planned on the storage requirement beforehand).

Both HQ-CG (Topaz 1.1.0) and Gaia-CG can cause a color shift, but they do it rather differently. HQ-CG had a *very* subtle lightening effect on DS9. I'm not even sure I dislike it, since one of the persistent complaints about the show is how dark it is. (If I was going to dislike HQ-CG it would be for detail removal, not the very small color shift).

Gaia-CG can deviate more widely than HQ-CG, but where HQ-CG had a very subtle effect on everything, Gaia-CG shows this less globally.

Also, while I haven't played with the lossless upscale option yet, I had planned to use a method like the one CaptRobau gave in his blog posts.

https://captrobau.blogspot.com/2019/05/tutorial-upscaling-video-with-topaz-ai.html

When I duplicated his workflow back when Gigapixel AI was the only tool available, I also duplicated his command and had no problems with it:

ffmpeg -r 29.970628 -f image2 -s 1440x1080 -i example%04d.jpg -i original-audio.wav -vcodec libx264 -crf 20 -pix_fmt yuv420p -acodec copy upscaled_video.mp4

Any specific changes you'd recommend?

Stereodude

28th May 2020, 14:32

Any specific changes you'd recommend?
1) Not using lossy compression until your final compression step.

2) I probably wouldn't convert to 8-bit H.264, but 10-bit HEVC instead.

3) I wouldn't load the images from a ffmpeg command line. HD video has rec.709 color space. DVD has rec.601. I'm guessing Topaz doesn't compensate for that. You need to make that conversion.

We discussed this with zapp7 earlier in the thread.

Convert to RGB in your AVIsynth script, and save the output to 16-bit tiff files. Run those through Topaz and then save the output as tiff files.

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Load the 16-bit tiff files into AVIsynth (if using ImageSource, don't forget pixel_type="RGB48") and convert them to 4:3 yuv420 16-bit video with a rec.709 color space.

ConvertToPlanarRGB()
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Then dither down to 10-bits with f3kdb

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

Feed that into x265 to compress it to 10-bit HEVC.

zapp7

28th May 2020, 17:17

I've given up on the 16-bit tiff files. I just don't have the requisite hard drive space as one episode in 16 bit tiff takes more than 2 TB.

I still do the conversion to RGBP16 before Topaz to preserve the full color range, but I save Topaz output as 8-bit .png instead and convert that to YUV420. I honestly couldn't see a difference between the 8-bit and 10-bit final versions.

Stereodude

28th May 2020, 18:25

zapp7

28th May 2020, 18:44

FWIW, if I couldn't only have 1 that's 16-bits, I'd go 8 bit in and 16-bit out.

The 16-bit in isn't an issue because the frame size is smaller. It's the 4K 16 bit files that are massive.

Stereodude

28th May 2020, 18:57

The 16-bit in isn't an issue because the frame size is smaller. It's the 4K 16 bit files that are massive.
Well, you don't have to do the full episode in one shot. You could do some number of frames and convert to a losslessly compressed intermediate file or go straight to HEVC in segments. But the Topaz folks should make it more versatile in terms of what it can open and export.

zapp7

28th May 2020, 19:29

Well, you don't have to do the full episode in one shot. You could do some number of frames and convert to a losslessly compressed intermediate file or go straight to HEVC in segments. But the Topaz folks should make it more versatile in terms of what it can open and export.

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.

ReinerSchweinlin

28th May 2020, 20:48

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.

Yeah, takes a looot of time... Topaz VEAI has CLI support, so you could whip up a batch file to use the two GPUs with one instance..

In general: I guess what Joel is trying to accomplish is to come up with a workflow that can be replicated by anyone so the upscale and encoding could be distributed to many people. I could join in with a s mall batch as soon as things are pinned down.

zapp7

28th May 2020, 21:30

Yeah, takes a looot of time... Topaz VEAI has CLI support, so you could whip up a batch file to use the two GPUs with one instance..

In general: I guess what Joel is trying to accomplish is to come up with a workflow that can be replicated by anyone so the upscale and encoding could be distributed to many people. I could join in with a s mall batch as soon as things are pinned down.

I saw CLI support mentioned on the Topaz forums but I can't find any info on how to use it. Do you know where to find this? I would love to have a batch file to make life easier.

Stereodude

28th May 2020, 21:35

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.
A single episode is what ~3.0TB in 3840x2160 uncompressed 16-bit TIFF files + ~150GB for the 720x480 uncompressed 16-bit TIFF files?

I guess you could buy a very large HDD for the project.

scharfis_brain

28th May 2020, 22:12

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

zapp7

28th May 2020, 22:19

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.

Stereodude

28th May 2020, 22:45

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.
That's only if there's actually more detail in the 2160p upscale vs. a 1080p upscale. Has anyone compared that?

johnmeyer

28th May 2020, 23:49

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.

hello_hello

29th May 2020, 01:57

You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.

I assume you haven't looked at any of the upscaled samples?

I'm not saying they're to everyone's taste, but they generally look better than the SD original. At least for the CGI upscaling samples I've seen.

Whether you can do just as well by filtering without upscaling is another question, but if you do you're still at the mercy of the player/display upscaling in respect to how it'll look.

Forteen88

29th May 2020, 08:11

...
Whether you can do just as well by filtering without upscaling is another question, but if you do you're still at the mercy of the player/display upscaling in respect to how it'll look.By "10 years from now", the computers will be so fast that players
can have very high quality resize.

ReinerSchweinlin

29th May 2020, 11:42

I saw CLI support mentioned on the Topaz forums but I can't find any info on how to use it. Do you know where to find this? I would love to have a batch file to make life easier.

It was present in an early version and there was a help file about it, then they removed the feature end re-introduced it some versions ago - but the help file is missing now, I just looked up.. I will recover some of the old betas and search for it and post it here, give me some time, got work to do...

ReinerSchweinlin

29th May 2020, 12:11

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

Good point :)

While dealing with AI Upscalers, I learned one thing (I am refering to techniques that deal with single frames only, to keep it simple)

IF the AI starts to "see" somehting, i can re-interpret what it sees and replace the low resolution or noisy stuff with something new - preferably somehting looking similar or at least more pleasing to the eye..

This is very easy with cartoons which mostly have simple geometrical forms as lines, areas, fades, rectangles, etc.. - not to many textures and no "real life footage". So the Ai only has to "know" these types of image content and has an easy job replacing everything with something that looks good and very "realistik"...

As soon as real life footage is to be dealt with, the content becomes endlessly complex... If a Model is trained on landscape footage like mountains, sky, clouds, grass, trees, etc... It will pick up these kinds of objects and "repaints" them - but it will fail horribly on faces and other stuff.

It´s obvious that no model can cover every possible situation. Companies like TOPAZ use training sets which have a wild mix of "most common footage" in them to get a good coverage of all possible situations. But of course, there is a lot missing in the models. That´s the reason why a spezialized model can work so much better on specific footage than a generic one. The game-upscale community is training models on very specific games or techniques to take that into account.

So when throwing DS) into VEAI from Topaz - TOPAZ only can do as much as it knows - which isn´t tooo much in the case of DS9.... I exeprimented with some older DVDs, originally filmed a few decades ago with conventional film-cameras, containing mostly live footage (no special effects, no "mean" postprocessig, effects, filters, just "point the cam and film")... With this kind of material, it works really well (I simply love "police squad" in 1080p now..). But DS9 - not much luck.

(In fact, I would propose whipping up an open source "upscale project" with some nifty AI Upscalers and the possibility to train own models and then take something similar to DS9 available in true HD, downscale it to DVD or record to VHS - train on that and make a "DS9" model.... Startrek Movies, Next Generation, etc... Most of the stuff neede is out there, it´s the matter of having enough GPU power, time, knowledge to whip it all together....

ok, back to DS9 and "how much is enough"...

If the AI picks up: "ah, hair" - it replaces it with new hair, almost similar to the old one. If the model is done right, the alog is good, etc... one could now go up to whatever the model and framework limit - 8K - no problem (with most models)..

But - if the AI doesn`t know what to do - it simply upscales the original, bicubic, Lancoz, etc.... In this case, no more detail is produced.

VEAI now has frameworks which are able to recover details from multiple frames and add some deconvolution stuff into the workflow - so in some cases even if the model isn´t getting "hair", it is able to get some detail back - works good on still shots - in some cases I even got garbage on a pc-screen to reveal the actual numbers on the screen (SG1 NTSC DVD in this case). This mostly is refered to "super resolution". But it has ists limits. A quick turning head simply doesn´t have enough previous and following frames with the same info as the frame in question to recover anything out of the time domain... The Deconvolution sometimes is able to sharpen a little or make it at least visually more pleasing - but in my oppinion, in the case of VEAI - Topaz has done too much of it... This is very visible with compression artefacts (blocking) or noise or halos or ghosting etc... These get pronounced too much by the VEAI implementation..

For the moment, I personaly would stick to 1080p and keep the source for future usage.
In the case of VEAI: I would wait for them to sort things out before making a final upscale.

If I had the time and knowledge, I´d go the route if implemting an open source solution and train a DS9 Model :)

ReinerSchweinlin

29th May 2020, 12:14

You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.
John, take a look at some examples on youtube. Some are garbage, but wiht others, you can really see "new" details.... Of course, this is "reinterpretation at higher resolution", not really "upscaling". And it can be wrong (I had some marvel busts in one video - the AI thought "oh, a head, lets pain hair into it"... But in many cases, the new details are quite pleasing to the eye...

So it really differs from tradtional upscaling (if it works, read my post above)

zapp7

29th May 2020, 21:22

z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Just curious, would there be any advantage to using YUV444P16 instead of YUV420P16 in this workflow?

Stereodude

29th May 2020, 22:39

Just curious, would there be any advantage to using YUV444P16 instead of YUV420P16 in this workflow?
You've got to get to 420 at some point, and this presumably is basically at the end of the processing flow aside from going to 10-bits for HEVC encode.

I'd argue it's best to only scale it once instead in of multiple steps.

ReinerSchweinlin

30th May 2020, 18:05

It was present in an early version and there was a help file about it, then they removed the feature end re-introduced it some versions ago - but the help file is missing now, I just looked up.. I will recover some of the old betas and search for it and post it here, give me some time, got work to do...
Here you go. It´s from an older version, so selecting new models isn´t included yet. I´ve asked Topaz for a newer version, waiting for a response.
BTW, they released a new beta with a new model yesterday.

ReinerSchweinlin

30th May 2020, 18:07

I'd argue it's best to only scale it once instead in of multiple steps.
I am not sure if we are talking about the same "steps", but in early versions of VEAI, sometimes processing the Video first with a 100% "cleanup" pass helped quite a lot.

JoelHruska

30th May 2020, 19:29

John,

For decades, the idea of "Enhance" -- as in, the recovery of detail levels over and above what was originally encoded in the source -- was a joke. Completely farcical. You're not wrong.

AI-based upscalers have turned science fiction into reality. There are different models that process the final image in different ways. Topaz Video Enhance AI is very much a work in progress.

If you want to see an example of the same video scene run through an upscaler without using YouTube as an intermediary, here:

This link is to samples Hello_Hello provided of some DVD clips I gave him:

https://www.sendspace.com/file/gwxxi0

This link is where you can download upscaled versions of those samples:

https://www.sendspace.com/filegroup/hAmG8UuXAfl3T2VT6e8pntkJJg0KsgtR

The SFE-1 and SFE-2 videos are upscaled using two different models -- Gaia-CG and Gaia-HQ, both available in Topaz Video Enhance AI.

You can see the significant level of improvement for yourself. You may not like it, of course, but you will not fail to see the difference.

Stereodude

30th May 2020, 21:01

I am not sure if we are talking about the same "steps", but in early versions of VEAI, sometimes processing the Video first with a 100% "cleanup" pass helped quite a lot.
I mean mean taking the output from Topaz and getting it into 2880x2160 YUV420. 444 to 420 requires scaling the chroma. You've also got to turn 3840x2160 into 2880x2160. It's probably best to do that in only 1 operation instead of compounding multiple separate steps that each can degrade the image.

zapp7

30th May 2020, 21:18

I mean mean taking the output from Topaz and getting it into 2880x2160 YUV420. 444 to 420 requires scaling the chroma. You've also got to turn 3840x2160 into 2880x2160. It's probably best to do that in only 1 operation instead of compounding multiple separate steps that each can degrade the image.

Here is my code to handle the output of 16 bit tiffs from Topaz VEAI. Should the TemporalDegrain2 come before or after the z_ConvertFormat step?

#outfolder is the topaz output directory, filecnt is # of images
ImageSource(file=outfolder+"\%06d.tiff",start=0,end=FileCnt-1,fps=23.976,pixel_type="RGB48")
ConvertToPlanarRGB()
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic",pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")
TemporalDegrain2(grainLevel=false)
neo_f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

ReinerSchweinlin

30th May 2020, 21:45

ReinerSchweinlin

30th May 2020, 21:50

For decades, the idea of "Enhance" -- as in, the recovery of detail levels over and above what was originally encoded in the source -- was a joke. Completely farcical. You're not wrong.
I´d like to add that it is important to understand, that most upscaling techniques actually "make stuff up" - so what we war used to see in movies ("real" details recovered by som e magic) actually is a little different. There are some apporaches to get a little detail back (also see my other post), like this one:
https://github.com/jiangsutx/SPMC_VideoSR
https://www.youtube.com/watch?v=0WnwS1EOx3M

But these have to be differentiated from the "repainting" models...

JoelHruska

31st May 2020, 00:34

Stereodude

31st May 2020, 01:17

Ah ok, I get what you mean. VEAI internaly only can upscale 2x, 4x, 8x - so it´s probably best in terms of quality to not use "1080p" preset, but rather output 2x or 4x in 16Bit TIFF, then do one downscale/chroma/encode pass... Internaly, if a preset with a fixed resolution is set in VEAI, it uses ffmpeg to downscale from the AI Output.
Oh, I didn't realize that. They would definitely want to do a 4x or 8x then (depending on whether 1080p or UHD is the output target). Perhaps a sharper downsampling kernel that avoids ringing should be used with it. SSIM 2D in madVR looks very good, but I don't know if there's an Avisynth equivalent of that.

hello_hello

31st May 2020, 04:58

Well I'll confess after trying some upscaling with Avisynth, I'm now starting to wonder just how clever the clever upscaling is by comparison.

I took the original lossless h264 DVD samples and upscaled them to 4k in Avisynth. I had to do it using 2 processes and hobble x264 a little, otherwise my poor old XP machine would run out of memory, but here's the files in the zip file and how they were created etc.

SFE-1 nnedi3 4k rec.709.mkv
SFE-2 nnedi3 4k rec.709.mkv

Both upscaled using the following script. I forgot the original upscales weren't color converted so the colors are a little different, but that's not what the comparisons were about.
There may be better ways to sharpen than with LSFMod, but without a lot of experimenting, it's the only sharpening method I trust not to look horrible.
There's only a frame or two of live action in each sample, but it looks pretty obvious that using the method below would require the live secions to either be denoised first, or sharpened far less so the result isn't sharpened noise, however this was primarily to look at how nnedi3 would compare for the CGI.

MP_Pipeline("""
LoadPlugin("C:\Program Files\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\SFE-1.mkv.lwi")
AssumeTFF()
ColorMatrix(mode="Rec.601->Rec.709", clamp=0)
Crop(8,0,-8,0)
TFM(pp=5, micmatching=0).TDecimate()
QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)
LSFMod(Strength=200)
### prefetch: 16, 0
### ###
""")
nnedi3_rpow2(rfactor=8, cshift="Spline64Resize", fwidth=2880, fheight=2160)
LSFMod(Strength=200)

JoelHruska's upscales
SFE-1_4.00x_2560x1920_Gaia-CG.mp4
SFE-2_4.00x_2560x1920_Gaia-CG.mp4

The included screenshots are the same frame from SFE-1 (the nnedi3 screenshots are directly from the Avisynth output (no-re-encoding) so any advantage they have in terms of accuracy has now been declared), and the versions upscaled by JoelHruska are upscaled from my IVTC'd SD encodes of the original source, so that's their disadvantage declared too.

The screenshots with 4K in the name are the full upscaled frame. The screenshots not labelled 4k were taken using the PrintScreen button with MPC-HC only displaying 1080p worth of the frame on my 1080p monitor. That way the 1080p screenshots can be compared without the need for resizing on a 1080p monitor. Because I forgot about the color correction thing when encoding, the nnedi3 screenshots include a version that wasn't color converted to rec.709.

The samples upscaled by JoelHruska are also included because I was starting to get confused with the multiple sample uploads myself, so the nnedi3 and "Gaia-CG" upscales are both included.

The clever upscaling does clean up the line wobbling/aliasing more, but the rest, I'm not so sure it's better. Maybe there's a better anti-aliasing filter than QTGMC that could be used first, before upscaling with Avisynth?

nnedi3 comparison.zip (https://www.udrop.com/6u1/nnedi3_comparison.zip) (79.8 MB)

Katie Boundary

31st May 2020, 09:02

Is 3840x2160 really necessary?

No, and in fact it's the wrong aspect ratio.

zapp7

1st June 2020, 01:47

Oh, I didn't realize that. They would definitely want to do a 4x or 8x then (depending on whether 1080p or UHD is the output target). Perhaps a sharper downsampling kernel that avoids ringing should be used with it. SSIM 2D in madVR looks very good, but I don't know if there's an Avisynth equivalent of that.

It seems that VEAI can't actually upscale by 8x. The maximum it will accept for me is 6x. So it looks like to do 4K with this workflow, there is a minimum of 2 resampling steps to the VEAI output.

JoelHruska

1st June 2020, 02:57

Zapp7,

Are you going to crop it for 16:9 aspect ratios?

zapp7

1st June 2020, 04:09

Zapp7,

Are you going to crop it for 16:9 aspect ratios?

No, I'm sticking with the 4:3 ratio.

Stereodude

1st June 2020, 11:42

It seems that VEAI can't actually upscale by 8x. The maximum it will accept for me is 6x. So it looks like to do 4K with this workflow, there is a minimum of 2 resampling steps to the VEAI output.
I'm not sure I'm following you. 6x would give you 4320x2880 RGB 16-bit tiff files. You can turn that into 2880x2160 YUV420 in a single scaling step.

zapp7

1st June 2020, 16:41

I'm not sure I'm following you. 6x would give you 4320x2880 RGB 16-bit tiff files. You can turn that into 2880x2160 YUV420 in a single scaling step.

Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!

ReinerSchweinlin

1st June 2020, 22:03

Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!
I´d have to check if the newest model can do other than 2x, 4x, 8x scale factors - not sure at the moment. The gaia and artemis are "factor of 2".
CLI can be used for batches, but it´s incomplete at the moment, some stuff missing. It´s on the roadmap of TOPAZ, hopefully they´ll stick with it and clean this feature up.

ReinerSchweinlin

1st June 2020, 22:34

Reiner,

What would you call an approach like ESRGAN? According to the description of the model:

"how do we recover the finer texture details when we super-resolve at large upscaling factors? ... Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks."

They use phrases like "recover" as opposed to "approximate" or "repaint." Are they obfuscating their own approach, or are they one of the models that actually recovers data?
Joel, I´ll write a more thorrough text when I get back from a project, but I don´t want to leave without a short comment:
"recover" is a fuzzy term - ESRGAN actually is one of the "we make stuff up" methods (looks good, but is not original, but pleasing to the viewer (in case of video - more about this later)). There are some, which CAN reveal some detail by looking at multiple frames. It´s best - in my oppinion - to combine both (VEIA actually does combine both methods to a certain extend - but I´d love to see a selection on what to turn on or off)..
More when I come back..

Stereodude

1st June 2020, 23:32

Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!
Oh, I thought you were saying it could get to 6x in 1 step. If it can't output 8x as is it's a bit of a mess then.

zapp7

2nd June 2020, 05:56

Oh, I thought you were saying it could get to 6x in 1 step. If it can't output 8x as is it's a bit of a mess then.

So this leads to my next question:

Which do you think is better: upscaling to 4K with 2 resampling steps or upscaling to 1080p with 1 resampling step?

Stereodude

4th June 2020, 02:08

So this leads to my next question:

Which do you think is better: upscaling to 4K with 2 resampling steps or upscaling to 1080p with 1 resampling step?
No idea. You'd have to try it and compare them.

hello_hello

7th June 2020, 05:23

zapp7

11th June 2020, 17:58

Did anyone check out my nnedi3 upscales here?
https://forum.doom9.org/showthread.php?p=1914291#post1914291

I think upscaling to 4k is over-kill myself, but aside from needing to find a way to deal with the remaining aliasing, I still think they look just as good, if not better, than any of the clever upscaling methods I've seen so far... for upscaling the CGI, at least.

I'm also not sure you couldn't achieve much the same thing by simply applying some decent sharpening and encoding, and letting the player do the upscaling, but I haven't compared that yet.

I had a look at your nnedi3 and it appears to have less detail than my method of Gaia-HQ (followed by TemporalDegrain2(grainLevel=false)) generated with 16 bit tiffs and dithered down to 10 bit HEVC.
Sample here (https://ln2.sync.com/dl/bfe630490/8r55pb2n-vjbacdj8-6zwz2czk-fcgqbkn2)

Comparison here, GaiaHQ on left, nnedi3 on right (https://ibb.co/30qHHWn)

I will say that the nnedi3 is appealing since the whole workflow is far less tedious and takes about 4ish hours per episode versus the 24+ hours for this GaiaHQ workflow. (On my system, at least)

The aliasing doesn't bother me too much. It's not noticeable when I watch on my 4K TV from my couch.

hello_hello

13th June 2020, 02:30

zapp7

14th June 2020, 05:31

zapp7,
It surprises me you're upscaling before TemporalDegrain2. I assume it looks better than de-graining first? I'm not saying it's the wrong way to do it, because whatever looks better....

The tiny bit of live action at the end of your sample looks quite good, and while most of the fuss here has been how to handle the CGI, I assume there's a lot more live action than CGI in most episodes. I watched the show years ago but I don't remember it all that well. I still think upscaling to 4k might be overkill though. How does it look if you just upscale to 1080p and let the TV do the rest? I do wonder if that'd make the GaiaHQ version look a little less "sharpened", while still looking better than the DVD.

The GaiaHQ sample does generally look a bit more detailed than my nnedi3 sample, but I think it's mostly in places where your upscaling fixes problems (aliasing) that needi3 doesn't. I'm not sure I don't prefer the GaiaCG sample JoelHruska uploaded though because the HQ version looks a bit over-sharpened in places to me. Mind you I'm sitting close to the monitor and nitpicking over comparisons of individual frames of CGI for a show that'd be mostly live action. It's what you prefer when the picture is moving and you're sitting on the couch that matters.

I compared the amount of detail with TemporalDegrain2 before vs after upscaling. It was pretty clear that applying it after upscaling was preserving more detail. My guess is that leaving the frame untouched before upscaling leaves more detail for the upscaler to work with, so that was my rationale. I like how the live action scenes look - and you're right, if I had to guess I would say 95%+ of the series is live action stuff.

I've flip-flopped a lot on the 1080p vs 4k. I might run some more tests, but I'm at the point where I feel like I'm reaching diminishing returns on quality improvement vs time spent fiddling.

ReinerSchweinlin

15th June 2020, 08:08

What would you call an approach like ESRGAN? According to the description of the model:

"how do we recover the finer texture details when we super-resolve at large upscaling factors? ... Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks."

They use phrases like "recover" as opposed to "approximate" or "repaint." Are they obfuscating their own approach, or are they one of the models that actually recovers data?
Hey Joel,

sorry for the long delay, "life to do" :)

ESRGAN actually "makes stuff" up. It uses models which are trained on a certain type of content and tries to recognize what it sees and then repaints what it thinks it sees. It´s pretty obvious if we take a look at some of the author´s example pictures:
https://raw.githubusercontent.com/xinntao/ESRGAN/master/figures/qualitative_cmp_04.jpg
https://raw.githubusercontent.com/xinntao/ESRGAN/master/figures/baboon.jpg

If you compare "HR" (high resolution = the original) or "ground truth" to the ESRGAN Example and look at the fine details, it´s eays to see, that the bamboo's beard looks good - but the hair are not the same ones as in the original. Also the grass looks rather detailed and good, but the single straws are not all the ones in the Original.

ESRGAN also only uses single frames, so it´s not aware of the prevoius and following frames in a video.
Therefore, it works quite good on single images IF your model is trained for the kind of content you want to upscale. There are quite a few trained models out there, named "faces", "Wood", "maps", etc... some are for realistic fotografs, others for special content, many for old 8 Bit Games etc...

So when they claim to "recover", what they actually are saying is: "we recognize hair, so we put hair there"... To be fair, it´s just teh terminology used in this field.. if one looks up the research papers, the distinction of which mechanism does what is not hung up on terms like "recover" :)

Although some outputs are very impressive, the limits of this apporach is obvious as soon as you have such low resolution that the guesswork at hand simply is making up false details. A very small shot of an actor shot from the side simply isn´t enough detail to pick up. Also, the model should be trained for the specific content.. Which most aren´t.. VEAI uses generic models, not "Scifi from the 90s" trained ones...

My proposal at Topaz was to train models for specific usecases - There is "DS9 - like looking" footage ou there which exists in HD, so a training set could be compiled and trained upon... In the case if ESRGAN, we could do it ourselfs, there are many tutorials out there how to set up training.

Of course, there are other "AI-Upscalers/denoisers/etc..." out there, in fact, the list ist enormous (the following list is just "a few" examples):
https://awesomeopensource.com/projects/super-resolution

With video, we have another source of detail, we can "tap into"... The previous and following frames often contain the same objects as the current one. Often so called "super resolution" methods can take many frames into account and mostly by subsampling are able to recover details. One good illustration of the basic principal in my oppinion is the somewhat older software photoaccute, which works on multiple shots from the same scene in order to process Fotos. This screenshot frmo the manufacturers website illustrates what can be done if several pictures are combined:
https://www.photoacute.com/studio/examples/mac_hdd/3b.jpg
With video, it´s a little more complicated, but research as gone a long way, so there are algos which are able to recover some lost details in video - the screenshots in the above post with the NNEEDI comparison are a good example. The red marked line on the spaceship is a recovered detail which only appears in the AI-Version of the upscale.
One example out of my tests (SG1, taken from NTSC DVD):
https://forum.doom9.org/attachment.php?attachmentid=17382&stc=1&d=1592203522
https://forum.doom9.org/attachment.php?attachmentid=17383&stc=1&d=1592203522

Some of the numbers are revealed, some look a little more "generic", maybe I get some more detail out of this (challenge accepted... hehe)... The workflow chain for this is... ugly... but VEAI is involved at some stage to do the upscaling (Artemis LQ in this example, a version from a few weeks ago...).

One IMO powerfull technique which is able to do this is SPMC:
https://eng.uber.com/research/detail-revealing-deep-video-super-resolution/
A good example (taken from the author's paper):
https://images.deepai.org/converted-papers/1704.02738/x4.png

There is a lot going on in this field - personaly I am always torn between "finish it up" and "oh, ah better method arose, let´s start from scratch all over again" :)

I hope my long writing made some sense and helped a little..

neo_sapien

17th October 2020, 03:52

I'm experimenting with DS9, both at 480p 23.976fps and upscaled to 1080p at 23.976fps. For some reason Vapoursynth runs 10x faster than Avisynth for me (I'm using Staxrip 2.1.3.0), I get 2.6fps with Avisynth and 26fps with Vapoursynth. Because of that, I'm trying to convert this Avisynth script to Vapoursynth:

TFM()
TDecimate()
QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, TR2=4)
Repair(QTGMC2,QTGMC3, 9)

This is what I've come up with in Vapoursynth, but I don't know if it's correct or not:

clip = core.std.SetFieldBased(clip, 2) # 1 = BFF, 2 = TFF
clip = core.vivtc.VFM(clip, 1)
clip = core.vivtc.VDecimate(clip)
QTGMC1 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC2 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", InputType=3, TR2=4)
clip = core.rgvs.Repair(QTGMC1,QTGMC2, 9)

Doing a test encode shows that the Vapoursynth version needs an extra 100kbps at CRF-14 in x265, so it sounds like it's not producing the exact same output. Can someone check to see if I have it right?

And I tried using nnedi3cl to upscale, but it seems like I have to use that filter to deinterlace as well, unless I'm misunderstanding. You have to use the "field" function:

field: Controls the mode of operation (double vs same rate) and which field is kept.

0 = same rate, keep bottom field
1 = same rate, keep top field
2 = double rate (alternates each frame), starts with bottom
3 = double rate (alternates each frame), starts with top

So this is what I'm coming up with in nnedi3cl, given that I can't figure out how to turn disable deinterlacing and just use it to upscale:

clip = core.nnedi3cl.NNEDI3CL(clip, field = 1, dh=True, dw=True)

Is there a better way to use NNEDI3CL to just upscale? Preferably to 2.25x the original resolution?