Deep Space Nine upscale project [Archive] - Page 5

poisondeathray

16th May 2020, 15:43

IVTC is a form of deinterlacing. I think you mean that bob-deinterlacing should only be used on the parts that need to be bobbed, which is exactly how my method works.:cool:

No, that's not I mean.

Deinterlacing (any kind) primarily involves resizing a single field (+/- temporal filtering) . If you use it on progressive content, you lose ~1/2 the resolution of a full progressive frame. A full progressive frame consists of 2 fields from the same moment in time - they just need to be matched and weaved.

Double rate deinterlacing should only be used for the 59.94 sections

Single rate deinterlacing of any form (or QTGMC in progressive mode for temporal antialiasing) should only be used for problem sections, orphan fields

IVTC is used for progressive content. This means primarily field matching to get back the full progressive frames (+ decimation if 23.976p), + /- post processing for residual combing (e.g. orphan field, that single field can get deinterlaced) .

If you deinterlace, (single or double) with any method, you will degrade >90% of the content. The main distinction is interlaced vs. progressive content

It's worse than that: supposedly, some sections suffer retrograde field behavior. We still haven't discussed how to handle that.

Maybe someone should post a sample, and if verify if there really is 59.94 content. Some people say "interlaced" when it's really only combed for 3:2 hard telecine, or really only an orphan field.

poisondeathray

16th May 2020, 15:45

I'm experimenting with different denoisers for the first season of DS9. I have seen other people advocating for QTGMC as a denoiser run in progressive mode (InputType=1). Is QTGMC run in this fashion still destructive to the resolution of the footage?

QTGMC(InputType=1, Preset="Medium", EzDenoise=0.1)

Yes it's destructive

It might be appropriate for the 2-3% problem sections, but you will destroy details in >90% of the other scenes

I've tested a few filters so far... MCDegrainSharp, TemporalDegrain2, QTGMC, but none really give me good results. One issue is that some will look good for the intro but not for film sections with characters.

Exactly. You degrade the 90-95%. No amount of neural net processing can return the lost details

JoelHruska

16th May 2020, 18:25

Katie,

I feel like you still don't understand what I'm trying to do, here.

That is unfortunate, because your approach of decimating an entire episode down to 23.976 FPS will create problems in 30 FPS or 60 FPS sections that are much worse than judder.

So. Let me try to be even clearer.

I don't care about the output frame rate.
I don't care about the output container.
I don't care about the file extension.

Why, then, am I mucking around with 23.976 fps footage? Because it exists. Why have I converted to every other frame rate? To see what they look like. To see which gives the best output.

Why did I output to 119.88 fps? Because the AviSynth Wiki declares that 119.88 fps output is the most-compatible way to fix judder between 23.976 and 29.97 fps content.

I want good-looking video. Coming into this project, I had no idea what I needed to address in order to get it.

You keep talking about this project like I have declared: "I am going to create the best 23.976 fps encode of DS9 because it MUST BE IN 23.976.

I couldn't give two s**** and a whistle if the final output frame rate is 23.976, 29.97, 35, 42, 49, 60, or 119.88 fps. I don't really care if the output is 1fps, but we play back the content 23.976x faster than normal, if the end result was good-looking footage.

I will admit to one practical boundary: I just spent eight days waiting for an RTX 2080 to finish upscaling "Emissary" after I did a 119.88 fps conversion on it but *before* I applied any other filters or processing. I did it once, so I'd be able to compare the output to future attempts to process the footage, but there's no way I'm waiting four days to process each and every episode of the television show.

I'd like to keep the total processing time below the 24-hour mark per episode, keeping in mind that VEAI imposes a 10 hour encode time all of its own. The DaVinci Studio runs currently take about two hours, so my current process is a minimum of 12 hours of processing per episode.

But that's it. That's all I care about. I'm not wedded to 23.976 fps. I have worked on a 23.976 fps version because my intent is to publish instructions for upscaling the show depending, in part, on what frame rate you want to target. Do you want 23.976 fps? Then I want a workflow for it. I was creating simultaneous comparison shots for 23.976, 29.97, 48, 60, and 119.88 so that people could choose what worked best for them.

I'd like nothing more than to be able to tell people: "Use this process to arrive at the best final project frame rate that will look better than [list of options previously enumerated]. Don't bother screwing around with this other stuff. Just do this."

But I can't possibly tell people that if I don't know what all of the other speeds look like. If I can find one solution that looks better than anything else, I'll recommend it. If I can't, I'll recommend multiple solutions and show people the outputs so they can choose for themselves.

I haven't just explored one frame rate. I've explored like, six frame rates simultaneously. I don't care which one of them the final project uses. I care which one of them makes the final project look the best.

Katie Boundary

16th May 2020, 19:00

I just want to be clear. By retrograde field behavior, we're talking about content that, when run through separatefields().doubleweave() to simulate what it looks like on an NTSC TV, looks perfectly fine, aside from the fact that it's interlaced:

https://imgur.com/a/b62ndpK

But, when you look through it one field at a time using a dumb, temporally naive, double-rate bob-deinterlacer like bob(), it's obvious that the fields are NOT in the right order:

https://imgur.com/a/JqHkaHL

Joel, can you confirm that this kind of pathological content exists in DS9?

Deinterlacing (any kind) primarily involves resizing a single field (+/- temporal filtering) . If you use it on progressive content, you lose ~1/2 the resolution of a full progressive frame.

Deinterlacing is literally any process that converts interlaced content into progressive content. The specific form of deinterlacing that you're talking about is called bob-deinterlacing. It got that name from the dumb, double-rate, spatial-only filters, which would cause horizontal edges to bob up and down (as seen in the second gif that I posted above).

You keep talking about this project like I have declared: "I am going to create the best 23.976 fps encode of DS9 because it MUST BE IN 23.976.

No, I keep talking about this project like you're willing to pay any price and introduce any new problem to get rid of judder, and I just pointed out that decimating to 23.976 is the least ideal solution you've experimented with.

poisondeathray

16th May 2020, 19:11

Deinterlacing is literally any process that converts interlaced content into progressive content. The specific form of deinterlacing that you're talking about is called bob-deinterlacing. It got that name from the dumb, double-rate, spatial-only filters, which would cause horizontal edges to bob up and down (as seen in the second gif that I posted above).

Not necessarily spatial only; deinterlacing can use temporal algorithms (e.g. QTGMC does)

Interlaced content means each field represents a different moment in time. 59.94 different moments in time represented

The problem is you're mixing up Inverse telecine or pulldown removal with Deinterlacing. These are different things. IVTC is NOT a form of deinterlacing as you said, because it's used when underlying content is progressive.

So if you want to use your own definition, you are wrong ("Deinterlacing is literally any process that converts interlaced content into progressive content")

Since >90% of this is progressive content, you should not deinterlace as you suggested

Katie Boundary

16th May 2020, 19:15

Not necessarily spatial only; deinterlacing can use temporal algorithms (e.g. QTGMC does)

Bobbing got its name from what the the spatial-only versions did. This does NOT imply that all bob-deinterlacing is spatial-only, nor did I say that it did.

Interlaced content means each field represents a different moment in time. 59.94 different moments in time represented

No, interlaced content means that the content has interlacing in it. How that interlacing got there is irrelevant. If a VOB file is encoded as 24 fps with soft pulldown, and I index it in DGindex with "honor pulldown flags", bam, it's interlaced now.

The problem is you're mixing up Inverse telecine or pulldown removal with Deinterlacing. These are different things.

I didn't get them mixed up. I know they're different things. But one is a subset of the other.

So if you want to use your own definition, you are wrong

I'm sorry but your definitions are wrong. Deinterlacing is any process that converts interlaced content to progressive. Bobbing is a form of deinterlacing. Field-matching (which is the first step of IVTC) is also deinterlacing. Blur(1.0) is a form of deinterlacing.

poisondeathray

16th May 2020, 19:22

Bobbing got its name from what the the spatial-only versions did. This does NOt imply that all bob-deinterlacing is spatial-only.

I never said it was. I included temporal and gave an example

Deinterlacing is any process that converts interlaced content to progressive.

That's one definition

Bobbing is a form of deinterlacing.

Yes

Field-matching (which is the first step of IVTC) is also deinterlacing.

No. These are different by definition

Field matching , as the name suggests is for progressive content. 2 fields from the same moment in time

Interlaced content has only 1/2 the spatial resolution in motion. Single fields.

Blur(1.0) is a form of deinterlacing.

It can be

I didn't get them mixed up. I know they're different things. But one is a subset of the other.

No . You got them mixed up.

poisondeathray

16th May 2020, 19:27

No, interlaced content means that the content has interlacing in it.

Well that's an illuminating statement . Circular definition ? :D

How that interlacing got there is irrelevant. If a VOB file is encoded as 24 fps with soft pulldown, and I index it in DGindex with "honor pulldown flags", bam, it's interlaced now.

Of course it's relevant

24FPS with repeat field flags honored will show combing. It's not interlaced content. The underlying CONTENT does not change. It's reorganized as fields, that's all. The content is still progressive, it did not change

"interlacing" as a term - it's is a terrible description of what you see. You mean "combing"

Katie Boundary

16th May 2020, 19:31

https://en.wikipedia.org/wiki/Deinterlacing

Game over. I win.

poisondeathray

16th May 2020, 19:50

https://en.wikipedia.org/wiki/Deinterlacing

Game over. I win.

LOL What are you? like 13 ?

If you use that definition, you're wrong again

The film content is progressive. The underlying content does not change when you add pulldown or telecine . i.e you're not starting with interlaced content .

Interlaced content means 59.94 different moments in time represented (or even and odd scan lines come from different moments in time) . Do you have that ? No. You only have 23.976. It's progressive content.

Katie Boundary

16th May 2020, 20:22

You are literally arguing with wikipedia at this point. If you want to do that, I can't stop you, but you'll get very little support from anyone else.

poisondeathray

16th May 2020, 20:51

You are literally arguing with wikipedia at this point. If you want to do that, I can't stop you, but you'll get very little support from anyone else.

Because wikipedia is the ultimate reference and has no errors :D

I think that definition is not very useful, because it lumps everything under the umbrella of "deinterlacing"

Ok, put it another way - is it better to be more specific, more clear in communication, or vague ?

Since IVTC is a very specific subset if you use that definition, then why not use it when you mean it ?

You could call it "video processing" that's a pretty big umbrella

johnmeyer

16th May 2020, 21:27

IVTC is a form of deinterlacing.That is simply not true (i.e., wrong). IVTC involves the removal of redundant fields and, if the video is viewed directly without re-encoding, IVTC involves zero loss and produces zero artifacts.

By contrast, deinterlacing always involves degradation of the video because you must manufacture fields that were not there in the original. You do this either through duplication, blending, motion estimation, or some other technique.

I am bothering to post this in what has become a thread that is increasingly filled with strange statements because, unfortunately, a lot of people new to dealing with video make the mistake of conflating IVTC and deinterlacing and will often use a deinterlacer on telecined footage and then wonder why they get such horrible results.

So, deinterlacing and IVTC are two completely different things, and you cannot use the tool for one of them to solve the other one's problems. They are orthogonal.

One thing that was correctly stated in this thread by one of the doom9.org experts is that you must do IVTC before applying any temporal filter.

That is 100% correct, and should be obvious.

Why is it obvious? Because if you have a filter that looks at adjacent frames (or, sometimes, a bunch of nearby frames), the total lack of any change whatsoever between some nearby frames or fields, but not others, will completely blow up the algorithms.

As a corollary -- one I found out first-hand when I started encoding VCDs and SVCDs 20+ years ago -- is that encoders have the same problem as temporal filters when you try to encode telecined material without first doing IVTC. In fact, if you try to encode telecined footage, you will need integer multiple larger bitrates to get the same quality. I still remember spending half a day trying to encode the Elton John music video "I'm Still Standing" onto a VCD back in the late 90s. It was shot on film and the capture I made off satellite was telecined. I encoded that telecined video to SVCD and all I could see was "mosquito noise." It was awful. After lots of research and dozens of encodes, I discovered the IVTC built into TMPGEnc. I used it, and the results were a hundred times better (they still look good, even by today's SD standards).

JoelHruska

17th May 2020, 00:21

The whole reason I began using QTGMC in the first place -- the entire one -- is because I could call QTGMC, throw the entire file at it (with zero trimming) and what was output as a result was both substantially denoised and had all of the interlacing / 3:2 pulldown artifacts resolved. (I also got really nice horizontal AA, which is a nice feature).

Now, people here including PDR and H_H have pointed out that there's a substantial negative associated with QTGMC (well, more than one, but one big one in particular): It's destructive, and running a destructive filter over content that doesn't require a destructive filter is less-than-ideal.

The reason I have been attracted to using QTGMC is because, at least with DS9, it has done a remarkable job of fixing what I don't like visually, while introducing relatively few problems that catch my personal eye. Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing?

(I'm going to use trim and H_H's proposed method of treating CGI footage on Sacrifice of Angels, but that's a bigger project and I've got a move coming up, so probably not happening right away. I'm completely new to this kind of video editing, so.. slow going). In the meantime, I'm still exploring global approaches.

johnmeyer

17th May 2020, 01:32

videoh

17th May 2020, 01:34

Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing? If, if, if. Do you want to do things correctly and optimally, or not? Happy with half-assed?

And what in heaven's name is an "artifact of 3:2 pulldown"?

I'm completely new to this Got it.

johnmeyer

17th May 2020, 01:51

And what in heaven's name is an "artifact of 3:2 pulldown".24fps video or film exhibits "judder" to the human eye when the camera pans horizontally. The artifact is entirely made up in the brain and actually does not exist (it is a failure of our "persistence of vision").

Here is some 12 fps hand-held, hand-cranked silent film from 1929 that I transferred and which I left at the original frame rate. Since it is half the normal sound film rate, the judder is really bad. Watch the vertical edges of the buildings in the background and the pipe on the roof to understand what judder looks like:

1928 Backyard Kids Football in Oak Park, Illinois (https://www.youtube.com/watch?v=1YekFxBYKNM)

Repeating fields (or, less commonly, frames) so the material can be shown on 29.97 television simply makes this judder artifact even more pronounced.

It would actually be interesting to take a 24p scene with a horizontal pan and show it on a display natively, and then show the same thing on the same display with 3:2 pulldown added. I've never done that, but I'm pretty sure it would show the "artifact of 3:2 pulldown" and that artifact would simply look like slightly worse judder.

JoelHruska

17th May 2020, 02:16

PoisonDeathRay,

I tried several methods of interpolating up to 119.88 fps and then decimating back to 23.976. The best method I found was to use DaVinci.

The smoothest motion I have produced thus far was expanding to 119.88 fps using optical flow, then decimating back down with "ChangeFPS(24000,1001) in AviSynth. That produced better motion smoothing in 23.976 fps than the following methods:

1). Using ChangeFPS in AviSynth to shift to 119.88 fps and then using my QTGMC repair method.

2). Using QTGMC to interpolate additional frames into the original video (it may have been a dumb idea, but I still tried it) and then trying different combinations of SelectOdd and SelectEven. I ran an entire set of tests where I played around with various methods of increasing and decreasing frame rates to muck around with fixing VFR issues.

There are two ways to push DS9 to 119.88 fps using DaVinci Resolve Studio, and they produce different effects.

A). You can use Nearest Neighbor and 119.88 fps output settings. This will duplicate the frames in the DVD source file and results in the vastly expanded file size. Each frame is repeated 5x or 4x. It takes 4 days to upscale 1x DS9 episode if we go this route. Not much fun.

B). You can use Optical Flow and 119.88 fps output settings. This will result in hypersmoothed motion.

If you choose Option #B and then use "ChangeFPS(24000,1001) in AviSynth, however, the final output is remarkably good. Please don't think that when I say it's "Remarkably good" that I mean you won't find major differences if you inspect frame to frame. I'm sure you will. But the overall judder and motion are remarkably reduced considering I'm interpolating 80% of the frames that AviSynth then used to generate its 23.976fps output. It's closer to the kind of quality I'm trying to achieve than anything else I've found. That doesn't mean I don't think there's better. It's just gotten me the closest.

Btw:

The reason I use the following QTGMC preset is because I found the specific Input Type sequence of 2 followed by 3 to be most helpful in terms of repairing the content I wanted to repair. Other preset tests (1,1, 1,2, 1, 3, 2,1, 2, 2, 3,1, 3,2, 3,3) did not yield the same level of improvement.

But the rest of my QTGMC script? QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.2, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")

Other than the Input Types and the Sharpness, I chose the other presets hoping for maximum quality relative to original content. If there are different QTGMC presets that I should use, I'll try them.

videoh

17th May 2020, 02:26

24fps video or film exhibits "judder" You're calling judder an artifact? OK, fine, then you avoid this "artifact" by performing IVTC.

manono

17th May 2020, 02:28

24fps video or film exhibits "judder" to the human eye when the camera pans horizontally.
Sorry, but I chuckled when reading this. You do know, don't you, that the man you're instructing is responsible for both DGIndex and DGPulldown, and knows as much or more about the subject than any man alive?

Is that what JoelHruska means by 3:2 artifacts? The judder? If so, he won't lose it with QTGMC.

Do you have any evidence that silent video is meant to be played at 12fps? Was it written on the film reel or did it have accompanying notes? To me the motion looks way too slow and it should be 16fps or more.

johnmeyer

17th May 2020, 02:35

You're calling judder an artifact? OK, fine, then you avoid this "artifact" by performing IVTC.NOT TRUE! Look at the film I linked to. That artifact of the vertical lines breaking & then healing as the camera pans horizontally is called judder.

However, I completely agree that you are in safe territory by saying that judder is only caused by telecine, because any simple Google search will turn up lots of sites which say exactly that. However, as my film sample shows, that is not true. Put another way, if you display any 24p, 25p, or even 30p (somewhat rare) material at its native rate, you will see judder, with or without added pulldown (i.e., telecine).

Telecine just makes it worse.

[edit]As is often the case, feeding the correct search parameters to Google will give you the correct answers. Use these search terms and see what you get:

film 24p horizontal panning artifact (https://www.google.com/search?&sxsrf=ALeKk03EGmij3JhAUMC-A_DGslgbv6MMfg%3A1589679410401&ei=MpXAXpaEGI7S-gSU-L2ADg&q=film+24p+horizontal+panning+artifact&oq=film+24p+horizontal+panning+artifact&gs_lcp=CgZwc3ktYWIQAzIHCCEQChCgATIHCCEQChCgAToECAAQRzoFCCEQqwI6BQghEKABOgQIIRAKOggIIRAWEB0QHlCtkQNY1LcDYLW4A2gAcAF4AIABlQGIAdYTkgEEMTkuN5gBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwiW5cCk4bnpAhUOqZ4KHRR8D-AQ4dUDCAs&uact=5)

First hit links to this page:

https://www.red.com/red-101/camera-panning-speed

which says this:
Being able to control panning is important because moving too quickly can cause unpleasant visual artifacts. Objects or backgrounds may appear to flash across the screen in discrete jumps, for example, whenever the on-screen displacement is too great compared to the duration between frames. This is commonly referred to as strobing or "judder," and has happened since the early days of film.As I'm sure you know the Red Camera is an ultra-high end product used by pros, and things posted on this site are very professional and very credible.

JoelHruska

17th May 2020, 03:15

If, if, if. Do you want to do things correctly and optimally, or not? Happy with half-assed?

And what in heaven's name is an "artifact of 3:2 pulldown"?

Got it.

I have said from the beginning what I am creating. Namely: A tutorial on a method for upscaling and reprocessing DS9 that anyone with DVDs can follow. A tutorial that does not require individually hand-coded optimizations for each and every episode and relies on free software as much as possible. A "best-fit" method that still improves the results you'll get from throwing a basic MKV of the show into Topaz VEAI and watching the result.

That's the project I started off doing. Even if I decide to extend the amount of work I put into it later, that's the project I'm going to complete first. That's the project I committed to finishing for my readers (and not incidentally, my boss).

I have a deliverable I'm expected to complete on this project. So I'm working to complete it.

And what in heaven's name is an "artifact of 3:2 pulldown"?

I should not have called it an artifact. I have not known exactly what to call the short sequences of 3:2 pulldown and interlaced footage sometimes injected into the show. I know that at least *some* of the show footage was completed interlaced and apparently transferred to video in that format.

The strange thing of DS9 is that you can be watching a progressive CGI clip that suddenly has interlaced or 3:2 frames in it, but literally only 1-5 frames in a 90 second clip. You can DS9 and drop it into an upscaler without deinterlacing it at all, and what you get (except for the credits) will actually look really damn good, on the whole... but you'll get periodic horror-show frames, since the upscaler really loves to emphasize what an interlaced or 3:2 pulldown frame looks like. So it's those handful of bad frames that have to be fixed. So far, QTGMC is the filter I've found that's capable of fixing them, but that doesn't mean there aren't better solutions.

I have been calling these occasional problem frames "artifacts" to myself as shorthand. I know that's not the proper term.

zapp7

17th May 2020, 08:27

I have said from the beginning what I am creating. Namely: A tutorial on a method for upscaling and reprocessing DS9 that anyone with DVDs can follow. A tutorial that does not require individually hand-coded optimizations for each and every episode

Why is this such a big concern? If you're making avs script with trim on a per-episode basis, those only need to be created once and then can be shared with others. Same goes for override files. It's more work up front, for sure, but not really an impediment to creating scripts that other people can download and run to make their upscaled versions. Chir and I already have override files for the first 13 episodes.

JoelHruska

17th May 2020, 17:48

It's more work up front, for sure, but not really an impediment to creating scripts that other people can download and run to make their upscaled versions. Chir and I already have override files for the first 13 episodes.

Several parts to this answer.

1). I have been concerned about the amount of time it takes to create each episode's scripting. My assumption -- at least as a person who is literally learning about this process from this thread for the first time -- is that it will take me considerably longer than it might take the other experts in the thread.

2). AviSynth and the various associated tools and front ends are not simple. The more details people have to modify in script files (for example, to point to their own paths), the higher the intrinsic difficulty.

I realize that very few people will ever perform this kind of work, but the reason I started this project is because I was angry at the quality of the show on Netflix and Amazon Prime. I wanted people to be able to see it at something more fairly representative of the actual quality. So I'm hoping to create a project that is as approachable as possible, with as few dependencies and snags as possible. It may be possible to write sophisticated scripts or processing front-ends that get around this problem, but I do not know how to do it.

hello_hello

17th May 2020, 18:12

Time to reverse my position.....

Given poisondeathray posted a link to an interlaced section of an episode, I thought I'd give VFR encoding a try. I took the sample we've been playing with previously and appended the interlaced sample to the end of it. Then I ran an analysis pass with TIVTC to create the metrics files and had a look at the output. TFM breezed through the problematic credits at the start without requiring a tweak. I did have to set micmatching=0 to prevent a glitch in the interlaced section. It's smoother through the fades between shots than the methods I used previously. VFR straight out of Avisynth is probably the way to go.

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing). When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better. New samples below.

The script for the analysis pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Output="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=4, Hybrid=2, Output="D:\TDecimateMetrics.txt")

I added trims to the script after creating the timecodes to apply filtering to the various sections. There's also a version included without any filtering. TFM just doing it's thing.

Encoding script without filters:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Input="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimateMetrics.txt", tfmIn="D:\TFMMetrics.txt", mkvout="D:\TimeCodes.txt")

Resize8(640,480)

Encoding script with filters:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

Crop(8,0,-8,0, Align=true)
TFM(Input="D:\TFMMetrics.txt", micmatching=0, PP=5)
TDecimate(Mode=5, Hybrid=2, Input="D:\TDecimateMetrics.txt", tfmIn="D:\TFMMetrics.txt", mkvout="D:\TimeCodes.txt")

__film = last
__t0 = __film.trim(0, 106).MCDegrainSharp()
__t1 = __film.trim(107, 590)
__t2 = __film.trim(591, 2978).QTGMC(InputType=1, Preset="very slow", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
__t3 = __film.trim(2979, 5556).QTGMC(InputType=1, EzDenoise=1)
__t4 = __film.trim(5557, 5979).QTGMC(InputType=1, Preset="very slow", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
__t0 ++ __t1 ++ __t2 ++ __t3 ++ __t4

Resize8(640,480)

# Tried QTGMC(InputType=1, EzDenoise=1) to see how it compares, but I still
# think I prefer MCDegrainSharp for this one even though QTGMC removes more noise

VFR Encodes, Take 2.zip (https://ufile.io/gsnvcw8r) (81.1 MB)

JoelHruska,
VFR isn't that hard to do, and this may be as close to a one size fits all method as it gets, and for the samples I've created at least, it looks the best. Avisynth outputs the average frame rate and the timecodes are used to make it variable at the encoding stage. You create the scripts for the analysis pass, run them, and TIVTC creates the metrics files. When I'm doing that sort of thing I use MeGUI, as it has an analysis pass mode and it's easy to add a bunch of jobs to the job queue. The trick is to never open the analysis scripts after running them, otherwise the metrics files are over-written and you have to run them again. Then it's just a matter of adding the encoding scripts to the job queue. If you specify unique names for the files for each episode, you can develop a system to do it pretty efficiently. I added Trims to one of the encodes above to apply the same filtering as before. It's easy to do, but it's not vital. You do have to decode with repeat flags being honoured though or it won't work.
Adding the timecodes to the x264 command line is done like this:

--tcfile-in "D:\Timecodes.txt"

PS. ChangeFPS(24000,1001) isn't the best way to decimate a 120fps source. It doesn't discriminate.
TDecimate(mode=7, rate = 24.0/1.001) tries to make sure the decimated frames are always duplicates.

hello_hello

17th May 2020, 18:19

NOT TRUE! Look at the film I linked to. That artifact of the vertical lines breaking & then healing as the camera pans horizontally is called judder.

Nope. That's called "tearing" and has nothing to do with telecine or telecine judder.

https://en.wikipedia.org/wiki/Screen_tearing

If it is tearing, you're possibly the only one seeing it, but your brain isn't making it up.

A player should reverse telecine for a progressive display just as it'd bob de-interlace (assuming hard-telecine), so you should end up with the same result as if it were progressive, with whole frames displaying for 2 refreshes and some for 3 on a 60Hz display. That's what causes the 24p/60Hz "judder".

Personally I'm not that sensitive to it, plus even when the frame rate is a multiple of the refresh rate, at much the same panning speed, objects can still "jitter" across the screen.

Film strobing or judder is something else again. I think that one relates to motion blur, or a lack of it, at certain low speeds of motion, so your brain can distinguish the discrete "steps" as objects move across the screen, but that's also probably about the same speed of motion where 24p/60Hz judder is noticeable.

Stereodude

18th May 2020, 01:56

24fps video or film exhibits "judder" to the human eye when the camera pans horizontally. The artifact is entirely made up in the brain and actually does not exist (it is a failure of our "persistence of vision").

Here is some 12 fps hand-held, hand-cranked silent film from 1929 that I transferred and which I left at the original frame rate. Since it is half the normal sound film rate, the judder is really bad. Watch the vertical edges of the buildings in the background and the pipe on the roof to understand what judder looks like:

1928 Backyard Kids Football in Oak Park, Illinois (https://www.youtube.com/watch?v=1YekFxBYKNM)

Repeating fields (or, less commonly, frames) so the material can be shown on 29.97 television simply makes this judder artifact even more pronounced.

It would actually be interesting to take a 24p scene with a horizontal pan and show it on a display natively, and then show the same thing on the same display with 3:2 pulldown added. I've never done that, but I'm pretty sure it would show the "artifact of 3:2 pulldown" and that artifact would simply look like slightly worse judder.
There's 3:2 cadence judder and then there's bad 24p content. The two are being incorrectly conflated. 24p can have very aggressive pans that look fine if the shots have an appropriate amount of motion blur in them. There's a reason for the 180 degree shutter angle rule. 24p will never look right when displayed at 60Hz with a 3:2 cadence. Thankfully decent modern TVs don't do that anymore. They display 24p at 120Hz with a 5:5 cadence.

Katie Boundary

18th May 2020, 02:47

Because wikipedia is the ultimate reference and has no errors :D

There are certain predictable weaknesses in Wikipedia. For example, pages on very obscure or specific subjects tend to be seen and edited by fewer people, and the people who do edit them tend to be obsessed fanboys who want to use Wikipedia as an indiscriminate data dump or a place to publish their original thoughts on subjects, so they can have quality-control issues. Other articles on politically charged topics tend to lean left because Wikipedia's notability and verifiability guidelines make it heavily dependent on what the mainstream media choose to report on, and the MSM are notoriously left-leaning. This latter problem is exacerbated by the fact that the few right-wing media sources that exist tend to be more open about their bias and less enthusiastic about fact-checking, while most of the left-wing ones do a very good job of pretending to be objective.

However, the article on deinterlacing is not one of these edge cases.

By contrast, deinterlacing always involves degradation of the video

Wrong. Bob-deinterlacing involves degradation of the video.

The reason I have been attracted to using QTGMC is because, at least with DS9, it has done a remarkable job of fixing what I don't like visually, while introducing relatively few problems that catch my personal eye. Is there a better filter that can reduce AA, reduce noise, and remove the artifacts of interlacing / 3:2 pulldown that are still baked into the DVD when they occur -- if I'm not using Trim to lock on to specific sections for processing?

There are lots of better filters. But most of them are focused on doing one specific thing, so you'd need to use all of them and in the right order, whereas QTGMC is a bunch of filters from a bunch of different people all tied together in a big knot.

For pure deinterlacing, you've already seen my custom method with NNEDI3, TFM, and Interleave. But this is purely for deinterlacing. It doesn't do any kind of cleanup or denoising.

For reducing general spatial noise, the best filter I've seen is TNLmeans, made by Tritical, the same author who gave us TFM. I can't give any advice on AA, as I've never much cared about it.

My biggest complaint with QTGMC is that it warps the frames in a half-assed attempt at performing a shitty form of motion interpolation. This is literally spending more CPU cycles to get a wrong result on purpose, AND it gets in the way of decimating the film sections back down to 24 fps.

Sorry, but I chuckled when reading this. You do know, don't you, that the man you're instructing is responsible for both DGIndex and DGPulldown, and knows as much or more about the subject than any man alive?

Videoh is Donald Graft?

Now that you mention it, that does feel like something I might have been told once before and then forgotten.

the upscaler really loves to emphasize what an interlaced or 3:2 pulldown frame looks like.... I have been calling these occasional problem frames "artifacts" to myself as shorthand. I know that's not the proper term.

It sounds like you're talking about ordinary interlaced or "combed" frames. Those aren't artifacts, but they really don't like being resized.

Katie Boundary

18th May 2020, 03:07

By the way, is there a way for us to send entire 1-2 GB VOB files to each other? I'd like to see the source footage myself but I can't buy my own copies yet because my Coronabucks haven't arrived in the mail.

huhn

18th May 2020, 07:17

there are without any doubt ways to use the internet to send files.
the practice is samples only for a good reason.

manono

18th May 2020, 08:45

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing).
Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example, you do this:

tdeintted = QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

That's from some included manual or other for TIVTC.

manono

18th May 2020, 08:53

By the way, is there a way for us to send entire 1-2 GB VOB files to each other?
If you use gmail at all, or other Google products, you should have access to your own Google Drive. I forget how large the files you can up/download. 5 GB, maybe? I hardly use it and I'm sure others know more about it.

hello_hello

18th May 2020, 09:12

JoelHruska,
Katie doesn't use QTGMC, and I suspect her latest advice comes without the realisation it has a progressive mode. QTGMC bob de-interlaces with nnedi3 by default, the same method she uses to create an interlaced clip, although oddly enough, after declaring QTGMC to be horrible, she's posting in the QTGMC thread looking for advice on how to get it to work again (https://forum.doom9.org/showthread.php?p=1912436#post1912436).

After bob de-interlacing with nnedi3, QTGMC applies various filters to reduce shimmering, denoise and sharpen etc. A process it gives you a fair amount of control over. Katie's method apparently involves de-interlacing, then maybe KLMeans for denoising, ignoring alaising, and running a bunch of unnamed filters in the correct order to produce a similar result. I'd like someone to tell me what motion-interpolation QTGMC does exactly, aside from interpolating the missing scanlines with nnedi3, which doesn't really qualify as the sort of motion interpolation people like johnmeyer and Katie accuse QTGMC of doing.

I get what Katie is trying to do with her method, so in that respect I'm not sure why there's an IVTC vs de-interlacing argument going on about it, although it appears to be a hard way to produce a "not as good" a result as TFM can do on it's own in VFR mode, assuming a VFR output is desired.

TFM is a field matcher and de-interlacer. For hybrid sources it field matches the telecined parts and de-interlaces the interlaced parts. It has a clip2 argument for taking pixels from a de-interlaced version of the clip to replace combed pixels instead of de-interlacing itself. The IVTC field matching process can sometimes result in combing when there's no perfect match (that could probably be caused by encoding artefacts amongst other things), in which case the newly created progressive frames need to be de-interlaced too. So for Katie's method every frame is bobbed to full height by nnedi3, TFM does it's thing and where there's combing it takes the pixels from the nnedi3 clip, the idea being that most of a progressive frame remains untouched and only the combed pixels are repaired, or TFM can be configured to use the entire frame from the nnedi3 clip when it detects combing, which would be the only time it could be accused of "bob deinterlacing" a telecined/progressive source.

In Katie's case she's using two instances of TFM, while forcing them to use specific matches, so in theory each TFM instance produces a 29.97fps clip where the telecined parts are field matched, combing is repaired, but without decimating the duplicate frames that result, and the interlaced parts are (bob?) de-interlaced. The two TFM instances combine to produce a 59.94fps clip. In a perfect world you'd have film sections where the progressive frames repeat in patterns of two and three duplicate frames, and the interlaced sections are bob de-interlaced for 59.94fps.

I guess the problem is, you can't just throw the same Katie method at every clip and expect a great result. If TFM is taking pixels to fix combing from a de-interlaced clip that has problems of it's own you could replace the combed sections with shimmering or additional aliasing, and I suspect even if the de-interlaced clip is clean, taking pixels from a second clip can still cause shimmering and aliasing if they don't match the rest of the frame exactly, so for the best result it pays to experiment. Even if the end result is perfect though, you've still got to decimate all the repeated frames in the film sections for a VFR, unless you want a 59.94fps clip, and given Katie consistently refers to doing just that, I'd be keen to learn how she does it.

I suspect the biggest obstacle to IVTCing and de-interlacing DS9 with TFM, is it's default de-interlacing doesn't work well with the CGI and causes lots of aliasing. Until I experimented with VFR encoding though, I'd concentrated on taking the combed pixels from a de-interlaced clip and hadn't considered how TFM's de-interlacing looks. After changing it's de-interlacing method, TFM's own de-interlacing looks quite good, and I suspect Katie's method might look better without the de-interlaced clip if different TFM de-interlacing was used. I only tried PP=5 for an encode, but I did briefly try simple blend de-interlacing (pp=2), and for the CGI sections that seemed to work well too.

Anyway, for DS9, based on my VFR examples, I wouldn't bother with any of that. Running an analysis pass, then letting TFM do it's thing using it's own de-interlacing (just not the default method) and outputting timecodes so the film sections are 23.976fps and the interlaced sections are 29.97fps has produced by far the best result of anything I've tried so far, without the need for a second deinterlaced clip and good enough not to require additional filtering to clean it up. Have a look at the samples I uploaded yesterday if you haven't already.

hello_hello

18th May 2020, 09:24

Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example, you do this:

tdeintted = QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

That's from some included manual or other for TIVTC.

That's what I've been doing for the samples I've uploaded, aside from the VFR encodes, although I suspect this might work better as it'd ensure the frames being de-interlaced by QTGMC are the same frames TFM wants to repair.

tdeintted = tfm(pp=1).QTGMC(FPSDivisor=2)
tfm(clip2=tdeintted).tdecimate()

But even then, because the source clip contains lots of aliasing, depending how closely the de-interlaced clip matches the clip TFM wants to repair, replacing combed pixels with QTGMC pixels could possibly cause shimmering that wasn't there before.

For DS9, if you change TFM's own de-interlacing it looks like the result will be better. I haven't played with every imaginable combination of settings, but for the VFR encodes I uploaded yesterday, switching to pp=5 and not using a QTGMC deinterlaced clip has produced the best result so far, compared to the rest of the samples I've uploaded, and I played around quite a bit with TFM's settings for those encodes.
The VFR encodes are clean enough for repairing the CGI sections with QTGMC to be fairly optional (there's a tad less shimmering than when the source is being IVTC'd by my video card), and even though the CGI section is telecined, VFR rate appears to allow TFM to treat the fades from one shot to the next as purely interlaced (where hard telecine is overlaid on hard telecine with a different field order, I think), and those transitions are quite smooth. I guess my video card does the same, as the source looks smooth through those transitions too, without any combing.
The "problem" telecined CGI section is roughly from frame 570 to frame 3000, after decimation.

# timecode format v1
Assume 29.970030
# TDecimate v1.0.3.1 by tritical
# Mode 5 - Auto-generated mkv timecodes file
0,971,23.976024
992,1831,23.976024
1837,1840,23.976024
1846,2585,23.976024
2591,2998,23.976024

For a VFR encode, if you use a de-interlaced clip for TFM you have to use it for the analysis pass too (otherwise TDecimate complains the CRC no longer matches the analysis clip), so if it's a QTGMC de-interlaced clip, it'd slow down a normally fast analysis pass quite considerably.

zapp7

18th May 2020, 09:49

There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample (https://ln2.sync.com/dl/5dd47a7d0/tanjt92v-e52sztfm-t9k5qzud-2tgukgp5)

hello_hello

18th May 2020, 10:19

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample (https://ln2.sync.com/dl/5dd47a7d0/tanjt92v-e52sztfm-t9k5qzud-2tgukgp5)

If they used the same shot, or a similar one, in another episode that's much better quality, you could always swap them out. :)

videoh

18th May 2020, 16:21

There's 3:2 cadence judder and then there's bad 24p content. The two are being incorrectly conflated.
Quite right, Stereodude. Also, tearing and pulldown are being conflated. Actually, pure 3:2 pulldown can be easily removed so it is rather strange to call it an artifact.

Katie Boundary

18th May 2020, 17:46

Yes it does, but it's easy enough to use any deinterlacer you like. To use QTGMC for post-processing, for example...

OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.

If you use gmail at all, or other Google products, you should have access to your own Google Drive. I forget how large the files you can up/download. 5 GB, maybe? I hardly use it and I'm sure others know more about it.

I do not use gmail because Google is evil.

There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?

TNLmeans or KNLmeans. Pure motherf***ing magic.

Should I re-send my friend request that you ignored? I feel so bad about Boris kicking you off my web site. You can come back any time and I will personally protect you. Your knight in shining armor!

Well the only reason I was on your website in the first place was to ask that ONE question about 48 -> 44.1 khz conversion and what the differences were between the quality settings. It's not like I was going to stick around one way or another. I will, however, continue to hawk DGindex as the #1 best way to get video content from a VOB file into AVIsynth :)

Joel, meet the legendary Donald Graft, also known as Neuron2. Back in the old days, he created he original Decomb plugin, which was the preferred deinterlacing tool used in the older versions of READFAG, written by the equally legendary Justin Emerson, also known as ErMaC. He also forked DVD2AVI (another program endorsed by early versions of READFAG) to create DGindex. Decomb has since been made obsolete by Tritical's TIVTC, but DGindex is still in use.

videoh

18th May 2020, 18:29

A little aside on tritical's stuff. It is heavily based on the Decomb design. He made it when I went AWOL on further development of Decomb due to my father's illness. tritical did a fantastic job on picking up the torch. I was actually the first person to develop IVTC based on field-matching plus decimation, although it wasn't my original idea. A poster at Avery Lee's VirtualDub forum first suggested field-matching plus decimation, but he was not a developer. I wish I could remember his name; sadly the forum is now off-line and I can't go look it up. If anybody remembers, please let me know.

Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.

The good old days!

JoelHruska

18th May 2020, 19:24

Hidden dependency: Probably of little interest to anyone except myself (since it's part of one of my own workflows), but in the name of sharing information:

My repair scripts that utilize QTGMC will not run against output from DaVinci Resolve Studio. Once the output has passed through DaVinci Resolve Studio and been converted to 119.88 fps, QTGMC's progressive repair mode will fail to engage (at least, when used in Staxrip). Instead, the application will hang. It will continue to do this, even if the framerate on the DRS output is decimated back to 23.976 fps using ChangeFPS(24000,1001). I am now attempting it on footage that used TDecimate instead of ChangeFPS and will report if it works.

You can still engage QTGMC in a standard preset mode, i.e., QTGMC("Medium") I am not sure if it is something specific about the flags I use or just the attempt to engage the progressive repair mode, but it doesn't work.

Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.

However, my Step #3 cannot be meaningfully applied in this workflow.

Not necessarily a problem or anything, just logging that it occurs.

JoelHruska

18th May 2020, 19:28

Zapp7,

Crapstation is a problem I haven't solved yet, either. Like you, I've been very unhappy to discover how bad the early footage is.

videoh

18th May 2020, 20:41

Probably of little interest to anyone except myself Correct, nobody cares.

hello_hello

18th May 2020, 20:41

OR, you could just tweak TFM's settings. Setting cthresh and mthresh to 2 or less, and using clip2 to retrieve pixels from a temporally-aware bob-deinterlacer like Yadif, works miracles.

Maybe if Katie wasn't ignoring so many people, she'd be aware of the full conversations. The statement was "TFM's default de-interlacing sucks balls for the CGI sections. When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better."

TFM is capable of six different types of deinterlacing.

Still, what do I know, so I ran another VFR encode using cthresh=2, mthresh=2 and Yadif as the de-interlaced clip.
Aside from the fact you've only got to look at the timecodes to see the low cthresh and mthresh settings cause TFM to think there's much more combing, and therefore to declare multiple slices of the film sections are interlaced instead of telecined, replacing the pixels detected as combed with Yadif pixels causes the same problems as TFMs default de-interlacing.
I could tell when TFM was going into interlaced mode for the Katie method, if for no other reason, because the stars would blink out.

cthresh=2, mthresh=2, Yadif de-interlacing
https://i.postimg.cc/rdWBzCns/1-katie.jpg (https://postimg.cc/rdWBzCns)

TFM pp=5
https://i.postimg.cc/5YVZzYT3/1-tfm.jpg (https://postimg.cc/5YVZzYT3)

cthresh=2, mthresh=2, Yadif de-interlacing
https://i.postimg.cc/CBs9sMVV/2-katie.jpg (https://postimg.cc/CBs9sMVV)

TFM pp=5
https://i.postimg.cc/30rVSMK0/2-tfm.jpg (https://postimg.cc/30rVSMK0)

manono,
I may have debunked my theory that this:
DeintClip = TFM(pp=1).Yadif()
Should be better than this:
DeintClip = Yadif()

When I compared the two using Katie's de-interlacing, it sometimes looked better, but other times it didn't, then for the last frame on a scene change where the next shot is mainly black space, I found this. I'm not sure I fully understand why, although it's an example of how much of a progressive frame can be unnecessarily de-interlaced with low cthresh and mthresh settings.

https://i.postimg.cc/XrTMVDgt/TFMThen-Deint.jpg (https://postimg.cc/XrTMVDgt)