Deep Space Nine upscale project

[Katie Boundary]

The good old days!

Haha yes. When Smartripper was new, monitors were fullscreen, ATI still made "All-in-Wonder" cards, and we all got our anime music videos from Kazaa and Morpheus because Youtube didn't exist yet. I wasn't into video editing back then, but my mentor (Fuzzy Chickens) was, so I have been instructed in the Old Ways :)

Of course the inspiration for DGMPGDec (DGIndex plus DGDecode) was jackei's amazing DVD2AVI. Trying to remember who wrote the first Avisynth DLL working with DVD2AVI...our late and beloved trbarry? Help me out guys.

Mpeg2dec3.dll is credited to "MarcFD, Nic, trbarry, Sh0dan and others", but was based on mpeg2dec2, which is credited to Mr. Barry alone. Mpeg2dec2 was in turn based on mpeg2dec, credited to "Dividee and others"

[Katie Boundary]

h_h, there's something very wrong with your settings or your script. That looks straight YADIFed, with no TFM at all (or like TFM is set to one of its "dumb" deinterlacing modes, which results in the same thing).

[hello_hello]

Typically I have 1). processed the DVD footage in AviSynth, 2). changed the frame rate in DaVinci Resolve Studio, and then 3). upscaled the final product in Topaz VEAI. In this workflow I decided to test; 1). changing the frame rate, 2). upscaling the video, and 3). Attempting to process the upscaled footage the same way I typically do, just to see an apples-to-apples comparison of the final output. I did this to avoid throwing data away early in QTGMC before performing the upscale. It took eight days to upscale the video in this fashion, which is why I'm just now reporting on the results of a test that I started like four pages ago.

Wow, and that's easier than adding a few Trims to a script?
Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?
The whole idea of 120fps is it's a multiple of both 24 and 60, so even though there's lots of repeated frames the interlaced sections still play at their original speed, as do the film sections.

Thinking about it, TDecmiate(mode=7, rate = 23.976) mightn't be the best tool for the job. The output will have a constant frame rate, so to achieve 23.976fps you might end up with the interlaced and film sections effectively playing at the wrong speeds. If the Avisynth output was 23.976fps though, it should be fine.
Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?

After trying some VFR encodes myself, using TFM for a VFR encode from Avisynth seems like the best method anyway, so I don't know why you wouldn't want to try it.

[hello_hello]

h_h, there's something very wrong with your settings or your script. That looks straight YADIFed, with no TFM at all (or like TFM is set to one of its "dumb" deinterlacing modes, which results in the same thing).

There was nothing wrong with my settings. I ran a VFR encode using the settings you recommended for TFM.

Crop(8,0,-8,0, Align=true)
DeintClip = Yadif()
TFM(Input="TFMMetrics.txt", clip2=DeintClip, cthresh=2, mthresh=2, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)

Not every frame looks so bad, just the ones where TFM was taking a lot of pixels from Yadif.

The "good" screenshots used the following script:

Crop(8,0,-8,0, Align=true)
TFM(Input="TFMMetrics.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="TDecimateMetrics.txt", tfmIn="TFMMetrics.txt", mkvout="TimeCodes.txt")
Resize8(640,480)

[Katie Boundary]

It shouldn't be taking ANY pixels from YADIF except in the parts that are interlaced, in frames that are interlaced. I've never seen results like that from my own scripts.

[hello_hello]

Your cthresh and mthresh settings are so low it thinks large chinks are interlaced, although I assume you actually mean combed.

I don't need to get into a debate about it. It's pointless anyway. I can still remember when you argued the evils of IVTC with TFM, or deinterlacing with NNEDI3 and Yadif.

I've run quite a few test encodes using the dual TFM method you love, a heavily modified version of it using two QTGMC de-interlaced clips to make it not suck, and encodes using a single instance of TFM with QTGMC de-interlacing that was virtually as good (for when a constant 23.976 frame rate was required), but after trying VFR encoding when poisondeathray linked to a sample of a purely interlaced CGI section, the above VFR output using PP=5 has easily produced the best result.
I've linked to sample encodes in quite a few posts if you want to look for yourself, assuming you can play MKVs like the rest of us now, but please don't just tell me I'm wrong or I've used the wrong settings if you think you can do better, because it's easy to upload a sample of your own to prove it.

There's links for the sample source files I used earlier in the thread somewhere.
Edit: I think this was the first one:
https://forum.doom9.org/showthread.php?p=1906994#post1906994
Second one here:
https://forum.doom9.org/showthread.php?p=1912067#post1912067

I joined them like this for encoding:

A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B

VFR samples here:
https://forum.doom9.org/showthread.php?p=1912388#post1912388
CFR sample here:
https://forum.doom9.org/showthread.php?p=1911841#post1911841

[Katie Boundary]

I never argued that those were "evil". Lies like that are why you're on my ignore list.

There is another argument to be made in favor of pp=5, however. My current deinterlacing algorithm heavily favors field-accuracy, the representation of the original fields and frames as accurately as possible in the newly deinterlaced frames, and the minimization of blended frames and residual interlacing. Why? because the obsessive-compulsive demons who whisper in my ear demanded it. It can't fully eliminate visible interlacing without a high false positive rate, which it conceals pretty well with temporally-aware bobbers (Yadifmod2). But there are types of content where that's not the right tradeoff to make, and blending is better than an autistically literal representation of the original content. The most obvious is the aforementioned retrograde field behavior, but fades, credit animations, and a few other effects are harmed less by blending than the rest of the frame is by bobbing. There are also times when the last field of one clip and the first field of the following clip will be interlaced together and encoded as a single frame, which causes their chroma planes to merge; in such cases, each field benefits from being matched and blended with the nearest field from its own clip rather than bobbed. This doesn't mean that cthresh and mthresh should be set higher, however. You can't do that without unacceptable amounts of visible interlacing getting through.

[JoelHruska]

Wow, and that's easier than adding a few Trims to a script?

Testbeds do 100% of the work. I set up the encode, walk away, come back 4-8 hours (or 4-8 days) later and check the output. The Trim method is something I need to learn from scratch, using tools I have not found easy to work with. I've been experimenting with the code you've posted and trying to figure out how this different approach to editing works, while continuing some evaluation on my own previously-developed workflow.

Why on earth are you outputting 120fps if you're just going to decimate it back to 23.976?

To see what the outcome would be. Exactly the same reason that I have two encodes running downstairs since last night, converting the 8-day Emissary encode I did -- one with ChangeFPS(24000,1001) and one with the TDecimate command you suggested, to see which handles the content better, and how smooth the output is.

I wanted to compare the outputs against what native 23.976 fps video looked like, versus video run through progressive repair mode in QTGMC, etc, etc.

Maybe I'm missing something, but I don't get it. If the output from Avisynth is 23.976 and you're decimating to 23.976, why the conversion to 120fps and back? Aren't you just creating a heap of duplicate frames that you have to process, only to throw them away later?

Well, keep in mind I reversed the workflow in this manner to test the impact of preserving detail for longer. I wanted to compare the visual impact of running QTGMC later in the workflow, which meant keeping the workflow identical except for running QTGMC later in the process. Unfortunately, this proved impossible, because my script won't execute after running DaVinci Studio Resolve.

EDIT: Using TDecimate instead of ChangeFPS doesn't change whether or not my repair script will run after DaVinci RS. It still will not. 2

I will not be retaining this workflow. I'm not spending 4 days of upscaling per episode.

But as for why I'd try it in the first place? Because I am new at this, and do not know what will work and what will not, and have been willing to burn CPU cycles on things I was pretty sure wouldn't work to see what they could teach me. As one simple example: I noticed using QTGMC in default mode would create an interpolated frame in-between each "regular" frame. After I noticed this, I ran multiple tests to look at the difference between SelectOdd and SelectEven. At one point, I double-ran QTGMC and then tried "SelectOdd,SelectOdd," "SelectOdd,SelectEven", "SelectEven, SelectEven", etc. Why? Because I wanted to understand what kind of output would be produced.

All of them were garbage, but some of them were more garbage than others.

"Test it and see what it looks like," has been my guiding philosophy on this endeavor.

[wonkey_monkey]

Correct, nobody cares.

Why didn't you just not say anything?

[hello_hello]

I never argued that those were "evil". Lies like that are why you're on my ignore list.

I'm on your ignore list because you're a child.

Obviously I'm not on it now though, so you're lying about that, but given you're claiming I'm on it, you couldn't have read my posts, which means you couldn't have replied, and therefore I don't have to bother reading it.

I said you "argued the evils". How many times have you accused others of not being able to understand plain English?


So, with almost no noticeable difference between AVIsynth's built-in Bob() filter and an actual proper field match, why should I believe that there's a huge difference between Bob() and more computationally intensive, easier-to-screw-up bob-deinterlacers like Yadif and NNEDI?

[hello_hello]

I wanted to compare the outputs against what native 23.976 fps video looked like, versus video run through progressive repair mode in QTGMC, etc, etc.

Keep in mind temporal filters need to see motion. If you up the frame rate there's going to be repeated frames, and those frames will be identical, so any noise will become static and noise removal won't work as well. That sort of thing. Even if they're interpolated frames, I think you'd be better of running any filtering first.

[manono]

Once the output has passed through DaVinci Resolve Studio and been converted to 119.88 fps, QTGMC's progressive repair mode will fail to engage (at least, when used in Staxrip).
All you use DaVinci for is to interpolate to 119.88fps? You can do that in AviSynth. I actually like that method of getting interlaced 29.97fps to 23.976 with a minimum of stuttering. But, I also agree that getting back to 23.976fps using ChangeFPS isn't optimal.

And I would also probably suggest the interpolate route only with the video portions and IVTC the film portions. But if you're looking for a set-it-and-forget-it method where you don't have to go through the episodes looking for the GCI parts, I suppose that'll do.

[videoh]

Why didn't you just not say anything? Why didn't you mind your own business?

[videoh]

"Test it and see what it looks like," has been my guiding philosophy on this endeavor. You'll get much better results, and much faster, when starting from valid theoretical considerations, rather than groping in the dark. But hey, even a blind pig sometimes finds a truffle.

[JoelHruska]

You'll get much better results, and much faster, when starting from valid theoretical considerations, rather than groping in the dark. But hey, even a blind pig sometimes finds a truffle.

Indeed. But when one does not know which theoretical considerations are valid and which are not, groping in the dark -- and a little use of the Mark I Eyeball -- will teach you things regardless. The fact that I had never seen what an interpolated frame looked like until I saw one as a result of my first QTGMC run didn't prevent me from recognizing what I was looking at.

Some of the work I have done has been for no other purpose than to show me things about how video behaved when manipulated. I have experimented with accelerating and decelerating frame rates. I ran Handbrake rips against MakeMKV rips so I could see the subtle difference in how they handled color. I learned about pixel aspect ratios from noticing differences in output between MakeMKV and Handbrake. When I wanted to see the impact of incorrect deinterlacing filters on content, I ran deliberately-incorrect deinterlacing filters and looked at the results.

[JoelHruska]

Time to reverse my position.....

Given poisondeathray posted a link to an interlaced section of an episode, I thought I'd give VFR encoding a try. I took the sample we've been playing with previously and appended the interlaced sample to the end of it. Then I ran an analysis pass with TIVTC to create the metrics files and had a look at the output. TFM breezed through the problematic credits at the start without requiring a tweak. I did have to set micmatching=0 to prevent a glitch in the interlaced section. It's smoother through the fades between shots than the methods I used previously. VFR straight out of Avisynth is probably the way to go.

Edit: It turns out TFM's default de-interlacing sucks balls for the interlaced CGI parts (lots of aliasing). When I realised it didn't look anywhere near as good as my video card's de-interlacing, I switched it out for PP=5. Much better. New samples below.

The script for the analysis pass:



I added trims to the script after creating the timecodes to apply filtering to the various sections. There's also a version included without any filtering. TFM just doing it's thing.

Encoding script without filters:



Encoding script with filters:



VFR Encodes, Take 2.zip (https://ufile.io/gsnvcw8r) (81.1 MB)

JoelHruska,
VFR isn't that hard to do, and this may be as close to a one size fits all method as it gets, and for the samples I've created at least, it looks the best. Avisynth outputs the average frame rate and the timecodes are used to make it variable at the encoding stage. You create the scripts for the analysis pass, run them, and TIVTC creates the metrics files. When I'm doing that sort of thing I use MeGUI, as it has an analysis pass mode and it's easy to add a bunch of jobs to the job queue. The trick is to never open the analysis scripts after running them, otherwise the metrics files are over-written and you have to run them again. Then it's just a matter of adding the encoding scripts to the job queue. If you specify unique names for the files for each episode, you can develop a system to do it pretty efficiently. I added Trims to one of the encodes above to apply the same filtering as before. It's easy to do, but it's not vital. You do have to decode with repeat flags being honoured though or it won't work.
Adding the timecodes to the x264 command line is done like this:

--tcfile-in "D:\Timecodes.txt"

PS. ChangeFPS(24000,1001) isn't the best way to decimate a 120fps source. It doesn't discriminate.
TDecimate(mode=7, rate = 24.0/1.001) tries to make sure the decimated frames are always duplicates.

Just an update on this. I had no problem getting the first part of the script working and successfully creating the analytics for TDecimate and TFMN.

I wasn't able to get the second part of the script to work, however. I could not find a way to load the D2V file to get the actual frame count off the two VOB files it is comprised of. When I attempted to load the individual VOBs in VDub2, they failed to load properly and displayed nothing like the actual frame count.

The MKV file output created by the analytics passes must not have the same number of frames as the VOB file it is based on. Setting the maximum number of frames in the clip to the same number of frames as the MKV file produced an Error 5 & 6 about how every frame must be accounted for (I do not recall the exact phrasing, but the implication was that the final frame number was incorrect).

So. If I cannot get the proper frame count from loading the VOB files in Vdub2, and I cannot use the MKV file output from the analysis pass, how do I know what the frame count is from the D2V file?


Also: Uncertain if your reference to timecodes is a reference to timecodes I'm supposed to have extracted from somewhere, or a reference to timecodes I'm extracting here for use in a separate encode step.

When I use DVDDecrypter to create VOB files and then input them into DGIndex to create a D2V file, I'm not creating a separate timecodes.txt file at any point -- at least not yet. Is this a step I need to be taking to make this editing method work?

[Katie Boundary]

Obviously I'm not on it now though

No, you're still on it, but I clicked on "view post" out of curiosity.

TFM() at default settings still delivers less desirable results than trbarry's uncomb(), and finding settings that delivered acceptable results took an unreasonable amount of time. Finding a practical application for its few advantages over uncomb() took even longer. Getting NNEDI3 to work at all can be torture. YADIF was somewhere between the two: it's tricky to get working, but not as much as NNEDI3, and while it doesn't have a lot of settings that need tweaking, it still took forever to find an application where its sole advantage over bob(b=0) really mattered.

None of these are "evils" that I was arguing. They're facts that you and several other people on the forum ignored while you criticized me for using tools that were guaranteed to work and to deliver acceptable results.

The biggest problem in my TFM+Interleave method is that it relies on TFM, and TFM seemingly doesn't look for interlacing, but for horizontal edges. There's a range of cthresh and mthresh settings where it'll incorrectly flag horizontal edges as interlacing while letting real interlacing through. If there was a filter that basically did what TFM does, but actually looks for interlacing instead of horizontal edges, then I could set cthresh and mthresh (or their equivalents) much higher, and there wouldn't be so much unnecessary bobbing of progressive content.

When I wanted to see the impact of incorrect deinterlacing filters on content, I ran deliberately-incorrect deinterlacing filters and looked at the results.

Well, deinterlacing filters don't get much more deliberately incorrect than QTGMC :D

[Groucho2004]

No, you're still on it, but I clicked on "view post" out of curiosity.I believe it was asked before - How old are you?

[Katie Boundary]

I believe it was asked before - How old are you?

Eight and a half.

[videoh]

how do I know what the frame count is from the D2V file? DGIndexNV puts the counts of coded and playback frames at the bottom of the DGI file (equivalent of D2V file). If you have an nVidia card, DGDecNV could help a lot.

For DGIndex, you can enable the info log and then the count of coded frames (also numbers of repeated fields/frames) will be in the log file.

[Katie Boundary]

Don, do you know why TFM loves to incorrectly flag horizontal edges as interlacing while it ignores real interlacing?

[Katie Boundary]

Well, there can be real detail at the same vertical spatial frequency as interlacing.

That's extremely unlikely, at least over areas larger than a few pixels

people say you don't give samples, but for your protector?

You are not my protector. You are not my knight in shining armor. You're someone who was an asshole to me 3 years ago for no reason, and whose current behavior I find extremely suspicious but not annoying or objectionable.

I'll see what I can do about uploading an Andromeda clip.

It's an interesting theoretical problem. Maybe we could bring the powerful Katie intellect to the problem. Think about it, dear, how would you distinguish between the two?

I can think of several approaches, none of which I think anyone here would bother to turn into a plugin. But for shits and giggles, here's one approach: Take any stack of pixels, 1 pixel wide by 5 pixels tall (same as TFM). Find the average luma value of pixels a, c, and e. Call it X. Find the average luma value of pixels b and d. Call it Y. Does the difference between x and Y exceed the differences between a and c, between c and e, between a and e, AND between b and d? Then flag the area as interlaced.

That approach might work better than TFM. Might not. I'd love to see its failure modes and tweak it accordingly.

[videoh]

Seems you lack a sense of humor. I won't trouble you again.

[Katie Boundary]

https://www.sendspace.com/file/urf1xo
https://www.sendspace.com/file/0iv2h6

Go ahead and try to find a combination of settings that will deinterlace the text in the first clip...

https://i.imgur.com/fxbqiyQ.png

...and the area in the red circle without wrecking the area in the yellow-green circle:

https://i.imgur.com/Wd4Bpxk.png

Seems you lack a sense of humor. I won't trouble you again.

Or perhaps you just weren't funny.

[wonkey_monkey]

Why didn't you mind your own business?

That's pretty much the gist of my question. You could have not said anything at all instead of actually going out of your way to be offensive.

[Katie Boundary]

Did one of Don's posts get deleted? I'd like to state for the record that I did not report it.

Anyway, I think we're on way too many tangents right now. We should focus on the retrograde field behavior question. Joel, load one of the VOB files into DGindex, use the "[" and "]" keys to mark the beginning and end of the clip you want to share, and select "File -> Save project and demux video". That'll create both a d2v file and an m2v file. Upload the m2v file to a file-hosting site like Sendspace and then post a link to it here so we can all run our own tests and see what you're dealing with and how to fix it, instead of telling you what tests to run and waiting for you to describe the results. When people here talk about providing a "sample", this is what they mean.

[hello_hello]

Well, deinterlacing filters don't get much more deliberately incorrect than QTGMC :D

Maybe if you didn't keep ignoring posts, by now you'd know QTGMC de-interlaces with NNEDI3 by default. The required plugins list:

MaskTools2, MVTools2, nnedi3, RgTools, Zs_RF_Shared.avsi

Everything from there attempts to repair problems. Temporal smoothing, denoising, sharpening etc. The things you've implied you'd do yourself after de-interlacing, using unnamed filters in the correct order.

Without your fingers in your ears you'd probably know QTGMC has a lossless mode. The original scanlines are output pixel for pixel. It's rarely the best mode though, because it faithfully reproduces artefacts and causes shimmering unless the source is very clean. There's a semi lossless mode that outputs the original scan lines before the final temporal smooth.

If you didn't ignore people who don't agree with you, you'd probably know QTGMC has source-match modes that attempt to match the output to the source without being lossless, and if you read the info on the wiki you'd know QTGMC's arguments give you a great amount of control over what it does.

If you looked at the optional plugins list you'd see it includes KNLMeansCL as a denoising option. Didn't you recommend using an NL-means denoising algorithm after de-interlacing yourself?

Of course QTGMC is deliberately "incorrect" as you put it, because it's defaults generally output a repaired de-interlaced clip, not a purely de-interlaced one. If you want the latter, then just keep using NNEDI3 or Yadif and apply your unnamed filters in the correct order afterwards.

[hello_hello]

Upload the m2v file to a file-hosting site like Sendspace and then post a link to it here so we can all run our own tests and see what you're dealing with and how to fix it, instead of telling you what tests to run and waiting for you to describe the results. When people here talk about providing a "sample", this is what they mean.

I linked to two samples for you in a previous post. They're the one's he's using. We're still waiting to see your results.

[hello_hello]

I wasn't able to get the second part of the script to work, however. I could not find a way to load the D2V file to get the actual frame count off the two VOB files it is comprised of. When I attempted to load the individual VOBs in VDub2, they failed to load properly and displayed nothing like the actual frame count.

The MKV file output created by the analytics passes must not have the same number of frames as the VOB file it is based on. Setting the maximum number of frames in the clip to the same number of frames as the MKV file produced an Error 5 & 6 about how every frame must be accounted for (I do not recall the exact phrasing, but the implication was that the final frame number was incorrect).

So. If I cannot get the proper frame count from loading the VOB files in Vdub2, and I cannot use the MKV file output from the analysis pass, how do I know what the frame count is from the D2V file?

I'm not sure what you're doing exactly, but you wouldn't open the vob files in VD or another GUI, you'd open the script. The script should open the vob files with mpeg2source. Once you've created the first pass script and run it, which I assume you've done by opening a script that opens the vob files with mpeg2source, you'd open the second pass script.

As I normally have MeGUI running, to combine the samples, I open the vob/m2v files with it, and get it to index them with DGIndex. MeGUI then creates a script to open each which I save. They look like this:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

and

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\space battle.d2v")

From there I copy/paste and manually create a script with Notepad, adding the other stuff to it.
My method it is to make the first pass and second pass scripts a single script, as it helps to keep things organised when you're working in batches. To switch from 1st pass mode to 2nd pass mode, I comment out the 1st pass lines and uncomment the rest. ie analysis pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B
Crop(8,0,-8,0, Align=true)

TFM(Output="D:\S01E01TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=4, Hybrid=2, Output="D:\S01E01TDecimate.txt")

# TFM(Input="D:\S01E01TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=5, Hybrid=2, Input="D:\S01E01TDecimate.txt", tfmIn="D:\S01E01TFM.txt", mkvout="D:\S01E01Timecodes.txt")

# Resize8(640,480)

Encoding pass:

LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
A = DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
B = DGDecode_mpeg2source("D:\space battle.d2v")
A + B
Crop(8,0,-8,0, Align=true)

# TFM(Output="D:\S01E01TFM.txt", PP=5, micmatching=0)
# TDecimate(Mode=4, Hybrid=2, Output="D:\S01E01TDecimate.txt")

TFM(Input="D:\S01E01TFM.txt", PP=5, micmatching=0)
TDecimate(Mode=5, Hybrid=2, Input="D:\S01E01TDecimate.txt", tfmIn="D:\S01E01TFM.txt", mkvout="D:\S01E01TimeCodes.txt")

Resize8(640,480)

So for encoding a bunch of episodes I create the appropriate scripts for each, add them all to MeGUI's job queue and let it run all the analysis passes. Obviously when encoding in batches, the metrics files need to be appropriately named for each script, as I did for the example above.
Once that's done I edit the scripts for the encoding pass. The Timecodes file is automatically created by TDecimate when the encoding script is opened. Giving it a unique name for each episode is optional. If you don't it'll be over-written each time a script runs. If you do, you need to modify the x264 command line for each job you add to the queue (it doesn't matter if the timecodes file doesn't actually exists at that stage). If you don't want to give it a unique name, you can keep the same command line for each encode. ie
--tcfile-in "D:\TimeCodes.txt".

If you want to process the AVS output first, rather than encode it, or especially if you don't use the timecodes when encoding (I think it's better to do it that way though), you can add the timecodes when muxing the output with MKVToolNixGUI for a VFR that way, so you'd want to give the timecodes files unique names. I generally get the x264 encoder to write the output directly to an MKV. If it's writing a raw h264 stream, you'd probably have to use the timecodes when muxing too, even if you add them to the x264 command line for encoding.

After the analysis pass has run, when you open the encoding script, the timecodes file is automatically generated by TDecimate and the script output you see is the script you'll be encoding, IVTC'd, de-interlaced and decimated, with new frame numbers to match. The only difference is the frame rate, as it's being decoded at the average rate, but what you see is what's encoded. I use MeGUI's preview, but opening the script in VD should still display the frame numbers as it normally does.

If I want to add different filtering to different frame ranges, the final step for me would be to open the encoding scripts with MeGUI's AVS Cutter and use it to find the frame numbers and add the Trims to the script.
Even if I don't want to apply different filtering to different frame ranges, I still might want to exclude certain sections from filtering, so I'd add trims the same way. The combined Trims must include every frame, otherwise frames that are skipped won't be encoded when they're spliced together.

https://i.postimg.cc/c60FHkcP/AVS-Cutter.jpg (https://postimg.cc/c60FHkcP)

Once that's done, I add the script to MeGUI's job queue for encoding, but that's because I don't use other programs to process the Avisynth output first. Generally if you want to do something, there's a way to do it with Avisynth.

PS. Be careful not to open the script after the analysis pass until you've edited it to make it an encoding pass script. If you do, the metrics files are over-written and you'll have to run the analysis pass again. I still forget now and then, especially when running test encodes, and it's annoying.

[Katie Boundary]

Maybe if you didn't keep ignoring posts, by now you'd know QTGMC de-interlaces with NNEDI3 by default....

Without your fingers in your ears you'd probably know QTGMC has a lossless mode.

I've known both of those things for years. Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching, and can even re-introduce the interlacing that we're supposed to be getting rid of. I re-tested it very recently just to be sure, and the results are not much different from what plain unmodified NNEDI3 would have given me, and worse than what my own method produces.

Did you really think I wouldn't experiment with something like this at one point or another?

mpeg2source("212.d2v")

A=qtgmc(sourcematch=3,lossless=1).selecteven()
B=qtgmc(sourcematch=3,lossless=1).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

Interleave(C,D)

Well, I did. Years ago. But hey, now that it's here for everyone to see, maybe someone will take it and modify it and find a way to make it produce better results than my standard approach.

If you didn't ignore people who don't agree with you

I don't ignore people who disagree with me. I ignore arrogant jackholes who repeatedly prove that they can't be reasoned with and love talking out their asses, as you're doing now.

Of course QTGMC is deliberately "incorrect" as you put it, because it's defaults generally output a repaired de-interlaced clip, not a purely de-interlaced one.

It's deliberately incorrect in much worse ways than that.

[hello_hello]

I've known both of those things for years. Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching, and can even re-introduce the interlacing that we're supposed to be getting rid of. I re-tested it very recently just to be sure, and the results are not much different from what plain unmodified NNEDI3 would have given me, and worse than what my own method produces.

What?? So QTGMC is a bad de-interlacer now because it doesn't field match too? Does NNEDI3 or Yadif field match? Ridiculous.
Of course lossless mode can can re-introduce problems you're wanting to get rid of.

Did you really think I wouldn't experiment with something like this at one point or another?

mpeg2source("212.d2v")

A=qtgmc(sourcematch=3,lossless=1).selecteven()
B=qtgmc(sourcematch=3,lossless=1).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

Interleave(C,D)

I'm not going to spend time testing that method again, because I've linked to samples earlier in the thread where I used a modified version of your "method" with good results (and your original method with NNEDI3 de-interlacing that produced horrible results), but aside from not needing to de-interlace twice, and the field order for TFM being the wrong way around for the times I've tried your method:

DeintClip=qtgmc(sourcematch=3,lossless=1)
A=DeintClip.selecteven()
B=DeintClip.selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)

You could use pp=2 for TFM, which would cause it to take the entire frame from DeintClip when combing is detected, not just bits of it that mightn't match the rest of the frame as such and cause problems that weren't there before, and then it'd be TFM's fault if it wasn't always right, not QTGMC's, and you could use far less aggressive combing detection settings, so it's not likely to take the whole DeintClip frame unnecessarily when IVTCing film sections, or you could adjust other settings to suit the clip instead of expecting the same method to produce the best result each time, and you could keep in mind QTGMC was designed to be a quality de-interlacer of interlaced video, without the knowledge that Katie would once again re-invent the wheel.

Well, I did. Years ago. But hey, now that it's here for everyone to see, maybe someone will take it and modify it and find a way to make it produce better results than my standard approach.

If I wasn't sure you were serious, I'd probably find that funny.
It doesn't make QTGMC a bad de-interlacer simply because it doesn't always work with your method, it just means your should use something else if the result isn't good.

I don't ignore people who disagree with me. I ignore arrogant jackholes who repeatedly prove that they can't be reasoned with and love talking out their asses, as you're doing now.

Your long standing poor attitude when asking for help certainly doesn't help. How many people are on your ignore list now? Obviously not enough to make you wonder if maybe you're the problem and not everyone else, and even though nobody else has felt the need to request forum features that allow them to effectively become a moderator of their own threads so they can't be disagreed with.
When did I make your ignore list? Was it while trying to explain that interlaced video is a thing and not 60p video with half the scanlines removed, or while trying to explain how field matching works?
Back then you had a quite a mental method, and anyone disagreeing was a troll or talking out of their arse. Now, after re-inventing the wheel again, something that doesn't work well with your current method is naturally labelled bad.

It's deliberately incorrect in much worse ways than that.

For example? Don't just throw out some unsubstantiated nonsense and expect me to buy it. You're the expert there apparently, so be specific.

[hello_hello]

Go ahead and try to find a combination of settings that will deinterlace the text in the first clip...

There's no combing. What's the problem?

For the second clip, why don't you try this:
TFM(pp=1).QTGMC(FPSDivisor=2)
Or this:
TFM(pp=1).NNEDI3()
Or this:
TFM(pp=1).Yadif()

TFM would be field matching without de-interlacing, and the whole frame is then de-interlaced.
I concluded TFM wasn't missing anything when the problem didn't go away, and was therefore baked into the the fields, but maybe I'm jumping to the wrong conclusion.

You can see by looking at the "light around the letter "n" that NNEDI3, and therefore QTGMC, have dropped half the fields to create a progressive frame, whereas TFM must be using both fields to create the new frame, and it seems logical to me the result of replacing just the combed pixels with pixels from a de-interlaced clip will sometimes look different according to how the de-interlaced clip is created.
If you switch between the field matched screenshot and the QTGMC and NNEDI3 screenshots, the QTGMC version is not as close a match to the field matched frame as NNEID3, as nothing has been "warped" by NNEDI3 on it's own. It doesn't make QTGMC's processing bad, because there's far less aliasing, it just means replacing combed pixels with pixels from a QTGMC de-interlaced clip mightn't always look the best.
If you look at the shape of the very light areas in the image though, you'll see NNEDI3 on it's own distorted some of them or reduced detail, so in that respect the QTGMC version is better.
As stand-alone interlaced frames though, the QTGMC version looks by far the best to me.

Telecined frame
https://i.postimg.cc/McrLpSFm/telecinded-frame.jpg (https://postimg.cc/McrLpSFm)

TFM(PP=1) Field matched, no de-interlacing
https://i.postimg.cc/G96DR5hw/filed-matched.jpg (https://postimg.cc/G96DR5hw)

TFM(pp=5)
https://i.postimg.cc/KKYVbzJY/TFM-pp-5.jpg (https://postimg.cc/KKYVbzJY)

TFM(pp=1).NNEDI3()
https://i.postimg.cc/1Vz8Sbf6/TFM-pp-1-NNEDI3.jpg (https://postimg.cc/1Vz8Sbf6)

TFM(pp=1).QTGMC(FPSDivisor=2)
https://i.postimg.cc/WFFHDyRL/TFM-pp-1-QTGMC-FPSDivisor-2.jpg (https://postimg.cc/WFFHDyRL)

[poisondeathray]

There's no combing. What's the problem?


KTB's clip1 does have combing, you can brighten it up if you can't see it with something like levels (raise the black level or boost gamma). You can brute force it with vinverse2, it does not really harm the text .

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

(Is this Hercules in space? I never got into this one either)

[poisondeathray]

There's one other problem that I have only seen briefly mentioned, and that is the abysmal source footage quality in the early seasons of the show. Later season episodes, like Sacrifice of Angels, look pristine by comparison. I've tried various noise filters (as well as no noise filter) after performing the IVTC on the film sections and the result, when upscaled by Topaz, looks like garbage.

Here's a sample of IVTC'd source footage of the station. How can this possibly be upscaled to look okay?
sample (https://ln2.sync.com/dl/5dd47a7d0/tanjt92v-e52sztfm-t9k5qzud-2tgukgp5)


Another option - some fans actually replace space scenes with their own CG footage . Search youtube there are many examples. You can do it in blender or other 3D programs. There are quite a few free models and textures from Star Trek universe (and many other sci fi shows)

It will stick out like a sore thumb, because even the low quality models and textures will look better , so you have to dumb it down a bit if you want to match quality with other scenes , especially with live actors and soft footage

Getting it to match exactly to match is a bit difficult , because the models might be slightly different, and the lighting will be difficult to mach exactly 100% . But in terms of framing, camera path, that's entirely "do-able"

[hello_hello]

KTB's clip1 does have combing, you can brighten it up if you can't see it with something like levels (raise the black level or boost gamma). You can brute force it with vinverse2, it does not really harm the text .

I've changed the levels, looked at it sideways, run it fullscreen, stepped through frames, and I'm stuffed if I can see anything in the text that needs de-interlacing. Are we both talking about "212 clip1.demuxed.m2v", or am I blind?

https://i.postimg.cc/Mv7vQLTJ/212-clip1-demuxed-m2v.png (https://postimg.cc/Mv7vQLTJ)

[hello_hello]

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

(Is this Hercules in space? I never got into this one either)

That's one I couldn't get TIVTC to run through smoothly using VFR encoding, although I didn't mess around too much.
I "think" my video card can do it, but if I start playing the clip from the beginning it totally misses IVTCing the first couple of frames of the shot of the ship and there's a judder in the motion, and the text part goes by before my brain can lock into the motion again, but I think it's pretty smooth after the first few frames.

I don't know anything about that show. I've never watched it.

[poisondeathray]

I've changed the levels, looked at it sideways, run it fullscreen, stepped through frames, and I'm stuffed if I can see anything in the text that needs de-interlacing. Are we both talking about "212 clip1.demuxed.m2v", or am I blind?

Look at the fade (in/out) parts

It's similar to text fades in DS9 (or even scene fades)

[poisondeathray]

That's one I couldn't get TIVTC to run through smoothly using VFR encoding, although I didn't mess around too much.

The BG is 23.976, but the text overlay is 29.97 - same deal as DS9

If you use no filters at all, and step through, you can see the text is pretty clean @ 29.97, except for the 1st few frames for the text wipe in - because the underlying BD has telecine and the overlay is partially transparent at that point . If you just filter the text and composite it back over, it's not too bad at 23.976. The wipe is not as smooth as compared to 29.97, but the BG is proper 23.976. Jerky ship is more noticable than jerky text wipe

Alternatively, you can completely retime the text at 23.976, but that's quite more work (might be able to do some of it with optical flow, but you'd need to clean it up manually)

[hello_hello]

KTB's clip2 is like DS9's titles, they are overlays composited on top before removing pulldown. You can mask them out and filter separately like DS9 if you didn't want to "harm" the other parts of the picture. The top of ship has aliasing too, so you can filter that part separately with another mask

Aliasing aside, there was enough image left after using TFM's own arguments for excluding sections of the frame from the field matching to allow it to decimate the CGI section to a CFR of 23.976fps. A pain to have to treat that section differently, although if it's the same intro for every episode I guess it's not too tragic.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate()
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2.mkv (https://ufile.io/awptek3s)

[poisondeathray]

Aliasing aside, there was enough image left after using TFM's own arguments for excluding sections of the frame from the field matching to allow it to decimate the CGI section to a CFR of 23.976fps. A pain to have to treat that section differently, although if it's the same intro for every episode I guess it's not too tragic.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate()
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2.mkv (https://ufile.io/awptek3s)

The problem with this - is a sort of "strobing" effect; every few frames is "blurred" slightly when the deinterlacer is applied

For the people doing the upscaling, jumping through hoops - I would go the extra mile to get it right, especially if it's some series that supposedly deserves better treatment . Many of the intros are recycled and the same, just the guest stars might be different

[hello_hello]

Look at the fade (in/out) parts

It's similar to text fades in DS9 (or even scene fades)

If it's what I'm seeing now, I'd never have noticed, or cared if I did, but is it combing as such, because if it was you could pick the right scanlines and interpolate the rest, but doing that looks like this:

nnedi3(field=-1)

https://i.postimg.cc/sB3tfw6c/clip1.png (https://postimg.cc/sB3tfw6c)

Edit: Here's a confession. I thought Katie had uploaded the wrong screenshot. I still use a Trinitron CRT for most things that aren't video, and don't have it set very bright, so when I looked at Katie's screenshot all I saw was black. Looking it it on the LCD I can see there be text now. :)

[poisondeathray]

If it's what I'm seeing now, I'd never have noticed, or cared if I did, but is it combing as such, because if it was you could pick the right scanlines and interpolate the rest, but doing that looks like this:

nnedi3(field=-1)



You could argue that nnedi3 produces "zebra-ed" result, where there are dark/bright bands on frame 331 . If you use something like vinverse2(sstr=0.5), you won't get those bands.

Also, nnedi3 "damages" the other frames more. If you look on a non fade frame , the horizontal line of "A" in "spiral" is obscured, and text less clear overall . (Although you could apply filters selectively only to specific frames)

clip1 is much "easier" because the BG is black

[SaurusX]

This is a case of too many cooks in the kitchen.

[hello_hello]

You could argue that nnedi3 produces "zebra-ed" result, where there are dark/bright bands on frame 331 . If you use something like vinverse2(sstr=0.5), you won't get those bands.

Also, nnedi3 "damages" the other frames more. If you look on a non fade frame , the horizontal line of "A" in "spiral" is obscured, and text less clear overall . (Although you could apply filters selectively only to specific frames)

clip1 is much "easier" because the BG is black

I agree, but I was just speculating if it's technically "combing" as such.

[hello_hello]

The problem with this - is a sort of "strobing" effect; every few frames is "blurred" slightly when the deinterlacer is applied

For the people doing the upscaling, jumping through hoops - I would go the extra mile to get it right, especially if it's some series that supposedly deserves better treatment . Many of the intros are recycled and the same, just the guest stars might be different

I'd also agree with that, but if it was me, just encoding the episodes to watch myself, I'd probably give it the QTGMC treatment to even it out a bit and call it a day. :)

After-all, it's only a few seconds of an opening credit. I don't think even my OCD extends beyond this.

A = last
B = A.TFM().TDecimate()
C = A.TFM(y0=110, y1=380, PP=5).TDecimate().QTGMC(InputType=1, Preset="very slow")
# TFM().TDecimate()
B.Trim(0,12) ++ C.Trim(13,0)

212 clip2b.mkv (https://ufile.io/lhsbtybk)

[hello_hello]

By the way, how would you go about masking out the text. I assume it'd have to be done frame by frame? It's not something I've thought about doing before.

[Katie Boundary]

My latest private message from Joel indicates that he might have been duplicating interlaced frames before attempting any kind of deinterlacing. If that's the case, then I think we've just identified a huge source of his problems.

What?? So QTGMC is a bad de-interlacer now because it doesn't field match too?

Not what I said.

It doesn't make QTGMC a bad de-interlacer simply because it doesn't always work with your method, it just means your should use something else if the result isn't good.

No, what it means is that its results aren't noticeably better than NNEDI3 alone, and worse than my script... which is exactly what I already said.

Your long standing poor attitude when asking for help certainly doesn't help.

I don't have a poor attitude when asking for help. I have a poor attitude when dealing with people who can't read.

How many people are on your ignore list now?

Three.

There's no combing. What's the problem?

Oh, no wonder. It turns out that you can't even see. That explains a lot.

(Is this Hercules in space? I never got into this one either)

I don't know because I never saw Hercules but I might be able to explain it in Star Trek terms: Imagine Captain Kirk getting frozen in time for 300 years and waking up to find that his crew is dead, the Federation has disintegrated, the whole galaxy has been overrun by Khan's genetically engineered supermen and a race of giant, technologically advanced, man-eating tribbles, and it's up to him and a group of low-budget actors from Vancouver to put civilization back together.

The first two seasons were pretty good.

[hello_hello]

Not what I said.

It's exactly what you said. How do you manage to deny what's there in back and white?

Of course, the so-called "lossless" mode is nowhere near lossless, since it still doesn't bother attempting to restore the original progressive frames where they exist or perform any kind of field-matching

No, what it means is that its results aren't noticeably better than NNEDI3 alone, and worse than my script.

If you don't think the QTGMC results are noticeably better than NNEDI3 alone, there's no point discussing it any further, because it's obviously far better than NNEDI3 alone for de-interlacing an interlaced source. If it wasn't, everyone would simply de-interlace with NNEDI3.
Worse than your script?? At what, de-interlacing a purely interlaced source as it's designed to do? Comedy Gold!!

I don't have a poor attitude when asking for help. I have a poor attitude when dealing with people who can't read.

The consensus here suggests otherwise.

Oh, no wonder. It turns out that you can't even see. That explains a lot.

That's been explained, read the rest of the posts and move on.

[Katie Boundary]

It's exactly what you said.

Wrong. Thank you for once again confirming everything I've said about your lack of reading skills.

The consensus here suggests otherwise.

There is no consensus. Thank you for once again confirming everything I've said about your lack of reading skills.

[poisondeathray]

By the way, how would you go about masking out the text. I assume it'd have to be done frame by frame? It's not something I've thought about doing before.

Not frame by frame, there is keyframe "interpolation" in most editors, you just do every nth frame and it fills the rest in, and you might have to fine tune a bit

It's called "rotoscoping" or essentially moving masks . It's a basic "101" skill set for programs like after effects, blender, natron, fusion, resolve, etc... most video editors. Mocha and silhouette are dedicated rotoscoping tools, they have some additional tricks that can help .

You don't need to be very precise for this or ds9, just a rough shape. (But you can be subpixel precise if you needed to)


The top of the ship in this one, or some of the DS9 shots , you're going to use pretty heavy filter stacks with, QTGMC in progressive mode, some AA filters, you don't want to "damage" the other parts, so you use masks. Especially important for the upscaling people, those strong AA filters reduce what little detail was there in the first place