Deep Space Nine upscale project

[zapp7]

Meaning it stops being a 3:2 pattern and any given group of five consecutive frames won't necessarily have exactly three frames that should be decimated and two that should be kept.

I've scrubbed through about 18 episodes in the first season creating TFM override files for each, and while that does happen, it is very rare. When it does happen it's usually the first frame of the next scene that is problematic, and QTGMC handles it pretty well. The effect isn't even noticeable upon playback.

[Katie Boundary]

No, it happens with practically every scene change.

[zapp7]

We must not be talking about the same thing then.

[hello_hello]

Can the two of you please stop? I've already asked hello_hello via PM to just leave you alone. I have no idea why he even persists. Now I ask you to leave him alone. Just ignore each other. Is it so hard? I thought he was on your ignore list. What happened? You just relished the thought of letting have it? Me, I'm easy, but you don't want Swede to make good on his threats.

By the way, both Swede and I can see what you wrote and then thought the better of it just above.

She went back through the thread, copied an old script I posted, then proceeded to post mostly incorrect nonsense about it. Why should I simply leave that uncorrected? Sometime in the future someone might think her nonsense should be taken seriously. If Katie copied something you'd posted and then proceeded to post incorrect assumptions about it, wouldn't you reply to correct her?

What did I write? I edited that post quite a few times to change and add things.
Edit: I've looked through the edits (I forgot you could do that) but I still can't see what you're referring to.

[hello_hello]

Careful, that's really not how TFM works. TFM doesn't replace pixels. It replaces entire lines of pixels. The TFM help file is misleading IMHO. I thought it only replaced the small areas of the frame that have detected interlacing. Like it made a mask of the moving part and would leave the underlying image. I'm nearly certain that it replaces the entire line for any line that has has detected interlacing in it. So in content like this DS9 stuff it's basically replacing the entire frame even if you're using PP>=5.

It may even be replacing anything between the top most line and bottom most line that have detected interlacing in them.

If you look at the screenshots in this post (https://forum.doom9.org/showthread.php?p=1913232#post1913232) you'll see I've used a script to show which pixels are being taken from the de-interlaced clip. A simple version would be this:

DeintClip = Tweak(Bright=200)
TFM(Clip2=DeintClip)

Most of the screenshots just show it taking specific pixels, but one of them shows it's replacing the whole frame, somewhat inexplicably to my eye, but the screenshot immediately below that one shows that with slightly different settings, it's only taking specific pixels. I think it may relate to setting the MI argument too low, although I haven't tested that, but the screenshots show it must be replacing just the combed pixels, and probably a small area around them, most of the time.

[hello_hello]

What do you mean by screwing up the cadence?

It means that normally for telecined content you can step through it and see a pattern of three clean frames and two combed frames repeating, but during the fades from one shot to the next, they've been overlaid without ensuring the pattern isn't interrupted. They're both still telecined with a pattern of 3 clean and 2 combed frames, but the clean and combed frames don't line up for each shot during the fades.

This is the sort of fade I'm referring to. I've made it brighter so it's easier to see. For the first screenshot, the shot that's fading in doesn't show telecine combing while the shot that's fading out does. A few frames later on it's the opposite.

https://i.postimg.cc/VS1VSQKq/A.png (https://postimg.cc/VS1VSQKq)

https://i.postimg.cc/nCy5JQ7w/B.png (https://postimg.cc/nCy5JQ7w)

Keep in mind though, DVDs were originally intended to be displayed on an interlaced display (CRT), so this sort of thing would display perfectly fine. For a progressive display, and when converting to progressive for encoding, it's a problem. If you simply keep decimating one in five frames for 23.976, one of the shots will probably have "jerky" movement during the fades.
One solution is to de-interlace those sections to remove the combing, and to keep them at 29.97fps. That way they'll look smooth, but the end result would be little sections of 29.97fps progressive amongst the 23.976 detelecined progressive frames, and you have to decide how to deal with it.

A player performing detelecine and deinterlacing on the DVD would probably treat those fades as video (29.97fps) and de-interlace them, effectively outputting a variable frame rate, which would look quite smooth.

[Stereodude]

If you look at the screenshots in this post (https://forum.doom9.org/showthread.php?p=1913232#post1913232) you'll see I've used a script to show which pixels are being taken from the de-interlaced clip. A simple version would be this:

DeintClip = Tweak(Bright=200)
TFM(Clip2=DeintClip)

Most of the screenshots just show it taking specific pixels, but one of them shows it's replacing the whole frame, somewhat inexplicably to my eye, but the screenshot immediately below that one shows that with slightly different settings, it's only taking specific pixels. I think it may relate to setting the MI argument too low, although I haven't tested that, but the screenshots show it must be replacing just the combed pixels, and probably a small area around them, most of the time.
Interesting... I saw a number of jello like rolling shutter sort of artifacts that were caused by the deinterlacing of CG portions that were subsequently avoided with flags=5. It looked to be replacing some entire lines. Specifically in S06E01 at around 105 seconds in the Defiant as it flies toward the camera.

[scharfis_brain]

@hello_hello:

from a theoretical standpoint it sould be perfectly possible to retrieve a non-jerky 24p decimation out of two 3:2 cadences blended together.
For sure field matching will be broken. But the decimation of a 50:50 blend of two cadences should be (manually) possible.

[hello_hello]

@hello_hello:

from a theoretical standpoint it sould be perfectly possible to retrieve a non-jerky 24p decimation out of two 3:2 cadences blended together.
For sure field matching will be broken. But the decimation of a 50:50 blend of two cadences should be (manually) possible.

I've got to say I'm extremely disappointed at being forced to evaluate my entire view of everything, but you're right.
There's been so much debate about how to handle this source that I've obviously became guilty of over-thinking it, so I went back to basics.

It appears TFM can sail through all the broken cadences on it own, except one. TFM().TDecimate repeats a frame, and I couldn't convince it not to, so I removed it manually.
As it's just the opening sequence, I waited until a fade to black to repeat a frame to make up for it.

As it turns out, disabling MicMatching doesn't even make a difference for the opening sequence as I'd come to believe it would. It does however, help with the progressive section I tacked onto the end of the sample for testing, which by the way is the only thing preventing me from wanting to stick my head in a hole somewhere now, because the existence of at least one progressive 29.97fps section still supports an argument for VFR encoding. :)

I'll upload a sample in a little while. There's still one section I'm not happy with at 23.976fps, so I'm working on it now.

[hello_hello]

Here's the new encode of the opening sequence.

Clip "A" does normal field matching for the majority of the CGI and all the film sections.
Clip "B" is me being pedantic about making the movement of the stars smoother after the wormhole (behind the text). It's not smooth even if you bob it, so that's probably as good as it gets.

The first lot of Trims combine the two clips, deleting one repeated frame during a transition in the CGI, and repeating a frame during a fade to black to compensate.
The second lot of Trims are just used for optional filtering on the decimated video.

I couldn't find a happy that allowed TFM to de-interlace the text as it fades in/out without unnecessarily de-interlacing half the picture, so I left it, as I think it's the lesser of two evils. Personally, I don't care about it that much. Whether it's encoded as VFR or 23.976, or 59.94, I'm now of the opinion fussing over making sure TFM caught all the combing was heading in the wrong direction. Based on the following test, TFM isn't de-interlacing a single pixel in the whole sample. At least none that I saw.

DeintClip = Tweak(Bright=200)
TFM(PP=5, MicMatching=0, Clip2=DeintClip)

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")

Crop(8,0,-8,0)

A = TFM(pp=5, MicMatching=0).TDecimate()
B = TFM(pp=5, y0=180, y1=300, MicMatching=0).TDecimate()

A.Trim(0,2118) + \
A.Trim(2120,2643) + \
B.Trim(2644,2799) + \
A.Trim(2799,0)

Smooth = \
"""QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)"""

Trim(0, 58).MCDegrainSharp() + \
Trim(59, 578) + \
Trim(579, 2971).Eval(Smooth) + \
Trim(2972,0).MCDegrainSharp()

Resize8(640, 480)

VTS_02_1_sample 23.976.mkv (https://www.sendspace.com/file/qf83vh)

[Katie Boundary]

I just realized that, since Joel is putting a GUIDE together and not distributing full video files, there's room for several different AVIsynth scripts and several different target frame rates, with notes on why people might want to select - or not select - each one.

I'd also love to see some examples of what happens when an interlaced frame gets put through AI upscaling.


Oh and I just noticed this article:

If you don’t want to run content at 29.97fps, there’s another option — re-encode it at 23.976fps. Convert the interlaced frames, lower the frame rate, and you’ll end up with imperfect, highly noticeable jerks and jumps during fast-paced space combat scenes — which are exactly the ones we are trying to preserve. This is known as judder. Judder sucks.

WHOA WHOA WHOA, no. That's NOT what judder is. Judder is when p24 content is displayed on a 60 hz device, or when telecined film content is bobbed or field-matched to p60, resulting in half of the frames lasting 1/20th of a second and the other half lasting 1/30th of a second, alternating, instead of each one lasting 1/24th of a second. It's not noticeable to humans, although some people claim it is. When you take higher-framerate (p30, i60) content and decimate it down to 24, that's called a "war crime".

[Asmodian]

While I do agree with most of your post I take exception to this:
It's not noticeable to humans, although some people claim it is.

I suspect you expected this feedback. ;)

All the things you care about yet you don't believe this is a problem? Humans cannot even see it? :p

[Katie Boundary]

If you're on drugs that have significantly altered your perception of time in such a way that the world around you seems to be moving in slow motion, like Fry in that episode of Futurama where he drinks 100 cups of coffee in a single day, then you might notice it.

[JoelHruska]

Per request, here are clips of specific scenes from Sacrifice of Angels from a VOB of the DVD. Clip explanations below. Clips are cut to 10 seconds or less per Manono's advice, unless it made no sense to split the scene at that point. Contiguous versions of two longer scenes (FFE = First Fleet Engagement, SFE = Second Fleet Engagement) are available below. I tried to start each clip with a few frames of padding on each side.

In most of the clips below, there are specific places I'm hoping to achieve better antialiasing results. I used QTGMC partly because it fixed some of the problems I'm going to describe below and would like to duplicate this (or improve upon the initial repair). I'm going to describe some specific problem spots I saw in the 23.976 version of this footage because it's all I have for reference and because I was attempting to repair very specific things in certain cases.

Apparently, I cannot offer a direct download link via SendSpace unless I have a premium account. A link to 13 of the 14 uploaded clips is here:

https://fs01u.sendspace.com/upload?SPEED_LIMIT=0&MAX_FILE_SIZE=314572800&UPLOAD_IDENTIFIER=1679361406.1590287955.42420C06.20.0&DESTINATION_DIR=5

You do not need to download "FirstFleeetEngagement-Complete" if you download "FirstFleetEngagement-Short1" through "Short3." The complete version of "SecondFleetEngagement" is only available on OneDrive below due to size restrictions. Files are abbreviated as "FFE" and "SFE" on OneDrive after I realized the file names were too long for easy display. The 14th file is "SFE-Complete" and it does contain a few scenes on the Defiant that are not embedded in SFE-1 -- SFE-5.

Alternately, if you want individual file downloads, you can use the OneDrive links below:


A link to the entire directory is here.

https://1drv.ms/u/s!AphTLFRW13WMjk7io5jlCJzdRO9I?e=3NcPQ3

Individual file links in-line below. Each individual file link will open a OneDrive link. OneDrive does *not* require you to have a login, and you can download by hitting the button in the upper-left-hand corner.

FirstDefiantFlyby:

https://1drv.ms/v/s!AphTLFRW13WMjlmk6jeSNMAA7rPM?e=UFmhtT

There's aliasing on both the Excelsior-class ships (inside the nacelles) and the Defiant's angle of attack in this scene is hard to fix. QTGMC against 23.976 fps did a very good job of resolving a troublesome flicker on the Defiant's left "shoulder" that gave me a lot of trouble, but I never had a great solution to the in-nacelle aliasing. Would like to nuke that from orbit.

FirstDominionFleet:

https://1drv.ms/v/s!AphTLFRW13WMjlKdB-r9XOdfALn3?e=RgO5tL

At the end of this clip there are three Jem'Hadar scarab fighters visible. All three have strong horizontal aliasing across their bottoms. I'd like to eliminate it as part of properly processing the scene. QTGMC using my repair script against a 23.976 MKV does a very good job of eliminating the horizontal lines.

SecondDefiantFlyby:

https://1drv.ms/v/s!AphTLFRW13WMjleXZXPXV9u0nKcK?e=mzSZ13

This is one where I've had trouble with some scripts causing backwards movement in the small fighters that fly across the screen. The background ships are basically fine no matter what you run against them.

First Fleet Engagement (FFE) 1-3

https://1drv.ms/v/s!AphTLFRW13WMjlTGczLl0T3HFrqU?e=mDKlE4

https://1drv.ms/v/s!AphTLFRW13WMjlHg_lI-zv5l2aGB?e=dfREGR

https://1drv.ms/v/s!AphTLFRW13WMjlYtPRhYSRJlENGV?e=yWZGR5

This scene hasn't been problematic, per se, but it's definitely one of the major CGI battle scenes of the episode, so it's included for those reasons. If you want to download the entire contiguous clip, you can do so here (100MB file).

https://1drv.ms/v/s!AphTLFRW13WMjlpFWKjMN_MpsSD_?e=gmrWr5

Second Fleet Engagement (SFE) 1- 5

https://1drv.ms/v/s!AphTLFRW13WMjljS5j6zP-Zek-we?e=nswJKR

https://1drv.ms/v/s!AphTLFRW13WMjlWMWPE-XeSI3x6M?e=DjUO8I

https://1drv.ms/v/s!AphTLFRW13WMjlMeGQWYCTBv6E3y?e=mhfNsh

https://1drv.ms/v/s!AphTLFRW13WMjlA2IsY_v3iH_ocz?e=YWsefX

https://1drv.ms/v/s!AphTLFRW13WMjk8D0hg-WBq98s-w?e=3ehQve

The second full fleet engagement. Major things I'm hoping to fix include nacelle-aliasing and the very aliased lines across the bottom of the Miranda-class vessel's ventral saucer. (SFE-1). The other scenes of SFE have been pretty solid, but are included here for completeness' sake. The SFE clips 1-5 omit the Defiant scenes that take place in-between the CGI shots. The full file, available here:

SendSpace is limited to 300MB, so if you want the full file, OneDrive is the only place to go.

https://1drv.ms/v/s!AphTLFRW13WMjlvPfszuhSSw7gN7?e=qOAQeq


Includes the Defiant interim scenes as well as the CGI scenes.

Trivia: According to the VFX team that built the show, this was the most-detailed fleet battle they ever shot. They brought in tactical experts specifically to help them design the fight. SFE is the arguable high-water mark for DS9's entire seven-year run, so if there was one CGI scene to absolutely nail, this one would be it.

Sisko-On-Defiant

Finally, I've uploaded a clip that I thought might be useful for seeing how DS9 mixes and matches VFR content together. All frame rate references measured using MPC-HC. The frame rate when the DVD is played back begins at 29.97 fps, but starts dropping immediately. By the time the shot switches to Defiant, the frame rate is 23.976 fps again -- but it starts increasing back to 29.97 fps when the Ferengi Nog appears on-screen. The clip stays at 29.97 (at least, MPC-HC reports 29.97 for several minutes thereafter).

https://1drv.ms/v/s!AphTLFRW13WMjl_bY-hbpvlTrpuU?e=5ETmw2

I have thought that this could be somehow related to the credits, but there's a later shot on DS9 itself that stays at 29.97 fps well after the credits are finished. I can clip that as well if people want to see it, but this seemed good for now.

[Katie Boundary]

The solution you your aliasing problems is to use a script that reconstructs the original progressive frames and then leaves them the goddamn hell alone.

BTW, I think you say "antialiasing" a lot when you mean "aliasing". Aliasing is bad. Antialiasing is good :)

[JoelHruska]

Fixed.

[JoelHruska]

When you take higher-framerate (p30, i60) content and decimate it down to 24, that's called a "war crime".

While it's obvious that you are far from the only person to have this opinion, I would prefer a better phrase when discussing 23.976 fps playback of p30/i60 fps footage than "war crime." Is there a specific term?

[Katie Boundary]

Kay, looks good :)

I've been running more tests and while I still think that both the interpolation and blend variations of my script gives better results than QTGMC in the vast majority of frames, there are still a few interlaced frames where even a straight blending script with no interlacing-detection (treats all frames and pixels as interlaced), no resharpening, and no other fancy stuff...

A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0)
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0)
interleave(A,B)

...STILL doesn't fully deinterlace:

https://i.imgur.com/BjSDFuh.png

...whereas QTGMC does:

https://i.imgur.com/d4u3JD2.png

Although this doesn't justify switching over to straight QTGMC, it does make me think that the best script for DS9 specifically might be a variation of my method that incorporates QTGMC. And because "lossless" QTGMC preserves the aberrant field order of the title at the beginning of the opening credits, but "normal" QTGMC smooths it out, this hybrid method would need to incorporate the "lossy" version of QTGMC. So, when you get a chance, try...

A=qtgmc().selecteven()
B=qtgmc().selectodd()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="601.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="601.d2v",flags=1,micmatching=0)

interleave(C,D)

A few fields around each scene change will get QTGMCed, but most of the frames in each shot will be perfectly restored and then left alone. For the opening credits, the comet's tail will be preserved perfectly because that part is encoded on the DVD as 24 fps progressive, but the rest of the opening credits will be 100% QTGMCed.

While it's obvious that you are far from the only person to have this opinion, I would prefer a better phrase when discussing 23.976 fps playback of p30/i60 fps footage than "war crime." Is there a specific term?

"looking like unmitigated shit" is another good one.

[hello_hello]

Per request, here are clips of specific scenes from Sacrifice of Angels from a VOB of the DVD. Clip explanations below. Clips are cut to 10 seconds or less per Manono's advice, unless it made no sense to split the scene at that point. Contiguous versions of two longer scenes (FFE = First Fleet Engagement, SFE = Second Fleet Engagement) are available below. I tried to start each clip with a few frames of padding on each side.

Are there any samples of the original video amongst that lot, or is it all re-encoded?
I gather it's lossless but it's encoded as progressive rather than interlaced, I don't know if that makes a difference when it's lossless, in respect to the chroma being upsampled properly. Not to worry though...

I only gave two of them a spin. Pure telecine as far as I can see.
You probably should take Katie's advice and use a method that reconstructs the original progressive frames and then leaves them the hell alone. Or are you too far down the rabbit hole with the 59.94fps idea?

The only reason I went the VFR route is because someone linked to a sample of 29.97fps progressive CGI, and that's the way I'd deal with mixed content, but if it's all telecined except for the odd episode, you can handle those differently when you find them. I gave the CGI the QTGMC treatment, and the film section the MCDegrainSharp treatment, but that was after the frames were reconstructed. If you come across a problem in an episode, cross that bridge when it happens.

zapp7 seem to be happy with detelecining most of it.

I linked to a CFR sample of the opening CGI section earlier. If it's the biggest problem in most episodes, you'll probably get efficient at creating scripts that'll handle it correctly if you want it to be perfect (the latest encode looks good to me at 23.976fps), but the remainder of each episode might mostly require nothing other than field matching and decimation.

samples.zip (https://www.sendspace.com/file/50bsai)

Edit: I forgot to resize the SFE-1 encode, it's just cropped.

Katie,
If the script you posted is any indication, you're fussing about combing while using the wrong field order for TFM again. At least for the sample you took the screenshots from.

[hello_hello]

Damn, I just noticed an oddity in the film section towards the end of the Sisko-on-Defiant sample. It's possibly a field order thing that wouldn't have happened working with the original video, but here's the fixed version.

Sisko-on-Defiant take 2.mkv (https://www.sendspace.com/file/udnpv7)

LWLibavVideoSource("D:\Sisko-on-Defiant.mkv.lwi")
AssumeTFF()
Crop(8,0,-8,0)

TFM(pp=5, micmatching=0).TDecimate()

Trim(0,168).QTGMC(InputType=1, Preset="slower") + \
Trim(169,0).MCDegrainSharp()
Spline36Resize(640,480)

I'm not sure about MCDegrainSharp() for the film sections in every episode. That one seems a bit noisy and it's leaving too much behind for my taste, but anyway....

[hello_hello]

I can't image you'll do much better than smoothing out the baked-in aliasing with QTCMC.

The noise in the film sections is hard to deal with. It looks like the noise removal used previously stabilised what it didn't remove, so it doesn't behave like noise. I think QTGMC would clean it up okay, but it might soften the picture too much. It might be better to just leave it.

Samples 2.zip (https://www.sendspace.com/file/gwxxi0) (7.4 MB)

[wonkey_monkey]

when telecined film content is bobbed or field-matched to p60, resulting in half of the frames lasting 1/20th of a second and the other half lasting 1/30th of a second, alternating, instead of each one lasting 1/24th of a second. It's not noticeable to humans, although some people claim it is.

It absolutely is noticeable, it just doesn't necessarily jump out at you as a problem (especially to people who grew in NTSC-land) until you view both versions together.

[StainlessS]

The noise in the film sections is hard to deal with. It looks like the noise removal used previously stabilised what it didn't remove, so it doesn't behave like noise.
That sounds like you mean the noise is same across frames ie temporally same, maybe somebody added fixed pattern grain. [bad idea]

[Katie Boundary]

I just downloaded and played with Topaz for the first time and I have two things to say about it:

1) It only allows scaling by a single factor, applied to both width and height. It cannot change the aspect ratio. If you want to upscale a fullscreen show (like DS9 or Buffy) from 720x480 to 1600x1200, you're screwed. If you want to upscale a widescreen show (like Andromeda) from 720x480 to 1920x1080, you're screwed. You have to use a conventional resampler like Lanczos at some point to correct the aspect ratio. Or perhaps use a different AI upscaler.

2) Contrary to my expectations, interlaced text, when upscaled with AI, looks absolutely bitchin'. I wish there was a way to make all the text in the show look like this whether it's fading in/out or not. Here's a sample of upscaled interlaced text: https://i.imgur.com/cH4788X.png

[JoelHruska]

Are there any samples of the original video amongst that lot, or is it all re-encoded?

Every sample was created using the following method.

1). Rip episode with DVD Decrypter
2). Read D2V file into StaxRip to create an MP4 with no other scripting applied. Said MP4 created using quality 12.0, Preset Slow.
3). Clips created in VirtualDub2.

I should have loaded the D2V file directly into VirtualDub2. Truthfully forgot you could do that -- I was pretty tired last night after spending most of the day packing and schlepping. If you guys want to see what that output looks like, I can do that.

[JoelHruska]

KB,

TVEAI is in pretty rough shape at the moment. The audio pass-through is so bad, you're better off assuming the audio will not work and just planning to do your own remuxing. It encodes in the most basic MP4 profile settings imaginable and there is zero control over output quality in any regard.

You can only do lossless upscaling if you output to PNG or TIFF and I've seen some people say that some content doesn't look as good when pulled apart into individual frames and then stitched back together, though I have not tested that myself. I don't know which content is supposedly affected or what the differences are.

Also, the storage requirements get pretty ugly. It's 4-5MB per frame per PNG and about 16MB per 16-bit TIFF, so you need between 342GB - 1.216TB per episode for upscaling output, and you'd want an SSD for certain over an HDD for rapid file writes (when upscaling) and faster reads (when reassembling). At these data sizes and rates, the gap would be material. I think 16-bit TIFFs may be overkill when upscaling DVD content.

Emissary, Way of the Warrior, and What You Leave Behind would require double this storage requirement if you wanted to do both episodes in a single gulp. So ~644GB - 2.4TB to store the output frames from the entire episode.

I actually liked the 1.1.0 version more in terms of output quality. There's a way to get the older versions if you were interested in comparing the outputs. They are meaningfully different.

[Katie Boundary]

I hadn't even played with video modes. I was just using their single-image upscaler. For video, I would have certainly wanted to test output options to see if exporting XviD AVI was possible. What you're saying is that you have to export from Topaz as a series of PNG files, and then import that series of images into a program like Vdub to turn them back into a video and export with the settings of your choice?

[SaurusX]

Katie, you’re in luck. If you want video output from Topaz then you should know that all it exports is xvid. In order to use a modern codec you have to take the sequence of images route.

[hello_hello]

Every sample was created using the following method.

1). Rip episode with DVD Decrypter
2). Read D2V file into StaxRip to create an MP4 with no other scripting applied. Said MP4 created using quality 12.0, Preset Slow.
3). Clips created in VirtualDub2.

You can split the vob files themselves.
DGindex will output an mpeg video stream, MKVToolNix will split and remux as MKV, TSMuxer with open them and split and remux as TS files.

DGIndex is probably easier as it has a preview, but for samples it's better to work with the source. Not that it matters to me as such, I've played around and re-encoded a few of the h264 samples and I know how I'd handle them.

Did you look at the last few re-encodes I uploaded? I'm just wondering what you thought. I'm not claiming extreme cleverness of anything. I just decimated to 23.976 and ran QTGMC over them. I was just wondering how they compared to whatever method you're using at the moment, that's all.

[JoelHruska]

Did you look at the last few re-encodes I uploaded?

Haven't checked any yet. Wanted to get the clips posted last night once I understood the DVD source was what people wanted to see. I will look tonight. So far today, the garage has been demanding my attention. ;) (I move on Thurs).

[Katie Boundary]

Katie, you’re in luck. If you want video output from Topaz then you should know that all it exports is xvid. In order to use a modern codec you have to take the sequence of images route.

But it does so in MP4, not AVI, and it offers no control over bitrate or number of passes or anything like that?

[JoelHruska]

All,

Updated set of shared files. These clips were exported via DGIndex directly from the DVD VOB.

All of the clip names are identical to what I used in the previous series of posts, except for the addition of ".demuxed" to the file names.

No need for OneDrive this time, and I can't post individual links because I'm not a SendSpace premium member, so here's the link to all 14:

https://www.sendspace.com/filegroup/Y%2B9HP%2BpjhYvgld3iuIz%2F5KHAfjHMjlgefsUUt%2B%2FkHq%2FvD5HuwCXmYhV4VwOIVnnw1%2Fg4Q59rK5s5tWAfQJIYIBN2Vfhgu7iIZU%2FC3z%2F9ympIrpc2%2FSo2GQ

The link below is for the original file descriptions:

https://forum.doom9.org/showthread.php?p=1913411#post1913411

[videoh]

These clips were exported via DGIndex directly from the DVD VOB.Now you're cooking with gas. ;)

[JoelHruska]

But it does so in MP4, not AVI, and it offers no control over bitrate or number of passes or anything like that?

You get four options:
*.MP4
*.PNG
*.TIFF (8-bit)
*.TIFF (16-bit)

You can change the file name and you can choose the upscale model. You can choose to enable audio pass-through and you can choose the multiplier or specific output resolution (within certain restrictions, as you've noted). You can also choose to only upscale part of a video rather than the whole thing.

For now, that's literally everything.

I cannot speak to how Artemis-LQ and Artemis-HQ will treat your own source, but my experience with both suggested they are semi-functional atm as far as DS9 is concerned.

https://www.extremetech.com/wp-content/uploads/2020/05/ImproperAIScaling.jpg

That's one of the Artemis models run against a generic upscaled DS9 MKV and against my own work on the right. Pretty sure HQ, in this case.

https://www.extremetech.com/wp-content/uploads/2020/05/Artemis-LQ-Upscaler.jpg

I know this one is Artemis-LQ, specifically. Again, MKV on left, my own output on right.

I'd be very curious to know if you get different results. Most people are sticking with either Gaia-HQ or Gaia-CG.

Also, be advised, Topaz *does* introduce color-shifts on some content. I don't know which or why, but Weyoun's jacket is often color-shifted towards a darker brown.

[zapp7]

I can't image you'll do much better than smoothing out the baked-in aliasing with QTCMC.

The noise in the film sections is hard to deal with. It looks like the noise removal used previously stabilised what it didn't remove, so it doesn't behave like noise. I think QTGMC would clean it up okay, but it might soften the picture too much. It might be better to just leave it.

Samples 2.zip (https://www.sendspace.com/file/gwxxi0) (7.4 MB)

I can confirm that the noise is difficult to deal with. I tried a bunch of different filters - mcdegrainsharp, temporaldenoise2, QTGMC, TNLMeans - and while the intro looked great, the film sections had too soft a picture for my tastes. The more filters I try, the more I prefer just field matching and decimating and leaving the progressive frames alone. I think the higher picture quality outweighs the benefits of denoising, in this case. Viewing distance is a factor too. The noise is less noticeable when I'm sitting on the couch watching on my 4K TV compared to a couple of feet from my computer monitor.

[JoelHruska]

H_H,

I've looked at SFE-1 on both Samples and Samples 2. I'll check Sisko in the AM.

They're both gorgeous, but I don't see all that much difference between them. When I watched frame-by-frame, I saw the ripple pattern move somewhat differently on the Miranda-class ship's hull when it fires, but I didn't see much else in terms of difference. Where should I be looking?

[hello_hello]

Just the resizing. I cropped it but didn't resize to the correct aspect ratio the first time, so I included it again with the second samples.

Looking at the script, for the first one I ran QTGMC over it like this:
QTGMC(InputType=1, Preset="slower")
and for the second one like this:
QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)

That may have changed the ripple pattern a tad. The second one should help retain any motion blur, but it doesn't make a huge difference for the CGI.
Or the ripple pattern may just look a bit different due to the resizing.

[hello_hello]

That sounds like you mean the noise is same across frames ie temporally same, maybe somebody added fixed pattern grain. [bad idea]

I think it's like the effect you can sometimes get denoising with QTGMC. Because it stabilises the picture, and therefore any remaining noise, if it leaves too much behind and you use a slow speed preset, the effect can be a little like watching video through a fly-screen. Sometimes if I'm leaving noise behind I use a faster speed preset so as not to stabilise it too much.
It doesn't look that bad for the DS9 samples I've seen though, but against a flat background it's fairly static.

I'd be interested to know what was used to filter the noise originally, and/or just how noisy it was. Aside from the static noise left behind, there's a bit of blurring during motion on occasion. That's probably why they didn't remove any more than they did.

[JoelHruska]

Hello_Hello,

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched. But they're both absolutely gorgeous.

Also, the lighter QTGMC approach still cleans up the aliasing on the lower hull very nicely.

[JoelHruska]

If anyone is curious to see what Hello_Hello's samples look like after being run through the two functional Topaz AI upscaler models:

https://www.sendspace.com/filegroup/hAmG8UuXAfl3T2VT6e8pntkJJg0KsgtR

I'll test the other two again (Artemis) on the properly deinterlaced clips to see if it functions now, but I'll be a little surprised if they do.

[Katie Boundary]

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched.

Well, that's because the faces ARE stretched, and so is everything else. These images are meant to be displayed on a device with non-square pixels. You need to resize to 640x480 (or some other resolution with a 4:3 aspect ratio) to get everything back to the right shape.

[hello_hello]

Pardon my stupidness, but what's the difference between CG and HQ?

I'm only looking at them on a 1808p display, but I ran the upscaled versions at full size (I only see 1080p worth of picture) then upscaled the DVD encode by roughly the same amount with MPC-HC. The CG versions are an improvement for the CGI. It looks better than I expected, because I was half expecting the upscaling to do things to it I'd hate. I wouldn't mind seeing a live action comparison. The HQ versions are over-sharpened for my taste.

1080p screesnshots.zip (https://www.sendspace.com/file/kys70d)

[hello_hello]

Hello_Hello,

I prefer the 640x480 crop to the 704, less for the CGI and more because it tends to make faces look a little stretched. But they're both absolutely gorgeous.

Also, the lighter QTGMC approach still cleans up the aliasing on the lower hull very nicely.

Technically you're better off cropping the DVD to 704x480 (which is an exact 4:3 display aspect ratio) and upscaling directly to 4:3 dimensions, as that way you might squeeze a tiny bit more detail out of it than you get resizing to 640x480 first. Can you specify the upscaled width and height individually?

[JoelHruska]

Hello_Hello,

Those are the names of the new models they use. I've done a fair bit of comparison between them and I've run full encodes of both on episodes.

HQ typically retains more noise than CG if used on scenes with people in them. CG has stronger default noise filtering and deblocking applied. Gaia-CG is typically criticized for being overly aggressive on noise reduction outside of CG scenes, and for applying a bit too much smoothing. Gaia-HQ is typically criticized for retaining too much noise.

I generally prefer Gaia-CG over HQ for even the character scenes, but the models are still under active training and there's going to be a big update dropping in the next few weeks. Neither is perfect.

[zapp7]

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

[JoelHruska]

I tried Gaia-CG on one of the first season episodes and it produced artifacts, kind of like lines or streaks. Is there something I'm missing?

Which episode and where? I'll see if I can duplicate.

[zapp7]

Which episode and where? I'll see if I can duplicate.

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

[JoelHruska]

S01E04 - A Man Alone. You can see them in the big CGI ball in the first scene of the episode.

I'll rip it and take a look.

[SaurusX]

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

[JoelHruska]

Gaia-CG is ugly as hell. Does way to much scrubbing away of detail.

The impact of Gaia-CG on CG scenes in DS9 is minimal compared to the uplift. It's more noticeable on non-CG content where, yes, there is some visible smoothing out of detail. I ran the denoise / deblock 100% render option on Gaia-CG multiple times on the same episode, just to see what would happen. In the "Sacrifice of Angels" recap of the previous episode, Sisko starts the clip wearing a quilted uniform top that has vertical lines on it -- seams, stitched into the fabric. You can see them clearly.

After a 2x runthrough Gaia-CG, they are noticeably less visible when he changes the angle of his stance. At three passes, they disappear altogether when he's standing at certain angles. Hopefully all of this will be tuneable one day.