Deep Space Nine upscale project [Archive] - Page 4

hello_hello

14th May 2020, 13:27

I don't know what I was thinking. A milder QTGMC repair over the whole CGI section is worth a tiny bit of lost detail, in my opinion.

A new sample, only requiring one set of Trims as the QTGMC repair is included in the QIVTC function. The QTGMC repair "may" look smoother due to the additional motion blur arguments.

function QIVTC(clip Video)
{
DeintClip = Video.TFM(Mode=6, PP=1, MI=20, Slow=2, CThresh=4).QTGMC().SelectEven()
Video.TFM(Mode=6, Clip2=DeintClip, MI=20, Slow=2, CThresh=4).TDecimate()
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

function IVTC_Denoise(clip Video)
{
Video.TFM().TDecimate().MCDegrainSharp()
}

DGDecode_mpeg2source("D:\VTS_02_1_sample.d2v")
Crop(8,0,-8,0)

Trim(0,65).IVTC_Denoise() ++ \
Trim(66,735).TFM().TDecimate() ++ \
Trim(736,3714).QIVTC() ++ \
Trim(3715,0).IVTC_Denoise()

Resize8(640,480)
GradFun3()

As an experiment, I modified the Katie method again to see how it'd compare if I gave it better de-interlaced clips to use. It's slow, as it has to de-interlace twice with QTGMC before repairing with QTGMC, whereas the more normal IVTC method only de-interlaces once, but where the CGI fades from one shot to another and the field order changes there are differences. I think the more standard IVTC generally looks a bit smoother through those transitions, but I've watched them so many times I can barely remember my name, so I've uploaded both encodes in case anyone else wants to look. The sort of transition I'm referring to is the fade from one shot to the next, a little bit after frame number 1800. The transition just after frame 2300 looks smoother for the Katie Mod encode, but I'm not sure that's always the case.

This is what I used as a modified version of the Katie method for IVTC in the CGI sections.

function QIVTC(clip Video)
{
A = Video.Tfm(field=0,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=1,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(field=0,mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

Both encodes can be download here (https://ufile.io/f/til02).

JoelHruska

14th May 2020, 14:15

Also, if above guesswork was correct, then maybe be a little more clear so as to avoid the guessin' bit.

I do not know how I could possibly have been more clear than "I removed the call to that filter." Previously, there was a call to that filter. I removed it.

Specifically, I changed the following line of code:

F = Clip.TFM().TDecimate().MCDegrainSharp()

to the following:

F=Clip.TFM().TDecimate()

This produces the video splice error.

I will quote my code changes in the future to avoid the ambiguity.

JoelHruska

14th May 2020, 14:56

Hello Hello,

Sorry for the ambiguity. References to Handbrake and MakeMKV in the same sentence should be read as a reference to using Handbrake or MakeMKV-derived source. I apologize for being unclear and will change how i refer to this in the future.

The reason I referenced both applications is because I was having the weird problem with MakeMKV playback speed in the first place. I wanted to make it clear that I tested both a CFR and a VFR ingestion of the file, and got the same "incorrect speed" error on both, in order to illustrate that the problems were unconnected.

JoelHruska

14th May 2020, 16:54

hello_hello

14th May 2020, 17:00

This produces the video splice error.

MCDegrainSharp doesn't touch the frame rate.

I have a theory....
I assume you removed MCDegrainSharp because of a "no such function" error. Once that error was removed you worked your way up the error chain.

The following line was used because in mode 7, TDecimate can't always decimate to the exact frame rate. It's generally so close it doesn't matter, but Avisynth needs exact, so AssumeFPS adjusts it to the correct frame rate.

TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)

This line will give you exactly the right frame rate (23.976), if the source frame rate is exactly right (29.97).

F = Clip.TFM().TDecimate()

If you opened the source with ffms2 without honouring the repeat flags, the source frame rate would be different, the "F" line above wouldn't have a frame rate of 23.976fps, instead it'd be relative to the source frame rate, while the earlier line would still be 23.976fps, because AssumeFPS told Avisynth to speed it up or slow it down to 23.976fps, regardless of the actual frame rate, so removing MCDegrainSharp fixed the "no function" error, only to expose the frame rate mismatch when splicing.

That's my best guess, after letting the last bit of info rattle around in my head for a while, but only because the additional details made me realise you were talking about a different script than the one I (and StainlessS) thought you were referring to.

FFMS2/FFVideoSource needs it's RFFMode argument enabled to honour repeat flags. That way it should decode the video ripped with MakeMKV at 29.97fps. ie

FFVideoSource("E:\VTS_02_1.mkv", RFFMode=1)

For LSmash, I think the latest version has it's repeat option enabled by default, but if not, or for older versions, it's Repeat=true instead.

DGIndex honours repeat flags, which is one of the reasons it's the recommended method for mpeg2 video, but it can't open MKVs. If you remux the video from the MKV as a ts file with TSMuxer as I suggested, StaxRip will probably open the ts file with DGIndex.

I'm still not clear on the Handbrake part. As far as I know it's CFR/VFR choice only effects the output. It should handle the input from VFR a MKV correctly. If you have the correct filters enabled (the Decomb filter I think, I don't use Handbrake) it should output a VFR if you tell it to.
In fact for Handbrake it's probably the best choice. If the input is VFR (combinations of interlaced/hard telecine and soft telecine) and the filtering is enabled, it'll decimate and de-interlace as normal, but if you tell it to output a constant frame rate of 29.97fps (for example), the interlaced parts will be okay, but the film parts will have lots of duplicate frames to increase the frame rate to 29.97fps, and that generally looks terrible because motion gets quite choppy. For hybrid sources it's generally better to let it output a VFR. If you want to use the output as the input for another program though....

hello_hello

14th May 2020, 17:23

Attempting to use MPEG2Source instead of FFVideoSource produces the following error in StaxRip: "Invalid D2V File, It is Empty!"

So, obviously I need to create a D2V file for each MKV if I want to use MPEG2Source.

When ffms2 indexes, it creates an index file with an ffindex extension. For Lsmash it's LWI and for DGIndex it's D2V. They're not interchangeable. StaxRip probably does it for you but DGIndex can't index MKVs.

After DGIndex has created an index file it's opened by DGDecode with mpeg2source. They're two separate programs but designed to work together. The index file is normally created by a GUI via the command line, as they can't run DGIndex via a script, although DGIndex has it's own GUI. The index files are normally opened directly as they contain the location of the source file:

mpeg2source("D:\VTS_02_1.d2v")

For ffms2 and lsmash it's an Avisynth plugin to both index and decode. If you open a file with either and there's no index file, they'll create one automatically.

FFVideoSource assumes the index file is in the same folder as the source. If it's not you can use the cache argument.

FFVideoSource("E:\VTS_02_1.mkv", RFFMode=1, cachefile="D:\SomeFolder\VTS_02_1.ffindex")

For LSmash you can do it either way these days. If you were to open an MKV it'd look for an lwi index file in the same folder. If it doesn't find one it'll index, unless you use it's cache argument, but you can also open index files directly as they contain the location of the source file.

LWLibavVideoSource("D:\VTS_02_1.lwi")

You need to remux the MKVs as TS files so StaxRip can open them instead, index with DGIndex and create a script to decode with mpeg2source, if you want to use DGIndex/DGDecode (unless you have the DVD files ripped and handy). I assume that's how StaxRip will open TS files, but it can't open MKVs with DGIndex/mpeg2source.

There's newer versions of DGIndex that can index MKVs and use a video card to decode rather than your CPU. I think DGIndexNV and DGDecNV are the Nvidia versions. I don't know a lot about them, but StaxRip probably supports the current ones. They're pretty cheap, but they're not free. http://rationalqm.us/dgdecnv/dgdecnv.html

videoh

14th May 2020, 18:02

I think DGIndexNV and DGDecNV are the Nvidia versions. I don't know a lot about them, but StaxRip probably supports the current ones. They're pretty cheap, but they're not free. http://rationalqm.us/dgdecnv/dgdecnv.html Yes, they will index and frame serve MKV. I believe they can be used in staxrip, but they are not bundled in staxrip.

JoelHruska

14th May 2020, 22:37

Hello_Hello,

Your assumption was accurate. I removed the call out to McDegrainSharp because of the error calling it created. I have managed to get your initial script (the one from back on Page 7) to run properly, though I had to remove the "resize8" command (don't have that filter installed at the moment, either, though I'll see about getting it).

I have a question for you.

Allowing for the fact that different people perceive quality differently, it has always been obvious that any effort to create the best possible version of Deep Space 9 or any other TV show would require scene-by-scene or frame-by-frame editing. I do not dispute this. Your version(s) of the credits are all better than what I'd produced.

My original goal, however, was not to create a scene-by-scene scripted guide to how to restore DS9. I do not believe I can afford the time required to painstakingly create a scene-by-scene idealized repair for every single episode -- and that assumes I had the skill to do so. I currently do not.

Here, then, is my question:

How would you approach the question of improving the DVD source if your goal was to improve as much of the underlying content as possible a relatively broad (though lightly applied) brush?

I realize this question may stand very much opposite to the way things are done, and I'm interested in exploring the type of repair you do at the scene-level and frame-level. This kind of low-level manipulation is incredibly interesting. But I do want to make sure that the project I've taken on strikes a balance between difficulty of setup (because I want people to be able to follow a tutorial), total processing time (obvious reason, if it hasn't been a concern for me personally to-date) and then the total difficulty of creating the tutorials in the first place, which lands on my shoulders.

I know that a lighter touch is always better on all but the most mucked-up video. Is it possible to find some light touches that can be run across the entire encode to repair various issues without causing the QTGMC impacts you dislike?

hello_hello

14th May 2020, 23:24

One thing before replying.... I looked at my last encodes again and I think the last modification of the Katie method for IVTC of the CGI section produces the smoothest result. I thought her method looked a bit awful originally (it did) but after experimenting and changing the dienterlaced clips it uses I think it's a little smoother through the fades between shots. The difference is very small, but that's probably the method I'd go with. Anyway....

The idea isn't so much to inspect every scene and apply scene specific filtering, it was to find a good way to IVTC the CGI stuff compared to the rest, assuming all the CGI has similar problems. I gather most episodes will predominantly be film, where standard IVTC can be applied, so if it was me, I would go through each episode (probably with MeGUI's AVS Cutter) and specify Trims to divide a script into film and CGI sections. I've done that sort of thing a lot myself. Once you've saved the Trims it's then easy to apply the same IVTC to each CGI section and the same IVTC to the film sections. You shouldn't need to create a new method of IVTC each time. That part's done now. That's the theory at least, having only seen one sample, but I think putting the extra time into creating the scripts in order to have a nice, clean CFR output will save you time and grief later.

I don't dislike using QTGMC to clean up video, I often run it in progressive mode for DVD sources to denoise, but I use it for that because often DVD video isn't very clean/stable, and as well as denoising, QTGMC can help with that. I'd run something like this over an entire video fairly regularly, even HD sources at times.

QTGMC(InputType=1, EzDenoise=1.5)

I just thought the two instances you were using to repair the CGI were a bit over the top and creating "mush" for want of a better word.

In the case of Deep Space Nine, the picture in the film sections is very stable, from what I've seen. There's a bit of noise, but the underlying picture is quite good. I did try running QTGMC on it at one stage rather than MCDegrainSharp, but the picture itself doesn't need stabilising, just some noise removal, so in this case I thought MCDegrainSharp looked better. Hopefully that makes sense. Aside from the horrid CGI, I haven't seen any issues in other sections yet. If the sample we've been playing with is any indication, standard IVTC and MCDegrainSharp will be fine for all the film sections.

In case it gets lost in the thread, here's the function to IVTC the CGI sections, assuming you want to split the scripts into CGI and film sections with Trim. It's the heavily modified Katie method, and a bit slow, but based on the sample, I think it'll be the best "one size fits all" method of dealing with the CGI parts.

function QIVTC(clip Video)
{
A = Video.Tfm(field=0,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=1,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(field=0,mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=1,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)
QTGMC(Preset="Very Slow", InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)
}

hello_hello

15th May 2020, 00:36

Something important.... that I wasn't thinking about while testing..

If you do divide the script into film and CGI sections, the IVTC process converts five telecined frames into four progressive frames. When you IVTC a whole video that's fine, but when you divide the video up into sections the way I've been doing it for testing, you risk losing a frame in each section, unless you stick to multiples of five for each Trim. Here's a quick example:

This Trim contains 100 frames and at 29.97fps, it's duration is 4 sec 170ms.
Trim(0,99)
Apply IVTC and it now only contains 80 frames at 23.976fps but the duration is the same.
Trim(0,99).TFM()TDecimate()

This Trim contains 101 frames and it's duration is 4 sec 212ms.
Trim(0,100)
After IVTC it contains 80 frames and it's duration drops to 4 sec 170ms
Trim(0,100).TFM()TDecimate()

Sometimes you might create a frame, but with lots of trims the errors could add up and effect the audio sync.
I just checked the effect of the trims I added to the sample, as they weren't multiples of five, and they reduced the total duration by 90ms, which is unsatisfactory, so either stick to exact multiples of five for Trims used for IVTC, or do it by creating two clips instead.

First, add basic IVTC to a script, then add the trims.

TFM()TDecimate()

Trim(0,62) ++ \
Trim(63,715) ++ \
Trim(716,3682) ++ \
Trim(3683,0)

Replace the basic IVTC with two IVTC'd clips, then modify the Trims to splice the clips correctly.

SomeVideoSource()
Video = last
Clip1 = Video.QIVTC() # The CGI IVTC function is applied to the whole clip.
Clip2 = Video.IVTC_Denoise() # Standard IVTC and denoising function is applied to the whole clip.

Clip1.Trim(0,62) ++ \
Clip2.Trim(63,715) ++ \
Clip1.Trim(716,3682) ++ \
Clip2.Trim(3683,0)

It's a bit more work, but you can pick any frame range for the Trims without worrying about audio sync because IVTC has already been applied. I should have been thinking about that from the start... well I did it that way for the first few scripts I posted, then I got lost in the IVTC tests and forgot....but I really thought I should point this out.

JoelHruska

15th May 2020, 02:33

I do not yet understand the organizing principle behind this TV show's use of various frame rates and content sources. All of the statements below are 100% true and I can back them up with specific scenes.

1). Much of the CGI footage is encoded at 23.976 progressive in Season 6. In other places, however, the CGI footage is 29.97 interlaced. This appears to be more common in Season 7.

2). The non-CGI sections of the show are mostly -- but not entirely -- in 23.976 fps. There are at least two places in "Sacrifice of Angels," where footage bounces up to 29.97 fps and then back to 23.976 fps in the middle of shooting a conversation between characters.

3). Sometimes, space shots are interlaced footage. Sometimes they are not. Sometimes, they are 29.97 fps. Sometimes they are not.

4). The credit sequence of the show is the ugliest, *ugliest* part of it, by far. The original DVD looks like it was recorded off someone's VHS. Watch when the camera pans over the station, in particular: https://www.youtube.com/watch?v=xL8nPEmLKLE

Fixing the credits the way you did is amazing, if only because they were so awful before.

I can *show* you something of the effect I am trying, specifically, to create. I created both of the videos below (please set YT to 4K -- and yes, while video takes a hit, you can still see the effect I'm specifically going for, so I'm not worried about it).

https://www.youtube.com/watch?v=DzeENB9JKmE

That's an upscaled MKV file of DS9. I took a pre-existing encode of the show and ran it through Topaz VEAI Gaia-CG. That's the result. When the clip starts, watch the leading edge of the Galaxy ship and how smoothly it moves towards the viewer. Watch the level of noise on the ship hulls. (I mostly care about the first 10 seconds of the clip as far as a demonstration of what I'm looking for).

Now compare it to this video (set for 4K) -- particularly how smoothly the approaching edge of the Galaxy saucer moves, but also hull noise and the like.

https://www.youtube.com/watch?v=8DwfHADU0QY

The file I call "Rubicon" is much smoother than the upscaled MKV, even though it's still running at 23.976 fps. But the production process for this image quality is whack as *hell*, and it can't be good for the file.

Start with a Handbrake CFR rip. Encode in AviSynth with the QTGMC process I posted originally + daa3mod. Run the result through DaVinci Studio Resolve, increasing the framerate to 119.88 fps. Then, run the result of *that* encode through AviSynth again via "ChangeFPS(24000, 1001)" filter to change the frame rate back to 23.976 fps.

Then upscale in Topaz VEAI.

This is obviously rather tortured, and there's evidence. The very first frame of the video (you may need to rewind to see it) shows a distorted stardrive deflector shield on the approaching Galaxy-class ship for a single frame.

What I like about Rubicon, specifically, is how it removes noise, has excellent smooth motion (to my eye, anyway -- I am not saying it couldn't be better), reduces flicker on the hulls of the ships (check the Galaxy-class vessel in the upper left for window flicker), and antialiases the hulls. Is there a gentler way to achieve this kind of frame rate improvement via IVTC?

poisondeathray

15th May 2020, 04:43

Start with a Handbrake CFR rip. Encode in AviSynth with the QTGMC process I posted originally + daa3mod. Run the result through DaVinci Studio Resolve, increasing the framerate to 119.88 fps. Then, run the result of *that* encode through AviSynth again via "ChangeFPS(24000, 1001)" filter to change the frame rate back to 23.976 fps.

Then upscale in Topaz VEAI.

Sorry if I missed it, but what is the reason for Resolve ? Are you only using resolve for optical flow 23.976=>119.88 ? ChangeFPS is just dropping what you just interpolated (*5 , /5). But what is the purpose ?

If Resolve resamples all frames (does not keep the original set of frames in the interpolation), then you will get slightly different no matter where in the cycle you choose. If it keeps original frames, then it depends where in the cycle the you select (0,1,2,3,4) . You have more control over the offset with SelectEvery .

There might be some other options to get what you want, and avisynth has optical flow options if you needed it for some reason

This is obviously rather tortured, and there's evidence. The very first frame of the video (you may need to rewind to see it) shows a distorted stardrive deflector shield on the approaching Galaxy-class ship for a single frame.

You would expected contamination and blending, if there is no built in scene detection for some types of temporal filters and optical flow.

Or you can control the application of filters to specific sections to avoid those sorts of issues

What I like about Rubicon, specifically, is how it removes noise, has excellent smooth motion (to my eye, anyway -- I am not saying it couldn't be better), reduces flicker on the hulls of the ships (check the Galaxy-class vessel in the upper left for window flicker), and antialiases the hulls. Is there a gentler way to achieve this kind of frame rate improvement via IVTC?

Using IVTC only - will not smooth over the flicker from aliasing. But possibly IVTC + other filters you might get similar (or maybe better) results . Post the original clip and someone will give you some suggestions

JoelHruska

15th May 2020, 04:48

One more note. I tested the following script from back on page 7:

FFVideoSource("%source_file%", cachefile="%source_temp_file%.ffindex", rffmode=1)
#AssumeFPS(25)
Crop(8,0,-8,0)

Clip = last

Y = nnedi3(field=-2)
A = Y.Selecteven()
B = Y.Selectodd()
C=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

TDecimate(mode=7, rate=24.0/1.001).AssumeFPS(24000,1001)

QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, InputType=2, MatchEnhance=0.75, Sharpness=0.7, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, prevGlobals="Reuse")
Repair(QTGMC2, QTGMC3, 1)

E = last

F = Clip.TFM().TDecimate().MCDegrainSharp()

E.Trim(0,75)++\
F.Trim(76,2971)++\
E.Trim(2972, 0)

GradFun3()

This did not work very well, as far as reducing judder or smoothing motion -- the judder was quite poor from frame to frame. Some of the CGI scenes were flawless, others were quite badly affected. I will obviously test the other code ideas -- just wanted to report back on the first one.

wonkey_monkey

15th May 2020, 07:54

2). The non-CGI sections of the show are mostly -- but not entirely -- in 23.976 fps. There are at least two places in "Sacrifice of Angels," where footage bounces up to 29.97 fps and then back to 23.976 fps in the middle of shooting a conversation between characters.

This may have been done to seamlessly alter the timing of the conversation and shorten a pause.

Katie Boundary

15th May 2020, 08:06

TBH that result speaks for itself, just drop it.

Any results that you might perceive of as wonky, are that way because that's exactly the way the DVD is encoded. My method perfectly preserves field-accuracy.

Hey all,

Just thought I'd post here and introduce myself. I'm Joel Hruska, from ExtremeTech, and I've been working to upscale Deep Space 9.

Hi Joel! World Domination Studios here. As you can see, most people here are hell-bent on making this process much more complicated than it really is. I see that you're still using Handbrake and Staxrip instead of Smartripper, DGindex, and AVIsynth, and you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?

Also, we did get your email about the retrograde field behavior. We apologize for not responding to it earlier. To use the trim command to get rid of it, the script would look something like this:

mpeg2source("arbitrary ds9 episode.d2v")

A=nnedi3(field=-2).selecteven()
B=nnedi3(field=-2).selectodd()
C=Tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=B,micmatching=0)
E=Tfm(field=1,mode=0,pp=5,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,micmatching=0)
F=Tfm(field=0,mode=0,pp=5,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,micmatching=0)

X=interleave(C,D)
Y=interleave(E,F)

Trim(X,0,82)+trim(Y,83,247)+trim(X,248,0)

...where frames 0 through 82 are in a "sane" order, frames 83 through 247 are the ones with the retrograde field behavior, and then frames 248 onward are clean again ("0" means "last frame" when used as the second parameter for trim). This will produce blended frames where the retrograde field behavior used to be, but it will at least preserve all of the original fields in their original locations in the timeline. If you're willing to sacrifice some orphaned fields to kill the blending, the script would look more like this:

mpeg2source("arbitrary ds9 episode.d2v")

A=nnedi3(field=-2).selecteven()
B=nnedi3(field=-2).selectodd()
C=Tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=B,micmatching=0)

X=interleave(C,D)
Y=separatefields().doubleweave().tfm(field=1,mode=0,slow=2,cthresh=2,MI=40,blockx=8,mthresh=2,clip2=A,micmatching=0)

Trim(X,0,82)+trim(Y,83,247)+trim(X,248,0)

Just remember that your sole priority here is to deinterlace so that when you upscale the footage, you're not upscaling combed frames. Any and all other bullshit, like denoising or trying to convert to variable frame rate, is a distraction, and should be dealt with after deinterlacing - not as part of a single process that tries to do 20 different things at once and sucks at all of them.

https://i.pinimg.com/originals/90/ce/d6/90ced64d3a6a76cd714042e8f7276d8f.jpg

Groucho2004

15th May 2020, 08:59

you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI.You're funny.

Just in case you weren't joking and suggesting to stuff H.264 / H.265 into AVI - You appear to operate in a different reality than most of us.

wonkey_monkey

15th May 2020, 09:34

Any results that you might perceive of as wonky, are that way because that's exactly the way the DVD is encoded. My method perfectly preserves field-accuracy.

Your method failed spectacularly on the sample clip. Don't blame the DVD; you posted an untested script and falsely claimed that it worked.

JoelHruska, I really can't stress enough how little credence you should give to anything Katie Boundary says. This is not me being rude; it's simply that more often than not, they just don't know what they're talking about, often insist on using objectively worse methods for nothing other than arbitrary reasons, and simply refuse to listen to any dissenting advice.

Stereodude

15th May 2020, 16:12

So before feeding into Topaz, I do ConvertToRGB24(matrix="PC.601"), and after upscaling with Topaz I convert back with ConvertToYV12(matrix="PC.601"). Is that right?

1) Why would you want to limit yourself to 8-bit RGB color?

2) The built in functions have chroma positioning errors. Using z_ConvertFormat (http://avisynth.nl/index.php/Avsresize) would be preferred.

zapp7

15th May 2020, 16:40

1) Why would you want to limit yourself to 8-bit RGB color?

2) The built in functions have chroma positioning errors. Using z_ConvertFormat (http://avisynth.nl/index.php/Avsresize) would be preferred.

I'm a noob and still just learning about color spaces. It seems the original DVD footage is 8-bit (at least as reported by MediaInfo on the .M2V file), so is there any advantage to going higher?

huhn

15th May 2020, 16:51

a colorsapce doesn't have a bit deep but a DVD is 8 bit.
if you do math with pixel you can end up with fraction so you need more bit deep to store these result accurately or you will get side effect like banding.

the "colorspace" used on DVD is usually BT 601 and what was the name SMPTE 170M there was more i can't remember EBU or something. so you have to check first what it is the sample was matrix bt 601 and primaries SMPTE 170M.

JoelHruska

15th May 2020, 16:57

Your method failed spectacularly on the sample clip. Don't blame the DVD; you posted an untested script and falsely claimed that it worked.

JoelHruska, I really can't stress enough how little credence you should give to anything Katie Boundary says. This is not me being rude; it's simply that more often than not, they just don't know what they're talking about, often insist on using objectively worse methods for nothing other than arbitrary reasons, and simply refuse to listen to any dissenting advice.

I don't know who knows what. The attitude I'm taking is that the code will either prove to work or prove not to work. Right now, I haven't found an IVTC-based approach that could be run against the entire episode to both fix the random impact of interlacing and 29.97 fps but *doesn't* really obviously create other frame timing issues, like people walking backwards in the middle of movement. I'm trying to avoid having to program frame-by-frame comparisons, though I'm going to test this approach on Sacrifice of Angels over the weekend, because that's the episode I've worked on by far the most.

I have only barely started testing IVTC, however, and the functions written by hello_hello and others are far more complex than anything I tested, so I want to look at this carefully before I start saying what will or won't work. What I want, ideally, is a function that will fix the occasional instances of interlacing and 3:2 pulldown without overly changing anything else, and I'm *really* hoping for something I can run against every DS9 episode as a global script.

see that you're still using Handbrake and Staxrip instead of Smartripper, DGindex, and AVIsynth, and you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?

Tons. In fact, you should assume every single decision I've made has been made deliberately, in an attempt to solve a problem. That doesn't mean I'm solving it in the best way, but there's no part of my process that isn't deliberate.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.

Stereodude

15th May 2020, 17:04

I'm a noob and still just learning about color spaces. It seems the original DVD footage is 8-bit (at least as reported by MediaInfo on the .M2V file), so is there any advantage to going higher?
8-bit YUV isn't the same as 8-bit RGB. Converting back and forth between color spaces isn't a perfectly lossless operation in most cases. The color space conversion math is decimal based matrix math and doesn't map exactly to the 255 discrete steps 8-bit has. The more bits you keep through your process generally the better the final output will be. Additionally, you're upscaling the chroma in that conversion. RGB has full chroma resolution. DVD uses 4:2:0 YUV color, so the chroma resolution is 1/4 of the luma. The Luma is 720x480 and the chroma is 360x240. So you are upsampling the chroma in the step converting to RGB with a very rudimentary algorithm (compared to what Topaz uses).

I'm not familiar with Topaz and what color formats and bit depths it accepts. It may be better to feed it the 8-bit 4:2:0 YUV video directly (if it accepts it) and save the upscaled output from Topaz in >8-bit losslessly compressed video.

zapp7

15th May 2020, 17:41

I'm not familiar with Topaz and what color formats and bit depths it accepts. It may be better to feed it the 8-bit 4:2:0 YUV video directly (if it accepts it) and save the upscaled output from Topaz in >8-bit losslessly compressed video.

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.

The z_ConvertFormat is a bit over my head, but based on the doc, my guess is the command I need to go from YUV -> RGB is:

z_ConvertFormat(pixel_type="RGBPS",colorspace_op="470bg:601:470bg:full=>rgb:601:470bg:full")

After upscale, to go from RGB back to YUV with Rec.709:
z_ConvertFormat(pixel_type="YUV420PS",colorspace_op="rgb:601:470bg:full=>709:709:709:full")

I should also discuss the command I'm using to make a video out of the avs script (uses the override file workflow StereoDude helped develop). This video will then be fed to Topaz.

ffmpeg -i inputfile.avs -c:v libx264rgb -preset ultrafast -crf 0 -vf scale=-2:480,setsar=10/11 outputfile.mkv

I'm encoding instead of using the copy command because I want to set the SAR to 10:11. I read in the ffmpeg docs that -crf 0 is essentially lossless encoding. The libx264rgb is used since the format is rgb.

I'm sure I messed up a bunch and would appreciate any advice. :)

Stereodude

15th May 2020, 17:50

I don't know who knows what. The attitude I'm taking is that the code will either prove to work or prove not to work. Right now, I haven't found an IVTC-based approach that could be run against the entire episode to both fix the random impact of interlacing and 29.97 fps but *doesn't* really obviously create other frame timing issues, like people walking backwards in the middle of movement. I'm trying to avoid having to program frame-by-frame comparisons, though I'm going to test this approach on Sacrifice of Angels over the weekend, because that's the episode I've worked on by far the most.

I have only barely started testing IVTC, however, and the functions written by hello_hello and others are far more complex than anything I tested, so I want to look at this carefully before I start saying what will or won't work. What I want, ideally, is a function that will fix the occasional instances of interlacing and 3:2 pulldown without overly changing anything else, and I'm *really* hoping for something I can run against every DS9 episode as a global script.

Tons. In fact, you should assume every single decision I've made has been made deliberately, in an attempt to solve a problem. That doesn't mean I'm solving it in the best way, but there's no part of my process that isn't deliberate.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.
My personal opinion having played around with the first few episodes of Season 6 is that there's no compelling reason to get a VFR output. The show is for all intents and purposes is (24/1.001)fps. Just because the DVDs have p24 encoded sections with a soft pulldown applied and i60 encoded sections with a hard pulldown applied doesn't mean the output should be VFR.

There are some i60 text graphics inserted on top of it, fades in and out of black at the commercial breaks that are i60 (on top of the telecined p24), and the end credits are not 24fps. The show intro is i60 edited telecined film so you can't perfectly IVTC the intro where it fades between shots. However, no one is watching the show for the end credits or the intro (that's in every episode). You're not losing anything of importance with p24 (drop frame) output. You can blend the end credits to p24. So what if the star field behind the credits is blended? And the intro ends up with a few 5 frame segments that gets blended to p24 instead of decimated to p24. Hardly a loss. That leaves you with very few frames in an episode where an IVTC can't perfectly restore the underlying footage. The underlying footage is the concern not the titles that are faded in and out on top of it right?

I haven't found any actual i60 content (that should be turned into p60 by deinterlacing) in the show itself in the episodes I've looked at that. It's all p24 content that was edited as video (i60) after the p24 content was turned into i60 via a telecine.

The IVTC works. The problem is that the extra things done to the show when it was edited as i60 video occasionally break the IVTC pattern detection algorithms and require per episode override files if you want the best possible output to fix the few frames that it gets wrong per episode. That blows up your desire to be able to treat every episode the exact same and just run them through an entirely automated process.

You might also get marginally better output on the frames that get blended to p24 (instead of decimated to p24) if you waited to blend them until after they were upscaled by Topaz, but that further complicates the whole process.

Stereodude

15th May 2020, 18:11

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.
I will have to go back and look at the posts closer. I only skimmed some of the earlier posts. It may or may not be a problem. Or the fix may be worse than the problem.

The z_ConvertFormat is a bit over my head, but based on the doc, my guess is the command I need to go from YUV -> RGB is:

z_ConvertFormat(pixel_type="RGBPS",colorspace_op="470bg:601:470bg:full=>rgb:601:470bg:full")

After upscale, to go from RGB back to YUV with Rec.709:
z_ConvertFormat(pixel_type="YUV420PS",colorspace_op="rgb:601:470bg:full=>709:709:709:full")
Not quite...

z_ConvertFormat(pixel_type="RGBPS", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

&

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Also, I suspect that Topaz may not accept RGBPS video, but it might.

I should also discuss the command I'm using to make a video out of the avs script (uses the override file workflow StereoDude helped develop). This video will then be fed to Topaz.

ffmpeg -i inputfile.avs -c:v libx264rgb -preset ultrafast -crf 0 -vf scale=-2:480,setsar=10/11 outputfile.mkv

I'm encoding instead of using the copy command because I want to set the SAR to 10:11. I read in the ffmpeg docs that -crf 0 is essentially lossless encoding. The libx264rgb is used since the format is rgb.

I'm sure I messed up a bunch and would appreciate any advice. :)
I'm pretty sure this doesn't do what you think it does. You will still get a YUV encoded output.mkv file. It will not be RGB.

poisondeathray

15th May 2020, 18:15

poisondeathray

15th May 2020, 18:19

My personal opinion having played around with the first few episodes of Season 6 is that there's no compelling reason to get a VFR output. The show is for all intents and purposes is (24/1.001)fps. Just because the DVDs have p24 encoded sections with a soft pulldown applied and i60 encoded sections with a hard pulldown applied doesn't mean the output should be VFR.

At least one space battle definitely has 29.97p content. Someone posted in another forum a sample. How many scenes like this ? I never got into watching DS9 compared to the other series

https://forum.videohelp.com/threads/396652-Deinterlacing-Deep-Space-Nine-NTSC#post2578796

Some people reported "interlaced" . But is there actually any real interlaced content in this series ? By that I mean actual content with 59.94 different moments in time . Not some interlaced fade, not text overlays, or some orphan field.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content. VFR is the only way to have everything run at the correct frame rate.

My opinion #2 - One filter for everything is works good on some sections, but not so good on others. Definitely I would filter the problem aliasing sections selectively , otherwise you kill the details on 95% of the other sections. Counterproductive when upscaling. Analogy - A F1 car is great on a track, but sucks on city streets with potholes or offroading. A 4x4 is great for off roading but sucks on the track.

Katie Boundary

15th May 2020, 18:21

You're funny.

Just in case you weren't joking and suggesting to stuff H.264 / H.265 into AVI - You appear to operate in a different reality than most of us.

I never said anything about h.264 or h.265

1) Why would you want to limit yourself to 8-bit RGB color?

Because that's already more colors than the human eye and brain can distinguish

I'm *really* hoping for something I can run against every DS9 episode as a global script.

Unfortunately, if the episodes do contain retrograde field behavior as you say, then there's no script that you can copy and paste 176 times, changing nothing but the episode number each time, that will deliver anything close to a perfect result. The only methods that exist to fix retrograde field behavior are very destructive and should not be used on other kinds of interlaced video. You're going to have to choose a poison here: damaging perfectly salvageable content by blending it, damaging perfectly salvageable content by discarding fields, or going through every episode frame by frame to find the RFB and using Trim to fix it. RFB is a curse straight from the pits of Hell.

Handbrake: Handbrake is being used because StaxRip will not properly align a MakeMKV VFR file. Handbrake is also being used because DaVinci Resolve Studio will not read MPEG-2 or MKV. Handbrake will output M4V files. M4V files are acceptable to DaVinci Studio Resolve.

StaxRip: StaxRip is being used because it offered access to the filters I wanted to run at the time, and the front-end makes it easier to launch multiple encode instances simultaneously. I do not like StaxRip very much and it is not really in active development. It has bugs. But I have not found a better front-end GUI yet. I am not wedded to StaxRip at all. I just want something that works and is relatively easy to use when launching multiple instances of the application at once.

MKV over AVI: I couldn't care less about container format, except that certain dependency chains must be fulfilled. StaxRip cannot encode a native M4V file without extracted timecodes. DaVinci Studio Resolve cannot ingest MPEG-2 or MKV. StaxRip also cannot align A/V playback when using FFVideoSource.

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

If that's correct, then you could get rid of the vast majority of your problems by just making this a flat 59.94 FPS constant frame rate project. AVI works with everything and its only drawback that is relevant to this project is that it can't do VFR, which shouldn't even be a priority anyway.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content.

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

zapp7

15th May 2020, 18:26

I'm pretty sure this doesn't do what you think it does. You will still get a YUV encoded output.mkv file. It will not be RGB.

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?

There is a negligible difference between 8bit vs. 16bit in this workflow given this source. Chroma upsampling/down for 1 generation using internal avs function vs. zlib is pretty negligible too

I believe you, but I might as well use a better command if available, even if improvement is marginal.

Stereodude

15th May 2020, 18:28

Stereodude

15th May 2020, 18:30

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?
Maybe I misunderstood the FFMPEG help. I think you need to go to .tiff or .png to get the advantage poisondeathray is talking about.

zapp7

15th May 2020, 18:34

Just to be clear, that ffmpeg command is for the input video into Topaz. I'm saving the output from Topaz as .png images because it's lossless and the only video alternative is an antiquated H.263 encoding in mp4 format. We're all hoping they add better video options in the future.

Stereodude

15th May 2020, 18:41

I was initially feeding it the YUV footage, however according to poisondeathray Topaz Video Enhance AI will convert to RGB and in the process it will clip the range to 16:255 instead of preserving the full range. That's why I was trying to convert to RGB first, and convert back to YUV after the upscale.
It looks like you are conflating two different things. You don't want to feed a lossy compressed intermediate file into Topaz or take a lossy output from Topaz. You would want to take the output from the IVTC script, losslessly compress it and feed that file into Topaz (presuming Topaz can't open the .avs). If you want to go the RGB route you would convert to RGB in the script, export the .avs to .tiff or .PNG, and run those through Topaz. Output from Topaz should be lossless also.

There are pros and cons to converting to RGB. If you want to color grade the episodes after upscaling, then going to RGB probably make sense (presuming Topaz can't accept >8-bit YUV video input). Otherwise, I'd probably steer clear unless you find that Topaz handles the YUV-RGB conversion poorly.

If you are planning to color grade it later those conversion lines I pasted are not the right ones to use. They are full to full. You would want to go full to limited in the first step. You will be compressing the colors range, so you definitely don't want to map it to 8-bit RGB.

z_ConvertFormat(pixel_type="RGBPS16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:l")

The step to come back is more complicated and depends on how you've color graded it (and where in the remaining steps you're going to color grade it). Which is why I haven't provided you the line because it's so highly variable on what is correct.

Just to be clear, that ffmpeg command is for the input video into Topaz. I'm saving the output from Topaz as .png images because it's lossless and the only video alternative is an antiquated H.263 encoding in mp4 format. We're all hoping they add better video options in the future.
Understood. Even if it real RGB, it's going to be limited to 8-bits, or 10-bits. If you're trying to get fancy with RGB you'd want to use Topaz on 16-bit .png or tiff files (for the input).

poisondeathray

15th May 2020, 18:50

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

Maybe "judder" would have been a better term

The reason some people in this thread should not do this is duplicate frames. Some filters (many temporal, "artemis" model in upscale AI) do not work as well with hardcoded duplicates. eg. If Joel wants to clean up aliasing sections with temporal antialiasing filters, duplicates will make it less effective

Another reason is why encode ~2x more frames if you don't have to? Upscale AI isn't the fastest, even if you have a monster hardware. You end up with larger filesizes too for intermediates and final versions

Even if I convert to RGB in the avs script and encode with libx264rgb? MediaInfo also reports that the outputted file has RGB color space. Is there another way to verify?

It depends on what Enhance AI is using to decode.

You can check the levels before and after.

I believe you, but I might as well use a better command if available, even if improvement is marginal.

Fair enough, the main trade off there is the filesize for the intermediates.

If you zoom in 400% - you can barely see the difference. But the other things I mentioned - you can easily see the differences, those are worthwhile

Groucho2004

15th May 2020, 19:08

I never said anything about h.264 or h.265You did not, but your question to JoelHruska was:
...you're also trying to encode to exotic containers like MKV instead of the more well-supported AVI. Any particular reasons for that?And from his early posts you can clearly see that his target format is either H.264 or H.265.

Also, calling MKV an exotic container is simply ridiculous.

Groucho2004

15th May 2020, 19:14

1) Why would you want to limit yourself to 8-bit RGB color?
Because that's already more colors than the human eye and brain can distinguish
You must have a really crappy monitor. For example, I can clearly see differences (on my 10 bit Eizo screen) between 24 bit and 48 bit images, particularly when it comes to banding in gradients.

JoelHruska

15th May 2020, 20:08

At least one space battle definitely has 29.97p content. Someone posted in another forum a sample. How many scenes like this ? I never got into watching DS9 compared to the other series

https://forum.videohelp.com/threads/396652-Deinterlacing-Deep-Space-Nine-NTSC#post2578796

Some people reported "interlaced" . But is there actually any real interlaced content in this series ? By that I mean actual content with 59.94 different moments in time . Not some interlaced fade, not text overlays, or some orphan field.

My opinion #1 - Single rate anything - IVTC, deinterlace, whatever - will be jerky in sections because you have mixed content. VFR is the only way to have everything run at the correct frame rate.

My opinion #2 - One filter for everything is works good on some sections, but not so good on others. Definitely I would filter the problem aliasing sections selectively , otherwise you kill the details on 95% of the other sections. Counterproductive when upscaling. Analogy - A F1 car is great on a track, but sucks on city streets with potholes or offroading. A 4x4 is great for off roading but sucks on the track.

I don't know enough to want to make absolute pronouncements, yet, but those clips that were linked are from "Paradise Lost," (Season 4, Ep 12) which means there's just all sorts of fun stuff I haven't seen. At this point, I think the only sane thing to do is to assume that DS9 has basically everything in it, somewhere. We know that most of the show is in 23.976 fps, but even if that's 90-95%+, we also know that the last 4-5% is made of lots of other things. There are interlacing artifacts baked directly into source (I believe this is "hard" telecine?).

Deep Space 9 did not have a formula where non-CGI was 23.976 and CGI was 29.97 or 60i or what have you. It seems to have been more along the lines of "Mostly 23.976, but we do what we have to, in order to make the rest work."

I'm starting to appreciate why TNG, DS9, Stargate, and Babylon 5 are all called out as tricky in the AviSynth wiki.

JoelHruska

15th May 2020, 20:22

You did not, but your question to JoelHruska was:
And from his early posts you can clearly see that his target format is either H.264 or H.265.

Also, calling MKV an exotic container is simply ridiculous.

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."

I am completely agnostic about this entire process, and it's a good thing I am, because the applications I'm using have proven to be far more picky than I am. I output to H.264 because it's convenient. I had intended to test H.265 because it might offer better compression. I'll break out DIVX and .mpeg files if they turned out to be the best way to remaster the show.

Groucho2004

15th May 2020, 20:33

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."Pragmatic approach, I agree.

I'll break out DIVX and .mpeg files if they turned out to be the best way to remaster the show.Neither of those are very efficient nowadays. I'd say H.264 is your best bet. For low(ish) bitrates it seems that H.265 is more efficient. Just try them both at the bitrate you're comfortable with.

zapp7

15th May 2020, 20:45

Not quite...

z_ConvertFormat(pixel_type="RGBPS", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

&

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

So I converted to RGB using:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Saved that as 16 bit tiffs. Ran through Topaz. Then when I convert back to YV12 using:

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

it throws an error saying "clip must be planar", likely due to it being RGB48? How can this be tackled?

Also, since I'm saving as tiffs I'm not sure how to apply the SAR of 10:11. The result is that for 2160p, Topaz wants to produce a frame size of 3168x2160, instead of 2880x2160. What is the best point in this workflow to fix that, and how?

wonkey_monkey

15th May 2020, 21:02

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

The irony of you telling someone not to insist on doing something odd is off the charts.

A pattern of frames that alternate between lasting 1/20th of a second and 1/30th of a second is NOT jerky, at least not to anyone with human eyes and a human brain.

It absolutely IS jerky to anyone with decent eyes.

It's not necessarily obvious - especially to someone who has grown up watching telecined video - but if you compare it to a video that's truly 24fps the difference is like night and day.

Stereodude

15th May 2020, 21:14

So I converted to RGB using:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Saved that as 16 bit tiffs. Ran through Topaz. Then when I convert back to YV12 using:

z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

it throws an error saying "clip must be planar", likely due to it being RGB48? How can this be tackled?
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

What is your plan for the final output resolution? Regardless I would generally suggest that you should not convert to 8-bit YUV here. I would suggest keeping 16-bits (as above).

Use f3kdb to convert down to 10-bits for your HEVC compression.

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

Also, since I'm saving as tiffs I'm not sure how to apply the SAR of 10:11. The result is that for 2160p, Topaz wants to produce a frame size of 3168x2160, instead of 2880x2160. What is the best point in this workflow to fix that, and how?
You can't force a particular resolution for the output of Topaz? Does it have bars on the sides, or does it fill the entire frame?

If it fills the frame you will have to squeeze it horizontally. I would probably squeeze it in the z_ConvertFormat line.

like:
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

As to which scalar (resample_filter) you should use, I don't know. There will be lots of opinions.

Then it will have a SAR of 1:1 and you can encode it with x265 as 10-bit HEVC.

zapp7

15th May 2020, 22:17

ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

What is your plan for the final output resolution? Regardless I would generally suggest that you should not convert to 8-bit YUV here. I would suggest keeping 16-bits (as above).

Use f3kdb to convert down to 10-bits for your HEVC compression.

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

You can't force a particular resolution for the output of Topaz? Does it have bars on the sides, or does it fill the entire frame?

If it fills the frame you will have to squeeze it horizontally. I would probably squeeze it in the z_ConvertFormat line.

like:
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

As to which scalar (resample_filter) you should use, I don't know. There will be lots of opinions.

Then it will have a SAR of 1:1 and you can encode it with x265 as 10-bit HEVC.

Thanks! This all seems to work pretty well and I'm getting a 10 bit HEVC output now.

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.

JoelHruska

15th May 2020, 22:25

It sounds like the vast majority of your dependency and compatibility problems are being caused by your insistence on odd container formats like M4V, which in turn I'm assuming is the result of a perceived need to make this a VFR project.

Not a "perceived need" as such. A desire to explore what outcomes looked like. I went to 119.88 to resolve VFR issues because that's one method of doing it. I also have tried various approaches with QTGMC to fix judder at 23.976 fps. I took the script you posted on Page 1 and combined it with my own repair formula to create a fairly good-looking 48 fps variant (if run through MakeMKV) or 60 fps (if used on a VOB).

Which one of these did I create? All of them. The only reason I didn't showcase far more footage than I did in my last article was because I ran out of time and the article had too many clips in it already. I rendered out six different clips from Sacrifice of Angels using native DVD, a generic upscaled MKV, and then my own work at 23.976 fps output, 48 fps, 60 fps, and 119.88 fps. Then I upscaled all of them in Gaia-CG and Gaia-HQ. (Obviously the DVD didn't get upscaled).

6 clips * 6 outputs * two upscale methods = Nearly 70 videos I created for that story. Turned out to be too much, so I dumped most of it. But I did the work.

The encode methodology I call "Rubicon" is not a 119.88 fps file. It's a 23.976 fps file that was temporarily pushed to 119.88 fps by DaVinci before being decimated back to 23.976 fps by AviSynth using ChangeFPS(24000,1001).

I'm not wedded to VFR. I'm wedded to nuking judder, preferably in a method I can apply across an entire episode at once. All of the container choices I have made have been attempts to work around problems I was having, not some attempt to use a particular file format or codec that I'm in love with.

Stereodude

15th May 2020, 22:26

Thanks! This all seems to work pretty well and I'm getting a 10 bit HEVC output now.

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.
Well, you can use AVSmeter to process the .avs as fast as possible vs. playing it real time presuming that's limiting factor.

I think you can also use ffmpeg to turn a .avs into a series of images but I've never done it and am not sure of the command syntax.

poisondeathray

15th May 2020, 22:39

One other question, I'm using ImageWriter in the IVTC avs script to save the output to 16 bit png images. What's the speediest way to do this? Right now I'm just clicking play in AvsPMod and it takes a long time.

Probably

avsr64 script.avs

But it's slow, probably not much faster.

How are you loading the 16bit TIFF's ? If using ImageSource, don't forget pixel_type="RGB48", or they will load as 8bit

zapp7

15th May 2020, 22:59

How are you loading the 16bit TIFF's ? If using ImageSource, don't forget pixel_type="RGB48", or they will load as 8bit

Good catch! I've added that.

Seems like avsr64 is similar speed to AVSmeter, however I can run multiple instances of avsr64 and run several episodes in parallel!

Katie Boundary

16th May 2020, 03:37

Maybe "judder" would have been a better term

The reason some people in this thread should not do this is duplicate frames. Some filters (many temporal, "artemis" model in upscale AI) do not work as well with hardcoded duplicates. eg. If Joel wants to clean up aliasing sections with temporal antialiasing filters, duplicates will make it less effective

An interesting point, but the only frames that should require temporal filtering are the ones derived from orphaned fields to begin with.

Another reason is why encode ~2x more frames if you don't have to? Upscale AI isn't the fastest, even if you have a monster hardware. You end up with larger filesizes too for intermediates and final versions

That's actually a very good point. I'm not sure it justifies complicating the workflow to the edge of unusability, but rescaling the same frame 3x does seem like a huge waste of CPU cycles.

Also, calling MKV an exotic container is simply ridiculous.

It's not an industry standard like MP4 is, and it's not compatible with most video editing programs like AVI is. It's as exotic as a format can get without being some proprietary bullshit that you'd never see outside a digital camera.

My target format is: "Whatever looks the best and works with the most software and lets me do what I want to do."

Well container format doesn't really have anything to do with how good the video looks. For working with the most software, AVI is pretty much the format that every program can import and every program can export. MPEG can be correctly interpreted by a very slightly broader range of hardware and software, but AVI can be exported by a vastly broader range of programs, like Premiere and Virtualdub; if you want to create a .mpg file, you'd have to create an AVI file or AVIsynth script or something and then load it into Tsunami MPEG Encoder, also known as TmpegEnc, to convert it to .mpg.

I'm not wedded to VFR. I'm wedded to nuking judder

That is unfortunate, because your approach of decimating an entire episode down to 23.976 FPS will create problems in 30 FPS or 60 FPS sections that are much worse than judder. Many years ago, I attempted to apply standard IVTC to an episode of War Planets, and noticed an odd skipping effect, as if decimate() was decimating the wrong frames. Looking at the source stream more closely, I discovered that the episodes were originally rendered at 25 frames per second, not 24, and were converted to NTSC by duplicating every fifth frame, not every fourth. I was decimating one frame per second too many, and it was already noticeable. Cutting 30 FPS content down to 24 fps kills 6 frames per second too many.

Judder in the 24 fps parts is the lesser of those two evils... by far.

If you think VFR is the lesser of all three evils, I won't try to dissuade you from that opinion, but conversion to VFR should be done as a separate step, after you've ripped, indexed, and deinterlaced in CFR. Worrying about judder when you're trying to deinterlace, or even just trying to get video from a VOB file into an AVS script, is like trying to decide what color your next Ferrari should be when your house is on fire.