Deep Space Nine upscale project [Archive] - Page 10

View Full Version : Deep Space Nine upscale project

Pages : 1 2 3 4 5 6 7 8 9 [10] 11

zapp7

26th May 2020, 15:41

So my previous comments about noise filters had one caveat: I was using Artemis HQ in Topaz, which I have discovered already applies its own denoising. So in actuality, I was denoising twice and that's probably why the image looked too smooth for my tastes.

I've produced some sample clips using Gaia-HQ and Gaia-CG in an effort to see how the noise filters look with those. The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking. My QTGMC call is QTGMC(InputType=1). These filters are applied before upscaling with Topaz. (I wonder, would there be any theoretical advantage to applying them after upscaling?)

I'm trying to find a filter that I could apply to everything, since this project is big enough without having to edit noise filters scene-by-scene.

I only ran MCDegrainSharp with Gaia-HQ because it's obvious that the result isn't good - it seems to leave a pattern on the character scenes that doesn't look good. I'm also not too fond of the QTGMC result - perhaps someone can suggest better settings as I'm not very familiar with them. I kind of like the TemporalDegrain2 result, but I fear my eye for detail is lacking as the file size for this one is about half what it is without denoising.

TFM and TDecimate were applied before applying the noise filters. Input and output from Topaz was in .png format. These clips were created with ffmpeg libx265, using the slow preset and RF of 20.

The clips can be found here. (https://ln2.sync.com/dl/9dace98c0/8mr6xhrr-dcbbbfpa-4paijz6x-myybx25t)

Forteen88

26th May 2020, 16:20

...The filters I tested are MCDegrainSharp, TemporalDegrain2 and QTGMC. I just used the default settings for all of those, so I'm open to suggestions for tweaking.When I use TemporalDegrain2, I prefer the setting: TemporalDegrain2(grainLevel=false)
Because that setting don't denoise as much (I'd like to know how to make TemporalDegrain2 denoise even less sometimes, but I don't know what setting to use for that).

Boulder

26th May 2020, 16:22

A suggestion for x265: set the parameter no-sao. SAO will smooth the image even more, sucking out details on its way.

zapp7

26th May 2020, 17:58

Thanks, I'll implement those suggestions and generate some new clips tonight.

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

Boulder

26th May 2020, 19:01

Another question, after the denoising I use gradfun3() for debanding since I saw it used elsewhere on this forum. Is this something that should always be used after denoising?

I use debanding (neo_f3kdb) myself in all my encodes as the last item. Using the preset "low", I haven't noticed any ill effects aside from a slight increase in needed bitrate but it should keep those ugly artifacts away.

zapp7

27th May 2020, 00:45

I did a quality comparison and I'm pretty sure that more detail is retained by applying TemporalDegrain2 after upscaling. This makes sense since more detail is available for the input to Topaz.

I made a new clip with TemporalDegrain2(grainLevel=false) applied after upscale and encoded with no-sao. The result is in "S06E06_GaiaHQ_post_TempDeg2.grainLevel0_nosao.mkv" in this folder. (https://ln2.sync.com/dl/9dace98c0/8mr6xhrr-dcbbbfpa-4paijz6x-myybx25t)

I think this result is pretty good. If I zoom in and compare specific frames with the GaiaHQ_NoDenoising clip, I can't see any significant difference in detail but the noise seems significantly reduced. Could use a second opinion to verify my sanity...

JoelHruska

27th May 2020, 02:30

zapp7

27th May 2020, 02:48

It looks to me like you conserve more detail with the -no sao and applying degrain after upscale.

https://i.imgur.com/YXqP44c.png

What does Gaia-CG look like with these settings?

I put the GaiaCG version in the same folder. I believe GaiaCG already has denoising so I'm also making a GaiaCG_NoDenoising_nosao that should be there shortly.

JoelHruska

27th May 2020, 03:27

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

zapp7

27th May 2020, 04:50

Artemis_HQ and LQ models are both still incapable of upscaling DS9 properly, even when fed the samples H_H decoded. They fail in the same way -- mouths and faces cause meltdowns.

I agree. What did you think of the MQ model in the previous versions? I read some people liked that one but I don't have it on hand.

Stereodude

27th May 2020, 22:08

Also, be advised, Topaz *does* introduce color-shifts on some content. I don't know which or why, but Weyoun's jacket is often color-shifted towards a darker brown.
Are you feeding it TIFF/PNG files or a video file and how are you making them?

Also, how are you getting the image files back into video?

We discussed this with zapp7 earlier in the thread. If don't right you shouldn't have a color shift (unless Topaz is broken or we don't understand what it's doing).

JoelHruska

27th May 2020, 23:11

Stereodude,

This was based on a video file, not PNG or TIFF output. I started to play with lossless at one point but had underestimated the storage requirements (Emissary and even PNG output don't mix unless you've planned on the storage requirement beforehand).

Both HQ-CG (Topaz 1.1.0) and Gaia-CG can cause a color shift, but they do it rather differently. HQ-CG had a *very* subtle lightening effect on DS9. I'm not even sure I dislike it, since one of the persistent complaints about the show is how dark it is. (If I was going to dislike HQ-CG it would be for detail removal, not the very small color shift).

Gaia-CG can deviate more widely than HQ-CG, but where HQ-CG had a very subtle effect on everything, Gaia-CG shows this less globally.

Also, while I haven't played with the lossless upscale option yet, I had planned to use a method like the one CaptRobau gave in his blog posts.

https://captrobau.blogspot.com/2019/05/tutorial-upscaling-video-with-topaz-ai.html

When I duplicated his workflow back when Gigapixel AI was the only tool available, I also duplicated his command and had no problems with it:

ffmpeg -r 29.970628 -f image2 -s 1440x1080 -i example%04d.jpg -i original-audio.wav -vcodec libx264 -crf 20 -pix_fmt yuv420p -acodec copy upscaled_video.mp4

Any specific changes you'd recommend?

Stereodude

28th May 2020, 14:32

Any specific changes you'd recommend?
1) Not using lossy compression until your final compression step.

2) I probably wouldn't convert to 8-bit H.264, but 10-bit HEVC instead.

3) I wouldn't load the images from a ffmpeg command line. HD video has rec.709 color space. DVD has rec.601. I'm guessing Topaz doesn't compensate for that. You need to make that conversion.

We discussed this with zapp7 earlier in the thread.

Convert to RGB in your AVIsynth script, and save the output to 16-bit tiff files. Run those through Topaz and then save the output as tiff files.

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="170m:601:170m:f=>rgb:srgb:170m:f")

Load the 16-bit tiff files into AVIsynth (if using ImageSource, don't forget pixel_type="RGB48") and convert them to 4:3 yuv420 16-bit video with a rec.709 color space.

ConvertToPlanarRGB()
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Then dither down to 10-bits with f3kdb

f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

Feed that into x265 to compress it to 10-bit HEVC.

zapp7

28th May 2020, 17:17

I've given up on the 16-bit tiff files. I just don't have the requisite hard drive space as one episode in 16 bit tiff takes more than 2 TB.

I still do the conversion to RGBP16 before Topaz to preserve the full color range, but I save Topaz output as 8-bit .png instead and convert that to YUV420. I honestly couldn't see a difference between the 8-bit and 10-bit final versions.

Stereodude

28th May 2020, 18:25

zapp7

28th May 2020, 18:44

FWIW, if I couldn't only have 1 that's 16-bits, I'd go 8 bit in and 16-bit out.

The 16-bit in isn't an issue because the frame size is smaller. It's the 4K 16 bit files that are massive.

Stereodude

28th May 2020, 18:57

The 16-bit in isn't an issue because the frame size is smaller. It's the 4K 16 bit files that are massive.
Well, you don't have to do the full episode in one shot. You could do some number of frames and convert to a losslessly compressed intermediate file or go straight to HEVC in segments. But the Topaz folks should make it more versatile in terms of what it can open and export.

zapp7

28th May 2020, 19:29

Well, you don't have to do the full episode in one shot. You could do some number of frames and convert to a losslessly compressed intermediate file or go straight to HEVC in segments. But the Topaz folks should make it more versatile in terms of what it can open and export.

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.

ReinerSchweinlin

28th May 2020, 20:48

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.

Yeah, takes a looot of time... Topaz VEAI has CLI support, so you could whip up a batch file to use the two GPUs with one instance..

In general: I guess what Joel is trying to accomplish is to come up with a workflow that can be replicated by anyone so the upscale and encoding could be distributed to many people. I could join in with a s mall batch as soon as things are pinned down.

zapp7

28th May 2020, 21:30

Yeah, takes a looot of time... Topaz VEAI has CLI support, so you could whip up a batch file to use the two GPUs with one instance..

In general: I guess what Joel is trying to accomplish is to come up with a workflow that can be replicated by anyone so the upscale and encoding could be distributed to many people. I could join in with a s mall batch as soon as things are pinned down.

I saw CLI support mentioned on the Topaz forums but I can't find any info on how to use it. Do you know where to find this? I would love to have a batch file to make life easier.

Stereodude

28th May 2020, 21:35

Yeah... problem is I have two GPU's that I want to run in parallel and doing one episode at a time in chunks will take forever with 176 episodes.
A single episode is what ~3.0TB in 3840x2160 uncompressed 16-bit TIFF files + ~150GB for the 720x480 uncompressed 16-bit TIFF files?

I guess you could buy a very large HDD for the project.

scharfis_brain

28th May 2020, 22:12

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

zapp7

28th May 2020, 22:19

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.

Stereodude

28th May 2020, 22:45

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.
That's only if there's actually more detail in the 2160p upscale vs. a 1080p upscale. Has anyone compared that?

johnmeyer

28th May 2020, 23:49

I've asked myself that same question. My gut says to future proof it as much as I can. In other words I may be disappointed that I didn't do 4K when it's so common 10 years from now.You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.

hello_hello

29th May 2020, 01:57

You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.

I assume you haven't looked at any of the upscaled samples?

I'm not saying they're to everyone's taste, but they generally look better than the SD original. At least for the CGI upscaling samples I've seen.

Whether you can do just as well by filtering without upscaling is another question, but if you do you're still at the mercy of the player/display upscaling in respect to how it'll look.

Forteen88

29th May 2020, 08:11

...
Whether you can do just as well by filtering without upscaling is another question, but if you do you're still at the mercy of the player/display upscaling in respect to how it'll look.By "10 years from now", the computers will be so fast that players
can have very high quality resize.

ReinerSchweinlin

29th May 2020, 11:42

I saw CLI support mentioned on the Topaz forums but I can't find any info on how to use it. Do you know where to find this? I would love to have a batch file to make life easier.

It was present in an early version and there was a help file about it, then they removed the feature end re-introduced it some versions ago - but the help file is missing now, I just looked up.. I will recover some of the old betas and search for it and post it here, give me some time, got work to do...

ReinerSchweinlin

29th May 2020, 12:11

Is 3840x2160 really necessary?
IMHO 1440x1080 should be enough. Thus cutting space consumption by a factor of five...

Good point :)

While dealing with AI Upscalers, I learned one thing (I am refering to techniques that deal with single frames only, to keep it simple)

IF the AI starts to "see" somehting, i can re-interpret what it sees and replace the low resolution or noisy stuff with something new - preferably somehting looking similar or at least more pleasing to the eye..

This is very easy with cartoons which mostly have simple geometrical forms as lines, areas, fades, rectangles, etc.. - not to many textures and no "real life footage". So the Ai only has to "know" these types of image content and has an easy job replacing everything with something that looks good and very "realistik"...

As soon as real life footage is to be dealt with, the content becomes endlessly complex... If a Model is trained on landscape footage like mountains, sky, clouds, grass, trees, etc... It will pick up these kinds of objects and "repaints" them - but it will fail horribly on faces and other stuff.

It´s obvious that no model can cover every possible situation. Companies like TOPAZ use training sets which have a wild mix of "most common footage" in them to get a good coverage of all possible situations. But of course, there is a lot missing in the models. That´s the reason why a spezialized model can work so much better on specific footage than a generic one. The game-upscale community is training models on very specific games or techniques to take that into account.

So when throwing DS) into VEAI from Topaz - TOPAZ only can do as much as it knows - which isn´t tooo much in the case of DS9.... I exeprimented with some older DVDs, originally filmed a few decades ago with conventional film-cameras, containing mostly live footage (no special effects, no "mean" postprocessig, effects, filters, just "point the cam and film")... With this kind of material, it works really well (I simply love "police squad" in 1080p now..). But DS9 - not much luck.

(In fact, I would propose whipping up an open source "upscale project" with some nifty AI Upscalers and the possibility to train own models and then take something similar to DS9 available in true HD, downscale it to DVD or record to VHS - train on that and make a "DS9" model.... Startrek Movies, Next Generation, etc... Most of the stuff neede is out there, it´s the matter of having enough GPU power, time, knowledge to whip it all together....

ok, back to DS9 and "how much is enough"...

If the AI picks up: "ah, hair" - it replaces it with new hair, almost similar to the old one. If the model is done right, the alog is good, etc... one could now go up to whatever the model and framework limit - 8K - no problem (with most models)..

But - if the AI doesn`t know what to do - it simply upscales the original, bicubic, Lancoz, etc.... In this case, no more detail is produced.

VEAI now has frameworks which are able to recover details from multiple frames and add some deconvolution stuff into the workflow - so in some cases even if the model isn´t getting "hair", it is able to get some detail back - works good on still shots - in some cases I even got garbage on a pc-screen to reveal the actual numbers on the screen (SG1 NTSC DVD in this case). This mostly is refered to "super resolution". But it has ists limits. A quick turning head simply doesn´t have enough previous and following frames with the same info as the frame in question to recover anything out of the time domain... The Deconvolution sometimes is able to sharpen a little or make it at least visually more pleasing - but in my oppinion, in the case of VEAI - Topaz has done too much of it... This is very visible with compression artefacts (blocking) or noise or halos or ghosting etc... These get pronounced too much by the VEAI implementation..

For the moment, I personaly would stick to 1080p and keep the source for future usage.
In the case of VEAI: I would wait for them to sort things out before making a final upscale.

If I had the time and knowledge, I´d go the route if implemting an open source solution and train a DS9 Model :)

ReinerSchweinlin

29th May 2020, 12:14

You do realize that you get zero improvement in quality by up-scaling and you are therefore completely wasting computer cycles, your time, and storage space forever.

Keep it at its original resolution, for heaven's sake. You are not "future proofing" anything.
John, take a look at some examples on youtube. Some are garbage, but wiht others, you can really see "new" details.... Of course, this is "reinterpretation at higher resolution", not really "upscaling". And it can be wrong (I had some marvel busts in one video - the AI thought "oh, a head, lets pain hair into it"... But in many cases, the new details are quite pleasing to the eye...

So it really differs from tradtional upscaling (if it works, read my post above)

zapp7

29th May 2020, 21:22

z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic", pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")

Just curious, would there be any advantage to using YUV444P16 instead of YUV420P16 in this workflow?

Stereodude

29th May 2020, 22:39

Just curious, would there be any advantage to using YUV444P16 instead of YUV420P16 in this workflow?
You've got to get to 420 at some point, and this presumably is basically at the end of the processing flow aside from going to 10-bits for HEVC encode.

I'd argue it's best to only scale it once instead in of multiple steps.

ReinerSchweinlin

30th May 2020, 18:05

It was present in an early version and there was a help file about it, then they removed the feature end re-introduced it some versions ago - but the help file is missing now, I just looked up.. I will recover some of the old betas and search for it and post it here, give me some time, got work to do...
Here you go. It´s from an older version, so selecting new models isn´t included yet. I´ve asked Topaz for a newer version, waiting for a response.
BTW, they released a new beta with a new model yesterday.

ReinerSchweinlin

30th May 2020, 18:07

I'd argue it's best to only scale it once instead in of multiple steps.
I am not sure if we are talking about the same "steps", but in early versions of VEAI, sometimes processing the Video first with a 100% "cleanup" pass helped quite a lot.

JoelHruska

30th May 2020, 19:29

John,

For decades, the idea of "Enhance" -- as in, the recovery of detail levels over and above what was originally encoded in the source -- was a joke. Completely farcical. You're not wrong.

AI-based upscalers have turned science fiction into reality. There are different models that process the final image in different ways. Topaz Video Enhance AI is very much a work in progress.

If you want to see an example of the same video scene run through an upscaler without using YouTube as an intermediary, here:

This link is to samples Hello_Hello provided of some DVD clips I gave him:

https://www.sendspace.com/file/gwxxi0

This link is where you can download upscaled versions of those samples:

https://www.sendspace.com/filegroup/hAmG8UuXAfl3T2VT6e8pntkJJg0KsgtR

The SFE-1 and SFE-2 videos are upscaled using two different models -- Gaia-CG and Gaia-HQ, both available in Topaz Video Enhance AI.

You can see the significant level of improvement for yourself. You may not like it, of course, but you will not fail to see the difference.

Stereodude

30th May 2020, 21:01

I am not sure if we are talking about the same "steps", but in early versions of VEAI, sometimes processing the Video first with a 100% "cleanup" pass helped quite a lot.
I mean mean taking the output from Topaz and getting it into 2880x2160 YUV420. 444 to 420 requires scaling the chroma. You've also got to turn 3840x2160 into 2880x2160. It's probably best to do that in only 1 operation instead of compounding multiple separate steps that each can degrade the image.

zapp7

30th May 2020, 21:18

I mean mean taking the output from Topaz and getting it into 2880x2160 YUV420. 444 to 420 requires scaling the chroma. You've also got to turn 3840x2160 into 2880x2160. It's probably best to do that in only 1 operation instead of compounding multiple separate steps that each can degrade the image.

Here is my code to handle the output of 16 bit tiffs from Topaz VEAI. Should the TemporalDegrain2 come before or after the z_ConvertFormat step?

#outfolder is the topaz output directory, filecnt is # of images
ImageSource(file=outfolder+"\%06d.tiff",start=0,end=FileCnt-1,fps=23.976,pixel_type="RGB48")
ConvertToPlanarRGB()
z_ConvertFormat(width=2880, height=2160, resample_filter="bicubic",pixel_type="YUV420P16", colorspace_op="rgb:srgb:170m:f=>709:709:709:f")
TemporalDegrain2(grainLevel=false)
neo_f3kdb(range=31, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)

ReinerSchweinlin

30th May 2020, 21:45

ReinerSchweinlin

30th May 2020, 21:50

For decades, the idea of "Enhance" -- as in, the recovery of detail levels over and above what was originally encoded in the source -- was a joke. Completely farcical. You're not wrong.
I´d like to add that it is important to understand, that most upscaling techniques actually "make stuff up" - so what we war used to see in movies ("real" details recovered by som e magic) actually is a little different. There are some apporaches to get a little detail back (also see my other post), like this one:
https://github.com/jiangsutx/SPMC_VideoSR
https://www.youtube.com/watch?v=0WnwS1EOx3M

But these have to be differentiated from the "repainting" models...

JoelHruska

31st May 2020, 00:34

Stereodude

31st May 2020, 01:17

Ah ok, I get what you mean. VEAI internaly only can upscale 2x, 4x, 8x - so it´s probably best in terms of quality to not use "1080p" preset, but rather output 2x or 4x in 16Bit TIFF, then do one downscale/chroma/encode pass... Internaly, if a preset with a fixed resolution is set in VEAI, it uses ffmpeg to downscale from the AI Output.
Oh, I didn't realize that. They would definitely want to do a 4x or 8x then (depending on whether 1080p or UHD is the output target). Perhaps a sharper downsampling kernel that avoids ringing should be used with it. SSIM 2D in madVR looks very good, but I don't know if there's an Avisynth equivalent of that.

hello_hello

31st May 2020, 04:58

Well I'll confess after trying some upscaling with Avisynth, I'm now starting to wonder just how clever the clever upscaling is by comparison.

I took the original lossless h264 DVD samples and upscaled them to 4k in Avisynth. I had to do it using 2 processes and hobble x264 a little, otherwise my poor old XP machine would run out of memory, but here's the files in the zip file and how they were created etc.

SFE-1 nnedi3 4k rec.709.mkv
SFE-2 nnedi3 4k rec.709.mkv

Both upscaled using the following script. I forgot the original upscales weren't color converted so the colors are a little different, but that's not what the comparisons were about.
There may be better ways to sharpen than with LSFMod, but without a lot of experimenting, it's the only sharpening method I trust not to look horrible.
There's only a frame or two of live action in each sample, but it looks pretty obvious that using the method below would require the live secions to either be denoised first, or sharpened far less so the result isn't sharpened noise, however this was primarily to look at how nnedi3 would compare for the CGI.

MP_Pipeline("""
LoadPlugin("C:\Program Files\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\SFE-1.mkv.lwi")
AssumeTFF()
ColorMatrix(mode="Rec.601->Rec.709", clamp=0)
Crop(8,0,-8,0)
TFM(pp=5, micmatching=0).TDecimate()
QTGMC(InputType=1,TR2=3,Preset="Slower",ShutterBlur=3,ShutterAngleSrc=180,ShutterAngleOut=180,SBlurLimit=8)
LSFMod(Strength=200)
### prefetch: 16, 0
### ###
""")
nnedi3_rpow2(rfactor=8, cshift="Spline64Resize", fwidth=2880, fheight=2160)
LSFMod(Strength=200)

JoelHruska's upscales
SFE-1_4.00x_2560x1920_Gaia-CG.mp4
SFE-2_4.00x_2560x1920_Gaia-CG.mp4

The included screenshots are the same frame from SFE-1 (the nnedi3 screenshots are directly from the Avisynth output (no-re-encoding) so any advantage they have in terms of accuracy has now been declared), and the versions upscaled by JoelHruska are upscaled from my IVTC'd SD encodes of the original source, so that's their disadvantage declared too.

The screenshots with 4K in the name are the full upscaled frame. The screenshots not labelled 4k were taken using the PrintScreen button with MPC-HC only displaying 1080p worth of the frame on my 1080p monitor. That way the 1080p screenshots can be compared without the need for resizing on a 1080p monitor. Because I forgot about the color correction thing when encoding, the nnedi3 screenshots include a version that wasn't color converted to rec.709.

The samples upscaled by JoelHruska are also included because I was starting to get confused with the multiple sample uploads myself, so the nnedi3 and "Gaia-CG" upscales are both included.

The clever upscaling does clean up the line wobbling/aliasing more, but the rest, I'm not so sure it's better. Maybe there's a better anti-aliasing filter than QTGMC that could be used first, before upscaling with Avisynth?

nnedi3 comparison.zip (https://www.udrop.com/6u1/nnedi3_comparison.zip) (79.8 MB)

Katie Boundary

31st May 2020, 09:02

Is 3840x2160 really necessary?

No, and in fact it's the wrong aspect ratio.

zapp7

1st June 2020, 01:47

Oh, I didn't realize that. They would definitely want to do a 4x or 8x then (depending on whether 1080p or UHD is the output target). Perhaps a sharper downsampling kernel that avoids ringing should be used with it. SSIM 2D in madVR looks very good, but I don't know if there's an Avisynth equivalent of that.

It seems that VEAI can't actually upscale by 8x. The maximum it will accept for me is 6x. So it looks like to do 4K with this workflow, there is a minimum of 2 resampling steps to the VEAI output.

JoelHruska

1st June 2020, 02:57

Zapp7,

Are you going to crop it for 16:9 aspect ratios?

zapp7

1st June 2020, 04:09

Zapp7,

Are you going to crop it for 16:9 aspect ratios?

No, I'm sticking with the 4:3 ratio.

Stereodude

1st June 2020, 11:42

It seems that VEAI can't actually upscale by 8x. The maximum it will accept for me is 6x. So it looks like to do 4K with this workflow, there is a minimum of 2 resampling steps to the VEAI output.
I'm not sure I'm following you. 6x would give you 4320x2880 RGB 16-bit tiff files. You can turn that into 2880x2160 YUV420 in a single scaling step.

zapp7

1st June 2020, 16:41

I'm not sure I'm following you. 6x would give you 4320x2880 RGB 16-bit tiff files. You can turn that into 2880x2160 YUV420 in a single scaling step.

Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!

ReinerSchweinlin

1st June 2020, 22:03

Reiner mentioned up-thread that VEAI internally upscales only to 2x, 4x or 8x. Based on that it's my understanding that if I were to upscale by 6x, Topaz would internally upscale 8x and use ffmpeg to downscale to 6x. I would then have to downscale again to 2160p in a second resampling step.

Maybe I misunderstood and it actually can scale by 6x internally?

Also, I just found out that Topaz VEAI can be invoked from the command line. If invoked with -? or -h, it will show a list of all available arguments. I haven't tested it yet, but this looks promising for incorporating VEAI into a batch script!
I´d have to check if the newest model can do other than 2x, 4x, 8x scale factors - not sure at the moment. The gaia and artemis are "factor of 2".
CLI can be used for batches, but it´s incomplete at the moment, some stuff missing. It´s on the roadmap of TOPAZ, hopefully they´ll stick with it and clean this feature up.

ReinerSchweinlin

1st June 2020, 22:34

Reiner,

What would you call an approach like ESRGAN? According to the description of the model:

"how do we recover the finer texture details when we super-resolve at large upscaling factors? ... Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks."

They use phrases like "recover" as opposed to "approximate" or "repaint." Are they obfuscating their own approach, or are they one of the models that actually recovers data?
Joel, I´ll write a more thorrough text when I get back from a project, but I don´t want to leave without a short comment:
"recover" is a fuzzy term - ESRGAN actually is one of the "we make stuff up" methods (looks good, but is not original, but pleasing to the viewer (in case of video - more about this later)). There are some, which CAN reveal some detail by looking at multiple frames. It´s best - in my oppinion - to combine both (VEIA actually does combine both methods to a certain extend - but I´d love to see a selection on what to turn on or off)..
More when I come back..