Topaz Video Enhance AI

[ReinerSchweinlin]

Good morning.
As for the deinterlacing section, I noticed that all the presets (Dione) also operate a noise reduction and sharpness, is there any possibility to control this?
Thank you

No, the Dione Models have no tuning parameters.

If you add Avisynth prior to VEAI and let Avisynth do the deinterlacing, you can choose whatever Model you want - andproteus gives you the choice of how much you want to add/reduce sharpness or noise, halo, etc...

[toot367]

Good morning.
As for the deinterlacing section, I noticed that all the presets (Dione) also operate a noise reduction and sharpness, is there any possibility to control this?
Thank you

I haven't seen v2.6.2.

FYI, QTGMC is no longer the best deinterlacer for standard definition. Dione Interlaced Robust v3 and Dione Interlaced Robust Dehalo v1 WITH the json mods to disable NR and sharpening provides much more detail for many SD videos. QTGMC still has value for 1080i though.

[poisondeathray]

FYI, QTGMC is no longer the best deinterlacer for standard definition. Dione Interlaced Robust v3 and Dione Interlaced Robust Dehalo v1 WITH the json mods to disable NR and sharpening provides much more detail for many SD videos. QTGMC still has value for 1080i though.


How is the bob flicker with the json mods ?

QTGMC's main reason for existence (and it's precursor, tempgaussmc_beta) is to reduce bob flicker, not "keep details". In fact, as a result of QTGMC's temporal smoothing, noise and details are lost - as expected - ie. it's a tradeoff . You can adjust QTGMC settings to retain more details at the expense of more flicker

[toot367]

How is the bob flicker with the json mods ?

Are you referring to Dione screwing up the field order so that the frame jumps up or down for a frame?

If so, it's not perfect. I had to fix about 10 jumps on a 92 minute VHS movie. I duplicated the frame in Windows Explorer and used FillDrops(thresh=0.4) to fix the duplicate judder. When it finds a duplicate, FillDrops interpolates the extra frame and replaces it.

It's a nifty function for VHS, because you can fix many glitches by duplicating the previous or next frame. I duped 312 glitched frames in the movie and FillDrops fixed them all. Note I had VEAI save to a png sequence specifically to work directly with the frames using Windows Explorer, Paint, Photoshop, and Irfanview. I got rid of the microphone sneaking into one scene too. There's probably a more efficient method.

I tried some script to find bad frames, but it was useless (FindBadFrames it was called). It generated a file with "bad" frames but none of them were bad. So I had to find them with several viewings. I wish there were an automated way.

With DVD video, I haven't had any issues.

[poisondeathray]

How is the bob flicker with the json mods ?


Are you referring to Dione screwing up the field order so that the frame jumps up or down for a frame?


No, just referring regular deinterlacing artifacts.

Such as aliasing, jaggies, when viewed in motion or over a few frame, it flickers.

The main reason QTGMC exists is to calm that flicker (at the expense of smoothing, detail loss)

[toot367]

Such as aliasing, jaggies, when viewed in motion or over a few frame, it flickers.

There are no jagges, artifacts, or flickering.

[Emulgator]

And here after some TB rendered: a small VEAI ingest/encode guide

VEAI 2.6.4 Do's and Don'ts

Besides of VEAI 2.6.4 solving most of its main tasks superior
one has to consider how to get the most out of it and not lose quality due to some unexpected behaviour.

First off: The recent Topaz VEAI 2.6.4 uses ffmpeg under the hood for decoding and encoding,

Input: I can happily feed .avs scripts
but
Output: Unfortunately all ffmpeg encoders have been left at defaults !

Audio: 2.0 Audio in .mp4 oder .mov will be transcoded to 96kbps aac.
The resulting 96kbps .aac I do consider unusable, hi-Freq morphing and mumbling at the same time.

Solution: Demux source Audio to 16bit .aiff using external ffmpeg, and remux into .mov after Topas VEAI using external ffmpeg !
Some NLEs, (Vegas 14 here) seems to be able to decode only 16bit .aiff

Video: The same situation.
All encoders are left at defaults (ffmpeg x264 core 157 r2970)
No CABAC, only CRF can be set.

CRF10 @ 1980x1080x50p: 157Mbps 1,524bpp NoCABAC 1Ref, GOP M=1, N=12 0,25s/f -> almost transparent
CRF12 @ 1980x1080x50p: 125Mbps 1,208bpp NoCABAC 1Ref, GOP M=1, N=12 0,22s/f -> blocks starting to appear
CRF14 @ 1980x1080x50p: 91,2Mbps 0,879bpp NoCABAC 1Ref, GOP M=1, N=12 0,22s/f -> fat blockiness
CRF16 @ 1980x1080x50p: 74,1Mbps 0,715bpp NoCABAC 1Ref, GOP M=1, N=12 0,22s/f -> unbearable blockiness
CRF20 @ 1280x720x29.97p -> 7Mbps -> Blockfest !
CRF20 @ 1280x720x59.94p -> 8,6Mbps -> Blockfest !

Go to regedit
Computer\HKEY_CURRENT_USER\SOFTWARE\Topaz Labs LLC\Video Enhance AI
This contains only the same settings you can reach via GUI.
So encoder parameters are really the ffmpeg hardcoded defaults...

Who wants to harvest quality has to use ProRes...

ProRes 422 10bit HQ @ 1980x1080x50p: 387Mbps 3,734bpp -> clean !

++++++++ 4K-Uprezzing ++++++++++
8-bit YUV444P8 Uncompressed mean 24bpp, 3840x2160x50p yield pixel rate 414Mpix/s
and need for 24bpp 10Gb/s. YUV422P8 would be 16bpp, -> 6,63Gb/s

ProRes 422 HQ 10bit @ 3840x2160x50p: 1545Mbps, 3,734bpp (23,5% of YUV422P8) 0,37s/f -> clean, faultfree und detailed picture !
BUT: Now a 43s short 147MB .MTS 140MB .264 explodes 55-fold to 8GB!
Well, a second thought comes up: Such huge files.
Can't we quick-transcode/remux ?

A RTX3080 can munch 4K-H264 or 4K-H265 at 33..34fps, then MP4 muxing
(MeGUI oder Simple Launcher, 2022-06-22 can mux H265 in .mp4)

NVENcC265 at 4K:
CQP10: 391/618Mbps, 0,942bpp, 25% .mov size: Water gradients usable
CQP12: 315/497Mbps, 0,759bpp, 20% .mov size: Water gradients stepping
CQP15: 203/388Mbps, 0,49bpp, 12,5% .mov size: Water gradients get horrible steps

NVEncC264 at 4K:
CQP10: 406/637Mbps 0,98bpp, (27% .mov size, 6,5% YUV422P8 uncompressed): Water gradients almost faultless, better as H.265.

Unfortunately Vegas 14 can't decode all these varieties, and still there is a visible quality loss.
So for me it is back to huge ProRes apch into Vegas 14.
1TB/h, well, so lets invest in even more storage X-|

P.S. Not NVEncC's fault, just tested Rigaya's 6.01 from within StaxRip 2.13 with improved tuning and it fares much better qualitywise,
I get almost transparent 4K50p quality out of 45Mbps, so I guess again: VEAI 2.6.4. left NVEncC at defaults...

[Emulgator]

BTW, Topaz are hiring now.
https://topazlabs.com/careers/
Dallas, TX. Anyone willing to help them out with a simple ffmpeg interface ?
I am not affiliated, BTW.

Or even better: proper deinterlacing as developed in this almighty forum ? ;_)

P.S. Seems that upcoming version 3 gets a better encoding preset system.

[miltonsoto658]

Back to OP: Topaz VEAI 2.6.2 under Win10, i9-11900K 128GB, 3080RTX 16GB.
Downloaded all 186GB of models manually:
6432 files in C:\Users\<user>\AppData\Roaming\Topaz Labs LLC\Video Enhance AI\models
6 model families, each in 1..6 subversions with all their kernels, in 1..3 builds each.
If you don't preload everything VEAI will phone home for every model it sees fit and tries to get the next 30MB .tz file

Artemis: seems to be a one-stop shop for starters
Chronos: Good fps interpolation, see below
Dione: Deinterlacing, not mature yet I would say (knowing (Q)TGMC)
Gaia: one-in-all for CGI, I haven't tested yet.
Proteus: Fine tunable restoration for Pros, see below.
Theia: I haven't tested yet.

Used so far on Panasonic FT-3 encoder mud (1980x1080x25p on 25i)
Aiming to stay in 1920x1080, but with less camera artifacts.
Then maybe interpolate to 1920x1080x50p, maybe end in 1280x720x50p.

First decoding with DGDecode, then QTGMC for getting Chroma back onto Y, then into VEAI.
Proteus could repaint convincing seawater structure back, and resynthed details of hair and skin,
When overdone repeating patterns occurred, but not too bad !
Going back to source I started to hate the source more than the result.

Chronos 50p interpolation: knocks Mvtools2 out of the water, almost no morphing blotches.
In hard cases soft blends of details, but only visible while framestepping and pixelpeeping

Other source VHS on DV: Dione Deinterlacing: QTGMC still wins by far for now.
But with improving AI models almost everything seems within reach.

For me Proteus and Chronos are well worth it now and will stay in my daily toolbox.
Will not replace, but accompany Avisynth for the tasks where they shine.

Running stable here, Proteus V3 @ 4,5fps, Chronos V3 @ 6fps.
With such hard- and software a lot of projects will see a second AI edition now.

A few GUI unusual behaviours: Model Update management does list incomplete model sets only by filesize, not by files.
Hard to find out which is missing, you have to dump the complete set 3..6GB and try to download the complete set gain
Preview framecounter seems to be ready for input of framecount, but only renders the same amount 30 or 50.
One cannot save his preferences (why oh why do I have to input manually for every new clip)...

Reading up their forums it seemed to be much worse in the beginning
and from the responses it looks like only one developer responding, but still:
he answers, and there is definitely progress in GUI usability and I guess in number and usability of models too.

Output: Avoid .mov (ProRes 422HQ) for now, I got small blocks all over the sky.
If output avc in .mp4, give enough bits (I felt safe with CQ12)
and add grain (I used 1,8) otherwise you get blocks again.
Any fps model loses audio, even if duration stays.
Uncompressed .avi or VfW encoder interface: not yet implemented
.tiff frames ask for huge storage, but this is well known.

P.S. Just started redoing a paused film restoration from 720x576x25p DVD 5Mbps.
Proteus Dehalo: never expected that this can be achieved.
Clean transitions in 1280x720 and 1920x1080 while almost undisturbed by the source's blockfest.
I can happily trash my previous Dehaloing attempts now.

Just tried the unspeakable just to saturate the rendering system: Run 2 instances of VEAI in parallel. Works !


Can i ask how did you download manually all the models?

[Emulgator]

Well, manually was not the correct term.
It was via GUI, repeatedly invoking and deleting incomplete downloads over the course of a few days.

[rgr]

25fps->50fps conversion + image enhancement (from youtube).

First Chronos, then Artemis Low Quality -- or vice versa?

[Emulgator]

Might be source dependent.
I go vice versa, first enhance/upscale then upfps.
Takes longer, but gives... better butter.

[James_b]

Emulgator, what do you think about a madVR-plugin? would be cool. Found some screenshots from huh'n https://forum.doom9.org/showthread.php?p=1967409#post1967409

[Emulgator]

ATM I see VEAI and other AI algos rather as a choice of offline reconstruction algos.
Once AI gets so cheap that it an be performanced realtime, maybe.
But I think that guessing at content (to such extent and in effect repainting it) is not what a renderer should do.
A few years later my today's opinion may well be overridden by development ;-)

To continue Topaz discussion:
I can not get any enlightening out of Version 3 so far up to 3.0.4:
Slower, clumsier behaviour,
(continues rendering costly preview threads under the hood, every time I have to untick something),
the two screens are not updated in realtime but stutter around (worked in 2.6.4),
still Deinterlacing can not do what I am inclined to believe it could,
"Ugly face syndrome" still unsolved
(while calcuating Proteus on them, if faces happen to be below a certain pixel size they degrade to ugly grimaces).

I haven't even tried to get the output options tested for better parameters.
And from this month on I would have to shell out the yearly price again to be updated...
So it is 2.6.4 for me still.

[ognirats]

There are no jagges, artifacts, or flickering.

Doubt

[James_b]

ATM I see VEAI and other AI algos rather as a choice of offline reconstruction algos.
Once AI gets so cheap that it an be performanced realtime, maybe.
But I think that guessing at content (to such extent and in effect repainting it) is not what a renderer should do.
A few years later my today's opinion may well be overridden by development ;-)


Hello Topaz master :)
I meant the other way around, mad vir in Topaz. super-xbr is pretty good. In Topaz one can not change the upscaler? as far as i concern. Only the clean up algos. Do you know what they use?

So back to the 2.6.4. And when speaking of 264, the x264-out have some what shallow dynamics, or is it only me? kicks of from 30.

And when playing some of my old avi i get 0-255 dynamics in many players, almost HDR (in dynamics). But the very good ProRes output from 16 black. Standard yes, but dynamics is not so much Enhance.

[Emulgator]

the x264-out have some what shallow dynamics, or is it only me? kicks of from 30.
Yes, depending on footage shadow values tended to brighten up a bit.
Could well have been a straight mismatch implemented while piping into ffmpeg x264.
Back then in a good number of cases I inserted a "lowering shadow values" gamma curve after Topaz (SmoothAdjust16),
but these days I am telling 2.6.4 to render into ProRes to lift further processing chain into 10bit precision.

super-xbr: I haven't tried seriously any other than VEAI after I got ridiculous and still watchable 450%..900% 4K upscales working, just for fun. But maybe I should someday...