Log in

View Full Version : vs-basicvsr


Pages : [1] 2 3 4 5

poisondeathray
25th August 2021, 03:38
Yet another contribution from HolyWu!

Thanks

https://github.com/HolyWu/vs-basicvsr

This one is true temporal SR for video, not single images

Also has tiling option for low GPU memory vs. task

Nice!

Zarxrax
25th August 2021, 22:12
What is this difference between vs-basicvsr, vs-realesrgan, and VSGAN?
As far as I can see, it looks like vs-basicvsr & vs-realesrgan have built-in models that you can use, wheras VSGAN lets you just use whatever model you want.
Is there something else that I am missing?

poisondeathray
25th August 2021, 23:38
What is this difference between vs-basicvsr, vs-realesrgan, and VSGAN?
As far as I can see, it looks like vs-basicvsr & vs-realesrgan have built-in models that you can use, wheras VSGAN lets you just use whatever model you want.
Is there something else that I am missing?

Yes,

other differences - vsgan implementation of ESRGAN does not have tiling (image is split up with padding and stitched up back togethe) - this helps when your GPU has low memory compared to image size; some other ESRGAN implementations have tiling. Quality wise you cannot tell the difference with large enough padding even zoomed in on still images - although it is detectable with amplified differences.

basicvsr is temporal - in general there is less aliasing and temporal flicker. It's meant for video and works on more than 1 frame. The others are meant for still images (except tecogan, but that does not have a VPY implementation). Single image based SR has less temporal consistency. Unfortunately I'm finding the 3 models provided for basicvsr generally only work for fairly clean sources. Just like with other GAN's you generally have to pre and post process to "massage" the input for a suitable model. It would be nice if you had a specific model for everything and thousands of different situations - I don't see it happening

But users can train their own models for basicvsr and realesrgan - the training code is published - but there isn't a public repository for them - only the published paper models are readily available. In theory you should be able to use other pre-trained models too, with minor adjustment to the script

aegisofrime
26th August 2021, 02:17
HolyWu is on a roll, holy molly.

Hoping to see VS implementations of AMD FSR or Intel XeSS next :D

kedautinh12
26th August 2021, 02:25
HolyWu is on a roll, holy molly.

Hoping to see VS implementations of AMD FSR or Intel XeSS next :D

I think Nvidia dlss is better but sad it's close source

aegisofrime
26th August 2021, 02:46
I think Nvidia dlss is better but sad it's close source

Yeah, that was the reason I didn't mention DLSS, unless nvidia decides to open source it due to competitive pressure. :D

edcrfv94
26th August 2021, 07:49
Yeah, that was the reason I didn't mention DLSS, unless nvidia decides to open source it due to competitive pressure. :D

DLSS need motion vector.

kedautinh12
26th August 2021, 07:53
DLSS need motion vector.

Yes, DLSS only for RTX card

Quadratic
26th August 2021, 12:04
HolyWu is on a roll, holy molly.

Hoping to see VS implementations of AMD FSR or Intel XeSS next :D

https://github.com/GPUOpen-Effects/FidelityFX-FSR/blob/master/ffx-fsr/ffx_fsr1.h
AMD FSR (EASU) is quite interesting, I would imagine one would probably be able to approximate the results with existing tools - but an actual implementation would be nice.

Julek
26th August 2021, 15:11
DLSS even if open would not be easy to port, the models are trained specifically for each game, there is no generic model.
But if you have an RTX card, you can at least use DLISR, via the Akarin plugin.
https://github.com/AkarinVS/vapoursynth-plugin#dlisr

aegisofrime
26th August 2021, 15:17
DLSS even if open would not be easy to port, the models are trained specifically for each game, there is no generic model.
But if you have an RTX card, you can at least use DLISR, via the Akarin plugin.
https://github.com/AkarinVS/vapoursynth-plugin#dlisr

OMG, I have been looking for a DLSS VS plugin for ages now, but I guess I either didn't bother Googling for it or my Google skills suck. Thanks for linking this! I wonder how many other plugins lie under my nose...

Selur
26th August 2021, 17:56
vs-basicvsr
Yet another contribution from HolyWu!
Nice, and there goes the weekend,.. ;)

edcrfv94
26th August 2021, 20:25
DLSS even if open would not be easy to port, the models are trained specifically for each game, there is no generic model.
But if you have an RTX card, you can at least use DLISR, via the Akarin plugin.
https://github.com/AkarinVS/vapoursynth-plugin#dlisr

DLSS 2.0 is generic model, but still need motion vector.
This means it cannot be used for video.

aegisofrime
27th August 2021, 02:10
DLSS 2.0 is generic model, but still need motion vector.
This means it cannot be used for video.

According to the plugin description, it is using NGX though which is indeed designed for video applications.

kedautinh12
27th August 2021, 02:48
According to the plugin description, it is using NGX though which is indeed designed for video applications.

I think DLSS don't same DLISR

Selur
27th August 2021, 21:06
Btw. has anyone an example where BasicVSR show some impressive results when starting from a 480p or 576p source?
Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.
Only thing that I found is that it does do well is upscaling QCIF content (https://gleitz.info/forum/index.php?thread/48279-super-resolution-erfahrungen/&postID=465168#post465168), and I wonder whether that is it's goal.

Cu Selur

ChaosKing
27th August 2021, 21:20
@Selur I think the link is broken.

Selur
27th August 2021, 22:00
Sorry, fixed now.

kedautinh12
28th August 2021, 00:38
The link work for me but almost German

Reel.Deel
28th August 2021, 03:06
The link work for me but almost German

That is because is a German forum :p

Selur
28th August 2021, 06:28
The interessting part are the screenshots, so the language shouldn't be an issue. Not much text so, Google Translate or similar should work too. ;)

Cu Selur

poisondeathray
28th August 2021, 14:10
Btw. has anyone an example where BasicVSR show some impressive results when starting from a 480p or 576p source?
Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.
Only thing that I found is that it does do well is upscaling QCIF content (https://gleitz.info/forum/index.php?thread/48279-super-resolution-erfahrungen/&postID=465168#post465168), and I wonder whether that is it's goal.



BasicVSR works ok in simular situations to where the ESRGAN base model works ok - ie. fairly clean, downsampled sources - really an oversampled SD source... Which is NOT what people typically have - it's usually some noisy, soft DVD source, or SD crappy home video that people want to upscale. But those clean datasets are what most publically available algorithms were trained on: Clean HR ground truth source, downscaled by 4. Usually with bicubic. BasicVSR used selections from the Reds4 and Vimeo90k datasets. qcif "suzie" is really an oversampled source. If you take some ok quality UHD, and downscale it, that will work too. What's needed is training of specific models that take account for noise, degredation, compression artifacts like mpeg2 artifacts, etc... stuff like bad YT compression and degredation after the downscaling step. There are specific datasets that include noise - but they are synthetic noise - and they weren't used for these default models .

So in those situations where the model is applicable - BasicVSR is going to produce more detailed images than , say lanczos3 + any form of sharpen, but less detailed results than default ESRGAN or ESRGAN+ models on single image examination. On single image, ESRGAN+ is going to look more "wow" in applicable source situations. The main difference is BasicVSR is more consistent temporally, when you look at actual video, or more than 1 image. It resembles tecogan in that regard, but produces better results detail wise than the default tecogan model (which isn't the same as the tecogan published paper model). For all single image SR approaches you often need to use some type of temporal smoothing to make it decent for video purposes (and that smoothing reduces the fine details too) , otherwise you are prone to flickering aliasing artifacts, objects changing shapes (as single image prediction is slightly different on different frames) .

So the question is can you use something like ESRGAN, add some QTGMC in progressive mode - to get similar temporally consistent results to BasicVSR, but hopefully keep more fine details? The answer is generally no. The artifacts are too "baked" in - that's the strength of basicvsr

poisondeathray
28th August 2021, 17:52
Some apng comparisons. They should animate and loop in most browsers

1:1 non-AR corrected crops, unfiltered. 720x480 => 2880x1920.

Note the temporal flickering artifacts (aliasing flicker, and prediction flicker).

You can see the earrings look like random lightning in the ESRGAN+ results.

https://i.postimg.cc/PxD42rQR/shimmer-compare-apng.png (https://postimages.org/)

No amount of sharpening will transform lanczos3 into as detailed as BasicVSR. No amount of temporal filtering will transform lanczos3, or ESRGAN+ results into as temporally consistent results as BasicVSR

https://i.postimg.cc/FsNZVvdT/stillface-compare-apng.png (https://postimages.org/)


The license plate, you can make out the numbers/letters with lanczos, but it's much more clear in the BasicVSR. It doesn't take a stretch of the imagination that slightly smaller or a more distant plate might be illegible when processed with lanczos , but partially readable by BasicVSR

https://i.postimg.cc/qgVxPfjv/compare-licenseplate-apng.png (https://postimages.org/)

ESRGAN and psnr model mess up letters/numbers/text (many GAN, NN models do), I won't include in the animated temporal demo, but here is a still image compare animated demo. ESRGAN default does good on BG foliage textures, but messes up text. None of the GAN scalers have "fuzzy" lines or bad aliasing - that's what they excel at. Early gen NNEDI/1/2/3 is NN (neural net) based, but signficantly worse in terms of aliasing artifacts for upscaling 4x than modern GAN scalers


https://i.postimg.cc/mr4BbVmX/stillcompare-licenseplate-apng.png (https://postimages.org/)


But these are cleanly downsample source examples, much like the training set. ie. They don't have MPEG2 macroblocks or or grain plate added on top like you might with some film DVD

Some trained degraded jpeg ESRGAN models work ok for single image jpeg interpolation - I would like to see more BasicVSR models that utilize some typical degraded aspects of common video

lansing
28th August 2021, 19:49
The car plate image is amazing

poisondeathray
28th August 2021, 19:55
Keep in mind those are "textbook" clean sources . Normal dashcam / security cam footage would have a bunch of data loss, compression artifacts. The training set for the 3 base models do not account for that

There is an improved version, BasicVSR++ , but the code has not been released yet . I hope HolyWu brings it to vapoursynth when it's released

Selur
28th August 2021, 22:55
@poisondeathray: Thanks for the explaination, that explains why I didn't get and impressive results from my usual content. :)

Selur
28th August 2021, 23:51
There is an improved version, BasicVSR++ ,..
Looks impressive: https://ckkelvinchan.github.io/projects/BasicVSR++/

I hope HolyWu brings it to vapoursynth when it's released
fingers crossed ;)

Cu Selur

edcrfv94
29th August 2021, 02:27
Keep in mind those are "textbook" clean sources . Normal dashcam / security cam footage would have a bunch of data loss, compression artifacts. The training set for the 3 base models do not account for that

There is an improved version, BasicVSR++ , but the code has not been released yet . I hope HolyWu brings it to vapoursynth when it's released

released 10 days ago
https://github.com/open-mmlab/mmediting/blob/master/mmedit/models/backbones/sr_backbones/basicvsr_pp.py

MMEditing v0.10.0 Release
Support LIIF-RDN (CVPR'2021)
Support BasicVSR++ (NTIRE'2021)
https://github.com/open-mmlab/mmediting/releases/tag/v0.10.0

poisondeathray
29th August 2021, 03:01
released 10 days ago
https://github.com/open-mmlab/mmediting/blob/master/mmedit/models/backbones/sr_backbones/basicvsr_pp.py

MMEditing v0.10.0 Release
Support LIIF-RDN (CVPR'2021)
Support BasicVSR++ (NTIRE'2021)
https://github.com/open-mmlab/mmediting/releases/tag/v0.10.0

Nice!! :D

aegisofrime
30th August 2021, 04:39
@poisondeathray: Could I ask what sort of speed are you getting in your test? Upscaling just from 720x480 -> 1280x720 I'm getting like 0.05fps :scared: with a Ryzen 5950X + RTX 3060Ti, and I'm wondering if I'm doing something wrong.

poisondeathray
30th August 2021, 14:09
@poisondeathray: Could I ask what sort of speed are you getting in your test? Upscaling just from 720x480 -> 1280x720 I'm getting like 0.05fps :scared: with a Ryzen 5950X + RTX 3060Ti, and I'm wondering if I'm doing something wrong.

Similar speeds - it's very slow.

gugglu
1st September 2021, 11:21
Hi folks, i am trying to test the vsbasicVSR but whenever i try to load the BasicVSR it says cuda not available. any hint what i am missing, tia.

script using to testimport vapoursynth as vs
import numpy
import torch
from vsbasicvsr import BasicVSR

core = vs.get_core()

src = core.dgdecodenv.DGSource(r'C:\Users\User89\Desktop\test\00000.dgi')
src = core.fmtc.bitdepth(src, bits=32)
src = core.fmtc.resample(src, kernel='spline36', css='444')
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)
ret = BasicVSR(src)

ret.set_output()

I get this error when checking the above scriptTraceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2242, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 2243, in vapoursynth.vpy_evaluateScript
File "C:/Users/User89/Desktop/VapourSynthEditor-r19-64bit/Untitled.vpy", line 12, in
src = core.fmtc.matrix(src, mat='709', col_fam=vs.RGB)
File "C:\Users\User89\AppData\Local\Programs\Python\Python39\lib\site-packages\vsbasicvsr\__init__.py", line 56, in BasicVSR
raise vs.Error('BasicVSR: CUDA is not available')
vapoursynth.Error: BasicVSR: CUDA is not available

poisondeathray
1st September 2021, 15:58
vapoursynth.Error: BasicVSR: CUDA is not available

Drivers ?

Did you install pytorch with cuda ?

Does cuda work ok for you in other applications, or vapoursynth ? eg. try eedi2cuda
https://github.com/AmusementClub/VapourSynth-EEDI2CUDA

gugglu
1st September 2021, 17:16
Drivers ?

Did you install pytorch with cuda ?

Does cuda work ok for you in other applications, or vapoursynth ? eg. try eedi2cuda
https://github.com/AmusementClub/VapourSynth-EEDI2CUDA

Hi poisondeathray, i've installed cuda (cuda_11.0.2_451.48_win10) but on its own here are some scrn.shots of process please have a look.

Cuda installation:
https://prnt.sc/1qyobff
https://prnt.sc/1qyocvh
https://prnt.sc/1qyodye
https://prnt.sc/1qyp184
https://prnt.sc/1qyp3ly
installed Drivers:
https://prnt.sc/1qyp9ae
and pytorch was installed with vsbasicvsr via pip as instructed on a github .
https://prnt.sc/1qyrk22
yes EEDi2CUDA works fine
https://prnt.sc/1qyrd78

:thanks:

poisondeathray
1st September 2021, 18:06
H
and pytorch was installed with vsbasicvsr via pip as instructed on a github .
https://prnt.sc/1qyrk22



You're missing pytorch with cuda . You only have pytorch cpu

There should be a folder torch-xxx-cuxxx

Where "xxx" is the version number, cu is for cuda

If you go here, and select the boxes that apply to you, it will give you command line to install correct version with cuda
https://pytorch.org/get-started/locally/

Selur
1st September 2021, 18:39
Also, if you want I can send you a link to the 'addon' for Hybrid I just uploaded, it's basically a portable Vapoursynth&Python install with:

vsbasicvsr (https://github.com/HolyWu/vs-basicvsr)
vsdpir (https://github.com/HolyWu/vs-dpir)
vsffdnet (https://github.com/HolyWu/vs-ffdnet)
vsgan (https://github.com/rlaPHOENiX/VSGAN) + a bunch of vsgan_models (https://upscale.wiki/wiki/Model_Database)
vsrealesgran (https://github.com/HolyWu/vs-realesrgan/)
vsrife (https://github.com/HolyWu/vs-rife)

packed size ~9GB
if you want a link to the download send me a pm.

Cu Selur

gugglu
1st September 2021, 20:12
You're missing pytorch with cuda . You only have pytorch cpu

There should be a folder torch-xxx-cuxxx

Where "xxx" is the version number, cu is for cuda

If you go here, and select the boxes that apply to you, it will give you command line to install correct version with cuda
https://pytorch.org/get-started/locally/

thanks a lot poisondeathray its working after downloading pytorch with cuda with windows command line from a pytorch link

Microsoft Windows [Version 10.0.19042.1165]
(c) Microsoft Corporation. All rights reserved.

C:\Users\User89\AppData\Local\Programs\Python\Python39\Scripts>pip3 install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.9.0+cu102
Downloading https://download.pytorch.org/whl/cu102/torch-1.9.0%2Bcu102-cp39-cp39-win_amd64.whl (1440.4 MB)
|████████████████████████████████| 1440.4 MB 2.8 kB/s
Collecting torchvision==0.10.0+cu102
Downloading https://download.pytorch.org/whl/cu102/torchvision-0.10.0%2Bcu102-cp39-cp39-win_amd64.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 3.3 MB/s
Collecting torchaudio===0.9.0
Using cached torchaudio-0.9.0-cp39-cp39-win_amd64.whl (215 kB)
Requirement already satisfied: typing-extensions in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torch==1.9.0+cu102) (3.10.0.2)
Requirement already satisfied: pillow>=5.3.0 in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torchvision==0.10.0+cu102) (8.3.1)
Requirement already satisfied: numpy in c:\users\user89\appdata\local\programs\python\python39\lib\site-packages (from torchvision==0.10.0+cu102) (1.21.2)
Installing collected packages: torch, torchvision, torchaudio
Attempting uninstall: torch
Found existing installation: torch 1.9.0
Uninstalling torch-1.9.0:
Successfully uninstalled torch-1.9.0
Attempting uninstall: torchvision
Found existing installation: torchvision 0.10.0
Uninstalling torchvision-0.10.0:
Successfully uninstalled torchvision-0.10.0
Successfully installed torch-1.9.0+cu102 torchaudio-0.9.0 torchvision-0.10.0+cu102

gugglu
1st September 2021, 20:20
Also, if you want I can send you a link to the 'addon' for Hybrid I just uploaded, it's basically a portable Vapoursynth&Python install with:

vsbasicvsr (https://github.com/HolyWu/vs-basicvsr)
vsdpir (https://github.com/HolyWu/vs-dpir)
vsffdnet (https://github.com/HolyWu/vs-ffdnet)
vsgan (https://github.com/rlaPHOENiX/VSGAN) + a bunch of vsgan_models (https://upscale.wiki/wiki/Model_Database)
vsrealesgran (https://github.com/HolyWu/vs-realesrgan/)
vsrife (https://github.com/HolyWu/vs-rife)

packed size ~9GB
if you want a link to the download send me a pm.

Cu Selur

that seems quite interesting, Selur thank you :)

poisondeathray
4th September 2021, 20:15
:D

Thanks HolyWu

vs-BasicVSR++
https://github.com/HolyWu/vs-basicvsrpp

Selur
5th September 2021, 09:43
nice!

Support either x4 upsampling (for model 0-2) or same size output (for model 3-5).
For model 0-2, input resolution must be at least 64.
For model 3-5, input resolution must be at least 256 and mod-4.
Parameters:
clip: Clip to process. Only planar format with float sample type of 32 bit depth is supported.
model: Model to use.
0 = REDS
1 = Vimeo-90K (BI)
2 = Vimeo-90K (BD)
3 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 1
4 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 2
5 = NTIRE 2021 Quality enhancement of heavily compressed videos Challenge - Track 3

Is there a difference between BasicVSR and BasicVSR++ if model 0-2 are used, or is it the same resizing as BasicVSR (model 0-2) and additional models for cleaning?

Selur
5th September 2021, 10:52
Also did anyone manage to install this in a portable Vapoursynth environment on Windows?
Calling:
python -m pip install --upgrade vsbasicvsrpp
first failed with
ERROR: Could not find a version that satisfies the requirement vapoursynth==54 (from versions: 39, 40, 41, 42, 43, 44, 45, 46, 47, 47.1, 47.2, 48, 49, 50, 51)
ERROR: No matching distribution found for vapoursynth==54
after renaming the dummy 'VapourSynth-53.dist-info', I created to install VSGAN, to 'VapourSynth-54.dist-info', calling:
python -m pip install --upgrade vsbasicvsrpp
failed with:
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
calling:
set CUDA_HOME=I:/Hybrid/64bit/Vapoursynth/Lib/site-packages/torch/cuda (not toally sure this is correct)
and then:
python -m pip install --upgrade vsbasicvsrpp
it fails with:
I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
I get the same error when calling:
python -m pip install mmcv-full==1.3.12 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.htm

-> that's the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know. :)

Cu Selur

poisondeathray
5th September 2021, 14:37
Is there a difference between BasicVSR and BasicVSR++ if model 0-2 are used, or is it the same resizing as BasicVSR (model 0-2) and additional models for cleaning?

I've only done a few tests so far , but some early observations/comments - basicvsrpp is marginally better with the same model interval size compared to basicvsr. Not a major difference. The default interval size is different, 30 for ++, vs 7*2+1=15

Models 3-5 are from the NTIRE 2021 Quality enhancement of heavily compressed videos Challenge , which take HEVC compressed videos using fixed qp and low bitrate encodings - so those pre-trained models should factor in some compression degredation (at least HEVC type, not necessarily MPEG2, or AVC). It' s nice to see some other types of degradation training and models, but 3 and 5 tend to be very smooth (ie. no detail) . 4 has more detail but more artifacts. Models 3-5 don't upscale

I haven't done enough testing to see if using a much larger interval size helps or hinders in general. It appears a very small interval size is worse. Larger sizes take more memory and are slower

poisondeathray
5th September 2021, 14:51
-> that's the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know. :)




I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
warnings.warn(f'Error checking compiler version for {compiler}: {error}')


Not sure, I used installed environment, but I had problems at first. My errors msg was slightly different - it needed MS Visual C++ compiler to build the "wheels" to install other components. I'm wondering how the compiler is accessed in a "portable" environment ?

Selur
5th September 2021, 15:29
I'm wondering how the compiler is accessed in a "portable" environment ?
No clue either, haven't run into the problem before. :)

Cu Selur

poisondeathray
5th September 2021, 16:25
HolyWu added update a few hours ago and made install "easier" on Windows. Maybe try this new one
https://github.com/HolyWu/vs-basicvsrpp


Installing mmcv-full on Windows is a bit complicated as it requires Visual Studio and other tools to compile CUDA ops. So I have uploaded the built file compiled with CUDA 11.1 for Windows users and you can install it by executing the following command.


pip install https://github.com/HolyWu/vs-basicvsrpp/releases/download/v1.0.0/mmcv_full-1.3.12-cp39-cp39-win_amd64.whl

Selur
5th September 2021, 20:49
Thanks ! using that call it works for me too. :)

Cu Selur

Selur
5th November 2021, 20:42
Has anyone tried https://github.com/HolyWu/vs-swinir ? (didn't want to create a new thread ;))
-> man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)

PatchWorKs
6th November 2021, 08:11
Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.
HolyWu has just ported - after my request - SwinIR (https://github.com/JingyunLiang/SwinIR) to VS, can someone make some "real world" test with it ?

https://github.com/HolyWu/vs-swinir

https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/real_world_image_sr.png

man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)
For individual frames (= pics) you can test it here (https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb), but a video-oriented colab notebook like this (https://colab.research.google.com/github/rlaPHOENiX/VSGAN/blob/master/VSGAN.ipynb) would be great (I don't own a discrete GPU at all) !


EDIT
Out of curiosity: do you think the new Apple chips (M1 Pro / Max (https://www.anandtech.com/show/17024/apple-m1-max-performance-review)) could speed up operations?

Selur
6th November 2021, 13:34
Out of curiosity: do you think the new Apple chips (M1 Pro / Max) could speed up operations?
Without:
a. pytorch support
b. rewriting of the exitistn plugins
-> no

For individual frames (= pics)
I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.

ReinerSchweinlin
7th November 2021, 19:54
I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.
thanx for testing :) How much vram did it use in your case? Is it really that GPU demanding or might the slow-down caused by not enough VRAM ? If you have something a unskilled person like me could use and test, I could throw it into a 12GB VRAM card and see what happens...