Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th September 2021, 10:52   #41  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Also did anyone manage to install this in a portable Vapoursynth environment on Windows?
Calling:
Code:
python -m pip install --upgrade vsbasicvsrpp
first failed with
Code:
ERROR: Could not find a version that satisfies the requirement vapoursynth==54 (from versions: 39, 40, 41, 42, 43, 44, 45, 46, 47, 47.1, 47.2, 48, 49, 50, 51)
ERROR: No matching distribution found for vapoursynth==54
after renaming the dummy 'VapourSynth-53.dist-info', I created to install VSGAN, to 'VapourSynth-54.dist-info', calling:
Code:
python -m pip install --upgrade vsbasicvsrpp
failed with:
Code:
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
calling:
Code:
set CUDA_HOME=I:/Hybrid/64bit/Vapoursynth/Lib/site-packages/torch/cuda
(not toally sure this is correct)
and then:
Code:
python -m pip install --upgrade vsbasicvsrpp
it fails with:
Code:
I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
      warnings.warn(f'Error checking compiler version for {compiler}: {error}')
I get the same error when calling:
Code:
python -m pip install mmcv-full==1.3.12 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.9.0/index.htm
-> that's the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know.

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..

Last edited by Selur; 5th September 2021 at 11:05.
Selur is offline   Reply With Quote
Old 5th September 2021, 14:37   #42  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,966
Quote:
Originally Posted by Selur View Post
Is there a difference between BasicVSR and BasicVSR++ if model 0-2 are used, or is it the same resizing as BasicVSR (model 0-2) and additional models for cleaning?
I've only done a few tests so far , but some early observations/comments - basicvsrpp is marginally better with the same model interval size compared to basicvsr. Not a major difference. The default interval size is different, 30 for ++, vs 7*2+1=15

Models 3-5 are from the NTIRE 2021 Quality enhancement of heavily compressed videos Challenge , which take HEVC compressed videos using fixed qp and low bitrate encodings - so those pre-trained models should factor in some compression degredation (at least HEVC type, not necessarily MPEG2, or AVC). It' s nice to see some other types of degradation training and models, but 3 and 5 tend to be very smooth (ie. no detail) . 4 has more detail but more artifacts. Models 3-5 don't upscale

I haven't done enough testing to see if using a much larger interval size helps or hinders in general. It appears a very small interval size is worse. Larger sizes take more memory and are slower
poisondeathray is offline   Reply With Quote
Old 5th September 2021, 14:51   #43  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,966
Quote:
Originally Posted by Selur View Post
-> that's the point where I gave up, so if anyone figures out how to install vsbasicvsrpp in a protable Vapoursynth environment please let me know.
Quote:
I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden
warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Not sure, I used installed environment, but I had problems at first. My errors msg was slightly different - it needed MS Visual C++ compiler to build the "wheels" to install other components. I'm wondering how the compiler is accessed in a "portable" environment ?
poisondeathray is offline   Reply With Quote
Old 5th September 2021, 15:29   #44  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Quote:
I'm wondering how the compiler is accessed in a "portable" environment ?
No clue either, haven't run into the problem before.

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 5th September 2021, 16:25   #45  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,966
HolyWu added update a few hours ago and made install "easier" on Windows. Maybe try this new one
https://github.com/HolyWu/vs-basicvsrpp

Quote:
Installing mmcv-full on Windows is a bit complicated as it requires Visual Studio and other tools to compile CUDA ops. So I have uploaded the built file compiled with CUDA 11.1 for Windows users and you can install it by executing the following command.

Code:
pip install https://github.com/HolyWu/vs-basicvsrpp/releases/download/v1.0.0/mmcv_full-1.3.12-cp39-cp39-win_amd64.whl
poisondeathray is offline   Reply With Quote
Old 5th September 2021, 20:49   #46  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Thanks ! using that call it works for me too.

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 5th November 2021, 20:42   #47  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Has anyone tried https://github.com/HolyWu/vs-swinir ? (didn't want to create a new thread )
-> man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..

Last edited by Selur; 5th November 2021 at 22:21.
Selur is offline   Reply With Quote
Old 6th November 2021, 08:11   #48  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 261
Quote:
Originally Posted by Selur View Post
Normal resizing using i.e. Lanczos and adding some contrast sharpening seems to produce more impressive results.
HolyWu has just ported - after my request - SwinIR to VS, can someone make some "real world" test with it ?

https://github.com/HolyWu/vs-swinir



Quote:
Originally Posted by Selur View Post
man this is too slow on my machine to be useful for normal usage on my gpu (Geforce GTX 1070ti)
For individual frames (= pics) you can test it here, but a video-oriented colab notebook like this would be great (I don't own a discrete GPU at all) !


EDIT
Out of curiosity: do you think the new Apple chips (M1 Pro / Max) could speed up operations?

Last edited by PatchWorKs; 6th November 2021 at 08:50.
PatchWorKs is offline   Reply With Quote
Old 6th November 2021, 13:34   #49  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Quote:
Out of curiosity: do you think the new Apple chips (M1 Pro / Max) could speed up operations?
Without:
a. pytorch support
b. rewriting of the exitistn plugins
-> no

Quote:
For individual frames (= pics)
I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 7th November 2021, 19:54   #50  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 403
Quote:
Originally Posted by Selur View Post
I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.
thanx for testing How much vram did it use in your case? Is it really that GPU demanding or might the slow-down caused by not enough VRAM ? If you have something a unskilled person like me could use and test, I could throw it into a 12GB VRAM card and see what happens...
ReinerSchweinlin is offline   Reply With Quote
Old 8th November 2021, 09:49   #51  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 261
Quote:
Originally Posted by Selur View Post
Without:
a. pytorch support
b. rewriting of the exitistn plugins
-> no
Well, pytorch support SEEMS on the go:
https://github.com/pytorch/pytorch/issues/47702


Quote:
Originally Posted by Selur View Post
I can run it for single pics fine, but I get like 0.005fps for sd->hd on my system, which simply is too slow for me to be usable.
Of course (that's why a colab can help), but can you please post some visual results ?

Thx !
PatchWorKs is offline   Reply With Quote
Old 8th November 2021, 18:32   #52  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
here are a few examples:

used:
Code:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'E:\clips\VTS_02_1-Sample-Beginning.demuxed.m2v'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading E:\clips\VTS_02_1-Sample-Beginning.demuxed.m2v using D2VSource
clip = core.d2v.Source(input="E:/Temp/m2v_5d36292e1f7f53fd6e26be51d50bbf8c_853323747.d2v")
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# DEBUG: vsTIVTC changed scanorder to: progressive
# cropping the video to 704x480
clip = core.std.CropRel(clip=clip, left=8, right=8, top=0, bottom=0)
from vsswinir import SwinIR
# adjusting color space from YUV420P8 to RGBS for VsSwinIR
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# resizing using SwinIR
clip = SwinIR(clip=clip, task="real_sr_large", scale=4, tile_x=352, tile_y=240, tile_pad=16, device_type="cuda", device_index=0) # 2816x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1920, h=1474, kernel="lanczos", interlaced=False, interlacedd=False)
# adjusting output color from: RGB48 to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()





some more using RealSR_large:




Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..

Last edited by Selur; 8th November 2021 at 18:50.
Selur is offline   Reply With Quote
Old 9th November 2021, 08:21   #53  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 261
Very nice results (especially on the faces), even if - of course - not yet optimal for everything...

...btw I hope to see a SwinIR version optimized for videos too.

Last edited by PatchWorKs; 9th November 2021 at 08:25.
PatchWorKs is offline   Reply With Quote
Old 9th November 2021, 21:08   #54  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Quote:
not yet optimal for everything...
Espeically the last example shows some real issues

Quote:
...btw I hope to see a SwinIR version optimized for videos too.
first a way faster version would be needed
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 10th November 2021, 08:13   #55  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 261
Quote:
Originally Posted by Selur View Post
first a way faster version would be needed
Already asked, of course: https://github.com/JingyunLiang/SwinIR/issues/47

Note: I've also just "fed" @HolyWu with this awesome collection, let's see if other interesting "VS-ports" will come out...

Last edited by PatchWorKs; 10th November 2021 at 10:34.
PatchWorKs is offline   Reply With Quote
Old 11th November 2021, 11:05   #56  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 403
Quote:
Originally Posted by PatchWorKs View Post
Very nice results (especially on the faces), even if - of course - not yet optimal for everything...

...btw I hope to see a SwinIR version optimized for videos too.
Is a video version planed?

for upscaling, algos like esrgan (single image) are not very suitable for real-life content. Too much flickering. So unless SwinIR doesn´t get some extensions for multi-frame usage / flow detection /whatever, one will always get flickering / stutters / inkonsistent movement...
ReinerSchweinlin is offline   Reply With Quote
Old 18th November 2021, 17:52   #57  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 478
Nvidia just open sourced Nvidia image scaling... Would this be a candidate for the next filter?
aegisofrime is offline   Reply With Quote
Old 19th November 2021, 20:09   #58  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
Has anyone tested https://github.com/HolyWu/vs-hinet ?

Here are a few screen shots: (not sure what to make of them and for what content this is really useful)

Mode: Deblur GoPro

Mode: Deblur REDS

Mode: denoise

Mode: derain
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..

Last edited by Selur; 19th November 2021 at 21:32.
Selur is offline   Reply With Quote
Old 20th November 2021, 08:22   #59  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 261
According to your tests on that frame, the highest fidelity seems to be achieved by derain model, btw here are some questions:
  1. how fast is it ?
  2. how does it performs (in terms of both speed and fidelity) compared to xClean ?
  3. its description claims that it perform "restoring" (aka denoising ?) function, but since is BasicSR-based does it upscale too ?

Last but not least (even if OT): did you tried RIFE ?
https://github.com/HolyWu/vs-rife

Last edited by PatchWorKs; 20th November 2021 at 16:31.
PatchWorKs is offline   Reply With Quote
Old 20th November 2021, 10:17   #60  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,672
speed: ~2-3fps for sd content, so not that slow
xClean: no clue about xClean, haven't played around with it too many options for my taste (+ would need to add znedi3 and nnedi3cl support to it)
upscale: at least the current interface offers no upscaling and the method does not upscale
rife: yes, I like it (with sceneChange added). Waiting for FrameRateConverter to properly support it
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.