Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th September 2018, 12:00   #81  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 540
I must be doing something wrong,
Code:
C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 update -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Local definitions already up to date: Sun, 09 Sep 2018 22:02:52 GMT

C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 installed -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Name                      Namespace       Installed   Latest      Identifier
+Bifrost                  bifrost         Unknown     v2.2        com.nodame.bifrost
Bilateral                 bilateral       r3          r3          com.invalid.identifier.bilateral
BM3D                      bm3d            r7          r7          com.vapoursynth.bm3d
+Cnr2                     cnr2            Unknown     v1          com.nodame.cnr2
CTMF                      ctfm            r4          r4          com.holywu.ctmf
+D2V Source               d2v             Unknown     v1.1        com.sources.d2vsource
DCTFilter                 dctf            r2          r2          com.holywu.dctfilter
Deblock                   deblock         r6          r6          com.holywu.deblock
DeLogo                    delogo          v0.4        v0.4        in.7086.delogo
DFTTest                   dfttest         r4          r4          com.holywu.dfttest
EEDI2                     eedi2           r7          r7          com.holywu.eedi2
+FFmpegSource2            ffms2           Unknown     2.23        com.vapoursynth.ffms2
+FFTW3 Library            fftw3_library   Unknown     3.3.5       org.fftw.fftw3
+fmtconv                  fmtc            Unknown     r20         fmtconv
*AddGrain                 grain           r5          r6          com.holywu.addgrain
*KNLMeansCL               knlm            v1.1.0      v1.1.1      com.Khanattila.KNLMeansCL
MVTools                   mv              v20         v20         com.nodame.mvtools
MVTools (float)           mvsf            r9          r9          com.nodame.mvsf
+NNEDI3                   nnedi3          Unknown     v11         com.deinterlace.nnedi3
NNEDI3CL                  nnedi3cl        r7          r7          com.holywu.nnedi3cl
NNEDI3 Weights            nnedi3_weights  r1          r1          com.deinterlace.nnedi3.weights
Retinex                   retinex         r3          r3          com.invalid.identifier.retinex
TCanny                    tcanny          r10         r10         com.holywu.tcanny
TDeintMod                 tdm             r10         r10         com.holywu.tdeintmod
Tonemap                   tonemap         R2          R2          com.ifb.tonemap
+vcfreq                   vcfreq          Unknown     2016-05-12  in.vcmohan.freq
VagueDenoiser             vd              r2          r2          com.holywu.vaguedenoiser
VSFilter                  vsf             r2          r2          com.holywu.vsfilter
+Waifu2x                  w2xc            Unknown     r7          com.holywu.waifu2x-w2xc
W3FDIF                    w3fdif          r1          r1          com.holywu.w3fdif
+AWarpSharp2              warp            Unknown     v4          com.nodame.awarpsharp2
Yadifmod                  yadifmod        r10         r10         com.holywu.yadifmod
ZNEDI3                    znedi3          r1          r1          xxx.abc.znedi3

C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 upgrade-all -f -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Fetching: https://github.com/dubhater/vapoursy...-v2.2-win64.7z
Traceback (most recent call last):
  File ".\vsrepo.py", line 455, in <module>
    inst = upgrade_all_packages(args.force)
  File ".\vsrepo.py", line 368, in upgrade_all_packages
    res = upgrade_files(get_package_from_id(id, True))
  File ".\vsrepo.py", line 346, in upgrade_files
    install_files(p)
  File ".\vsrepo.py", line 310, in install_files
    result = subprocess.run([cmd7zip_path, "e", "-so", tfpath, fn_props[0]], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "subprocess.py", line 403, in run
  File "subprocess.py", line 709, in __init__
  File "subprocess.py", line 997, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified
When I extract portable 7zip to the folder C:\SOFTWARE\Vapoursynth-x64 I see a 7za.exe rather than a 7z.exe

Suggestions welcome.
hydra3333 is offline   Reply With Quote
Old 17th September 2018, 12:12   #82  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Sigh, just rename it to 7z.exe and see if it works. I didn't even consider 7zip having a portable version too...
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 17th September 2018, 12:33   #83  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 540
thank you, that works ...
hydra3333 is offline   Reply With Quote
Old 5th October 2018, 05:31   #84  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 540
tried it today and got this message ...
Code:
.\python.exe .\vsrepo.py -p -t win64 -f -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts upgrade-all
Fetching: http://rationalqm.us/hdr/DGHDRtoSDR_1.9.rar
Traceback (most recent call last):
  File ".\vsrepo.py", line 455, in <module>
    inst = upgrade_all_packages(args.force)
  File ".\vsrepo.py", line 368, in upgrade_all_packages
    res = upgrade_files(get_package_from_id(id, True))
  File ".\vsrepo.py", line 346, in upgrade_files
    install_files(p)
  File ".\vsrepo.py", line 311, in install_files
    result.check_returncode()
  File "subprocess.py", line 369, in check_returncode
subprocess.CalledProcessError: Command '['7z.exe', 'e', '-so', 'C:\\Users\\u\\AppData\\Local\\Temp\\vsmez18y1v_', 'x64/DGHDRtoSDR.dll']' returned non-zero exit status 2.
Tried a 7z manually
Code:
7z.exe l DGHDRtoSDR_1.9.rar
7-Zip (a) 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30
Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive
The .rar opens OK with winzip.

Advice welcomed.
hydra3333 is offline   Reply With Quote
Old 5th October 2018, 10:22   #85  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Maybe you have an older 7z version? I tested this package and had no problems extracting it.

//edit:
hmm ok, 18.05 is the newest one. But why does it work for me...
I can also open and extract rar files with the 7z gui -> 7zFM.exe

//edit2
on the 7zip page rar is listed as supported
Try to reinstall 7zip I guess...
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 5th October 2018 at 10:28.
ChaosKing is offline   Reply With Quote
Old 6th October 2018, 00:58   #86  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 540
OK, I'll re-download try again. And the 32bit version too I suppose.
Out of interest, did you try the commandline version ?
Code:
7za.exe l DGHDRtoSDR_1.9.rar
edit: freshly downloaded the x32 and x64 standalone versions from https://www.7-zip.org/download.html (I never install it) and ...
Code:
E:\SOFTWARE-Win10\ZIP\7zip\x>7za.exe l DGHDRtoSDR_1.9.rar

7-Zip (a) 18.05 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30

Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive

Errors: 1

E:\SOFTWARE-Win10\ZIP\7zip\x>x64\7za.exe l DGHDRtoSDR_1.9.rar

7-Zip (a) 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30

Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive

Errors: 1
Winzip definitely opens it OK. How strange.

Last edited by hydra3333; 6th October 2018 at 01:12.
hydra3333 is offline   Reply With Quote
Old 6th October 2018, 01:09   #87  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
It seems that there is a difference between 7z.exe and 7za.exe. Listing also works with 7z.exe.
Use this https://www.7-zip.org/a/7z1805-x64.exe


https://superuser.com/questions/1105...xe-and-7za-exe
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 6th October 2018, 01:37   #88  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 540
Ah, thanks for that and the superuser link.

edit: DG is considering .zip or something instead of .rar

Last edited by hydra3333; 9th October 2018 at 09:37.
hydra3333 is offline   Reply With Quote
Old 21st October 2018, 13:25   #89  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
I fixed the two biggest issues but as usual there may be a typo or something so help me test it. Just grab vsrepo.py from git because I'm lazy.

The first change is that it no longer uninstalls a previous version of a package before the new version is downloaded AND has the hashes verified. So much harder to get your plugins and scripts into a broken state by accident.

The second change is that packages where the download fails are skipped and reported as failed at the end, that way scripts with huge dependency lists won't end up broken as often. Note that multiple failures to fetch the same file may happen if multiple other packages depend on it.

Btw, I think DG secretly updates things after release to fix minor bugs => hash mismatches

I don't think there's a good way to handle that and it still barfs an exception. Discuss this one and think of a better solution if you can.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 21st October 2018, 14:00   #90  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
In my initial idea for a database / plugin manager I had planned to create a bot which checks gist, github, bitbucket and http links periodically for new releases.
Maybe a http last-modified header check could be implemented, so one can keep up with "secret" updates, deletes or new releases ... I don't think there is an easy solution for this but only if the author would follow certain steps.

I can add a cronjob in my vsdb databse to check for non github links and report it if something changes like a hash mismatch or delete.

//Edit:
Auto generating an updated json could be also implemented. => just check if ok and push to github, less work


OR

This was discussed in the github issue: A myplugin.json could be maintained by the author itself and vsrepo just grabs it.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 21st October 2018 at 14:10.
ChaosKing is offline   Reply With Quote
Old 21st October 2018, 14:12   #91  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by ChaosKing View Post
In my initial idea for a database / plugin manager I had planned to create a bot which checks gist, github, bitbucket and http links periodically for new releases.
Maybe a http last-modified header check could be implemented, so one can keep up with "secret" updates, deletes or new releases ... I don't think there is an easy solution for this but only if the author would follow certain steps.

I can add a cronjob in my vsdb databse to check for non github links and report it if something changes like a hash mismatch or delete.

//Edit:
Auto generating an updated json could be also implemented. => just check if ok and push to github, less work
I guess your idea to regularly scan for secretly changed packages is required. Don't feel like implementing it due to how ugly it is though. I'll think about it...

Automating the current thing is trivial. The problem is if files are added or removed in a package because then it needs manual help so now and then it'd push completely broken updates. Minimal human review once per week is probably a must.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 21st October 2018, 17:20   #92  |  Link
gonca
Registered User
 
Join Date: Jul 2012
Posts: 1,213
Quote:
Btw, I think DG secretly updates things after release to fix minor bugs => hash mismatches
Actually, I believe he slip streams minor fixes, and appends the slip stream number to the version number, but the number on the download from his web site does not change.
No secret, all very public and announced to the users by e-mail and in the binaries update thread
gonca is offline   Reply With Quote
Old 21st October 2018, 18:19   #93  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Quote:
Originally Posted by Myrsloik View Post
I guess your idea to regularly scan for secretly changed packages is required. Don't feel like implementing it due to how ugly it is though. I'll think about it...

Automating the current thing is trivial. The problem is if files are added or removed in a package because then it needs manual help so now and then it'd push completely broken updates. Minimal human review once per week is probably a must.
I don't think there will be many plugins that add or remove files. In 99% it stays the same. I thing this would be the easiest way:

Lets say we have a http://asd.xd/superplugin.zip url and a superplugin.json
- implement a scraper / bot function that checks if a) all urls of superplugin are still valid and b) if one of the "versions" have a different "last-modified" date. (maybe the whole zip could be hashed?)
- if a zip/rar has changed, download it, re-hash all known files that are stored in superplugin.json AND list also all contents of the zip/rar AND print a warning if a file is missing. Maybe it is a good idea to store the previous files list somewhere for comparison so that any new files could also be detected.
=> result: human review is reduced to a minimum.

@gonca yes, but it is only obvious if one checks the forum. VSRepo can't do that
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 21st October 2018, 19:04   #94  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by ChaosKing View Post
I don't think there will be many plugins that add or remove files. In 99% it stays the same. I thing this would be the easiest way:

Lets say we have a http://asd.xd/superplugin.zip url and a superplugin.json
- implement a scraper / bot function that checks if a) all urls of superplugin are still valid and b) if one of the "versions" have a different "last-modified" date. (maybe the whole zip could be hashed?)
- if a zip/rar has changed, download it, re-hash all known files that are stored in superplugin.json AND list also all contents of the zip/rar AND print a warning if a file is missing. Maybe it is a good idea to store the previous files list somewhere for comparison so that any new files could also be detected.
=> result: human review is reduced to a minimum.

@gonca yes, but it is only obvious if one checks the forum. VSRepo can't do that
I'd say it's simply beyond the scope. Let's just go shit on people who don't use github. It's so much faster and easier. Shaming works.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 21st October 2018, 21:21   #95  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
.\vsrepo.py -p install adjust
adjust already exists. Output text is "Nothing done". I think it showed something like "Already installed" before...
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 21st October 2018, 21:26   #96  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by ChaosKing View Post
.\vsrepo.py -p install adjust
adjust already exists. Output text is "Nothing done". I think it showed something like "Already installed" before...
I'm going to improve that text later.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 7th December 2018, 14:53   #97  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Here's an issue someone with far too much time can help with:
https://github.com/vapoursynth/vsrepo/issues/80

Too lazy to do it myself and it's not that important.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 7th December 2018, 19:17   #98  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Can't you just check for invalid (empty) releases with vsrupdate and then just add it to the ignore list? I mean isn't vsrupdate checking for all releases anyway, empty ones included?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 7th December 2018 at 19:20.
ChaosKing is offline   Reply With Quote
Old 8th December 2018, 12:50   #99  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by ChaosKing View Post
Can't you just check for invalid (empty) releases with vsrupdate and then just add it to the ignore list? I mean isn't vsrupdate checking for all releases anyway, empty ones included?
No. Empty entries can either mean that the release has no relevant binaries (for example see HolyWu's linux build system updates that do nothing for windows users) OR that the filename guessing algorithm failed to create a new entry. It needs to be manually checked to determine which one it is.

This issue about editing existing entries and doing it manually is probably faster than writing a script. Or about the same speed since it's a one time thing.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 8th December 2018, 14:46   #100  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
There are only 2 plugins with empty release tags left.
https://github.com/IFeelBloated/RGSF/releases <- see my commit
https://github.com/HomeOfVapourSynth...-w2xc/releases <- older models have some different filenames/file count. No one will install the older releses anyway... not worth the work.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.