Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd May 2016, 12:06   #1  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Oyster, placebo compression artifacts buster

readme and more
https://github.com/IFeelBloated/Oyster

How this thing works:
certain component of the signal (something other than noise or blankness) would require either high resolution in time domain or frequency domain, but not both (uncertainty principle).

videos are non-stationary signals obviously, so ideal Fourier transform (something like DFTTest(sbsize=10000000000000001, tbsize=10000000000000001)) won't work, transform like that has a high frequency resolution but very poor time resolution, and leads to catastrophic results on components carrying lots of time features (sharp edges, delicate details, etc.), ringings, edges blurred out, and shit like that.

so, short time Fourier transform (something like DFTTest(sbsize=3, tbsize=3)), well, this kind of transform has a relatively higher time resolution but poor frequency resolution, and yeah, you know what kind of shit is about to happen next (washed-out, posterized, etc.), and lim(window size -> 0) gives the completely untransformed data! (frequency resolution -> 0)

anyways, it all comes down to one word, "resolution", you need higher resolution to get better quality, and BM3D, state of the art denoiser enhances frequency resolution by matching similar macroblocks. once a group of similar blocks have been matched, the group of signals go (mostly) stationary as similar blocks share similar copies of frequency distribution, and that's good news, and now go large-window-sized Fourier transform and get a high frequency resolution. and no worries about all that time resolution stuff cuz stationary signal simply has no time feature.

so why do we need more, I mean, BM3D sounds definitely awesome, the thing is, BM3D maximizes frequency resolution by re-positioning (block matching) the video signal, and after all these smart and well designed procedures, signal goes MOSTLY stationary but apparently not COMPLETELY stationary so there's still loss of time feature, BM3D manages to minimize that, but couldn't cancel that out completely, and then you notice that loss of sharpness on edges and delicate details and that's why NLMeans, a time domain filter should step in and clean the mess, and about MDeGrain? well, BM3D simply isn't aggressive enough to compression artifacts, you need something brutal and fierce to wipe the shit away first regardless of detail loss, and then recover the lost details and that's why MDeGrain stays around.

MDeGrain -> BM3D -> NLMeans, it just gets more and more delicate each time and details are restored gradually, and artifacts gets its ass kicked

log:
r4:
strict sanity check in every single function, a lot more user friendly and should be ok to work like any typical module now
added some temporal padding to KNLMeansCL
re-licensed to LGPL v3.0
trivial modifications to the doc
.gitignore

r3
algorithmic update, so quality boost

r2.1
removed global core cuz Myrsloik said it was a horrible idea, simple "import Oyster" works now
new function Search
rewrote Basic
trivial modifications to Final
removed SATD, SAD works better on compression artifacts
removed TrueMotion
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 19th August 2016 at 15:55.
feisty2 is offline   Reply With Quote
Old 3rd May 2016, 13:00   #2  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 706
Who is that?
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 3rd May 2016, 14:33   #3  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by jackoneill View Post
Who is that?
what?
you mean the sample clip?
Kylie Minogue - Come Into My World
one of the best music videos like ever
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 4th May 2016, 00:22   #4  |  Link
AzraelNewtype
Registered User
 
AzraelNewtype's Avatar
 
Join Date: Oct 2007
Posts: 133
Quote:
Originally Posted by feisty2
Waifu2x(dunno why but it looks kinda cartoonized somehow)
Because waifu2x is a neural network scaler trained to upscale cartoons, not live action? Its ability to fight compression artifacts (typically jpeg artifacts) is in service of that goal, not general purpose cleaning.
AzraelNewtype is offline   Reply With Quote
Old 4th May 2016, 00:24   #5  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by AzraelNewtype View Post
Because waifu2x is a neural network scaler trained to upscale cartoons, not live action? Its ability to fight compression artifacts (typically jpeg artifacts) is in service of that goal, not general purpose cleaning.
it has a photo mode indeed, but still...
no luck with that
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 4th May 2016, 02:12   #6  |  Link
BakaProxy
Registered User
 
Join Date: Jan 2015
Posts: 47
#3DWaifusExist

Verstuurd vanaf mijn SM-A500FU met Tapatalk
BakaProxy is offline   Reply With Quote
Old 4th May 2016, 03:49   #7  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by BakaProxy View Post
#3DWaifusExist

Verstuurd vanaf mijn SM-A500FU met Tapatalk
Please do tell, where
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 4th May 2016, 12:46   #8  |  Link
BakaProxy
Registered User
 
Join Date: Jan 2015
Posts: 47
Quote:
Originally Posted by feisty2 View Post
Please do tell, where
They're called real people

Verstuurd vanaf mijn SM-A500FU met Tapatalk
BakaProxy is offline   Reply With Quote
Old 4th May 2016, 13:10   #9  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by BakaProxy View Post
They're called real people

Verstuurd vanaf mijn SM-A500FU met Tapatalk
okay, not funny.
thought an inter-waifu2x was out. (waifu2x is intra)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 9th May 2016, 07:56   #10  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
r2.1
removed global core cuz Myrsloik said it was a horrible idea, simple "import Oyster" works now
new function Search
rewrote Basic
trivial modifications to Final
removed SATD, SAD works better on compression artifacts
removed TrueMotion
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 11th May 2016, 08:02   #11  |  Link
RazorBurn
Registered User
 
Join Date: Aug 2014
Posts: 14
Got error on r2.1

Code:
Failed to evaluate the script:
Python exception: Crop: cropped area extends beyond frame dimensions
Traceback (most recent call last):
  File "src\cython\vapoursynth.pyx", line 1491, in vapoursynth.vpy_evaluateScript (src\cython\vapoursynth.c:26897)
  File "C:/Temp/StaxRip Temp/SampleClip_temp/SampleClip.vpy", line 11, in <module>
    ref = Oyster.Basic(y,vec)
  File "C:\Apps\Python\lib\site-packages\Oyster.py", line 171, in Basic
    _mask           = helpers.genblockmask (ShufflePlanes (src, 0, vs.GRAY))
  File "C:\Apps\Python\lib\site-packages\Oyster.py", line 70, in genblockmask
    clip            = CropAbs (clip, src.width, src.height, 0, 0)
  File "src\cython\vapoursynth.pyx", line 1383, in vapoursynth.Function.__call__ (src\cython\vapoursynth.c:25204)
vapoursynth.Error: Crop: cropped area extends beyond frame dimensions
My Code

Code:
import vapoursynth as vs
core = vs.get_core()
import mvsfunc 
import edi_rpow2 as edi
import mvmulti
import Oyster
clip = core.lsmas.LibavSMASHSource(r"D:\Installers\Multimedia\Video Test Samples\SampleClip.mp4")
clp = mvsfunc.ToRGB(input = clip, depth=32)
y = core.std.ShufflePlanes(clp,0,vs.GRAY)
vec = Oyster.Search(y)
ref = Oyster.Basic(y,vec)
y = Oyster.Final(y,ref,vec)
clip =  core.std.ShufflePlanes([y,clp], [0,1,2], vs.YUV)
clip.set_output()
RazorBurn is offline   Reply With Quote
Old 12th May 2016, 14:18   #12  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by RazorBurn View Post
Got error on r2.1

Code:
clp = mvsfunc.ToRGB(input = clip, depth=32)
y = core.std.ShufflePlanes(clp,0,vs.GRAY)
...
???
and what's the resolution of your clip?
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 3rd July 2016, 17:24   #13  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
r3
algorithmic update, so quality boost
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 5th July 2016, 16:58   #14  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
so, bad news here.
been hours and I tried like a million sets of parameters to investigate if there's any chance of quality-speed tradeoff, and, failed to get rid of the abysmal performance even tho had reached the bottom line of almost, everything.
I lowered the precision down to uint8_t, and picked jackoneill's heavily asm optimized mvtools
I used fast profile on V-BM3D
I ditched Oyster.Super, made pel 1, lowest value possible, switched vector searching algorithm from exhaustive to hex
I disabled the varying search window of NLMeans, and fixed that to 4, constant
and, the performance went from 0.00fps up to 0.11fps on 720x480 GRAY, but, I mean, that's still outrageously SLOOOW..
guys, I really tried, but... yeah, and I'm out of ideas of performance boost..
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 5th July 2016, 17:46   #15  |  Link
Mystery Keeper
Beyond Kawaii
 
Mystery Keeper's Avatar
 
Join Date: Feb 2008
Location: Russia
Posts: 704
feisty2, 'tis okay. There was a time when I had to ditch simple anime anti-aliasing because it was too slow on my ancient PC. Eventually we'll have enough power to put your script into good use.
__________________
...desu!
Mystery Keeper is offline   Reply With Quote
Old 6th July 2016, 12:23   #16  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by Mystery Keeper View Post
feisty2, 'tis okay. There was a time when I had to ditch simple anime anti-aliasing because it was too slow on my ancient PC. Eventually we'll have enough power to put your script into good use.
I actually think there will be a huge performance boost if someone could make a GPU version of BM3D...

and I wrote the underlying mechanism at #1, hopefully not some kind of "utterly bizarre misunderstanding"
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 7th July 2016, 00:31   #17  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,110
Is this 64bit only? The required mvtool plugin only has a 64bit version
lansing is offline   Reply With Quote
Old 7th July 2016, 06:47   #18  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by lansing View Post
Is this 64bit only? The required mvtool plugin only has a 64bit version
uhmm, I could compile an x86 binary if you need that, this thing consumes A LOT OF ram, so that probably won't work anyway, you got that 4GB upper limit on x86 programs
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 7th July 2016, 13:07   #19  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 271
Tell me about that, I was testing if I had all the dependencies right with a 1080p source and ran OOM on my 16GB Ram machine.
Are_ is offline   Reply With Quote
Old 8th July 2016, 08:09   #20  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by Are_ View Post
Tell me about that, I was testing if I had all the dependencies right with a 1080p source and ran OOM on my 16GB Ram machine.
I got 16G also and didn't do any 1080p test, that's just way too heavy to my crappy i7-4790k, guess you'll need Xeon E5 or maybe Xeon E7 to go for a 1080p test, the highest resolution test I did is 720p, and it ran for an hour or so to just reconstruct one single frame

and also you will have to vspipe intermediate clips (basic estimation, etc) to your hard drive to avoid OOM on 720p
basically, something like this
Code:
ref = Oyster.Basic (clp, Oyster.Super (clp))
clp = Oyster.Destaircase (clp, ref, block_step=2)
clp = Oyster.Deringing (clp, ref, block_step=2)
goes like
Code:
ref = Oyster.Basic (clp, Oyster.Super (clp))
ref.set_output()
#vspipe xxx.vpy ref.rgb
Code:
ref = core.raws.Source ("ref.rgb")
clp = Oyster.Destaircase (clp, ref, block_step=2)
clp.set_output()
#vspipe xxx.vpy clp.rgb
Code:
ref = core.raws.Source ("ref.rgb")
clp = core.raws.Source ("clp.rgb")
clp = Oyster.Deringing (clp, ref, block_step=2)
clp.set_output()
#vspipe xxx.vpy whatever.rgb
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 8th July 2016 at 08:22.
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.