Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th June 2015, 07:12   #201  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
One problem is that if you tried that extreme exposure shift with an 8bit input, there would be any details to lead to banding -- it'd be one solid flat white or black with a few sprinkles of color here and there. 3dsmax starts with 64bit and downconverts (with lousy dithering IMHO) from there; it doesn't start with 8bit and work up. What you start with is far more important for your process than what you could do with it under perfect conditions.

If you're creating ultra-high-bit CGI, then you shouldn't need to apply extreme grading or filtering: That should be part of the generating process.

Like I said, float has a tangible and very valuable benefit over int -- the whole universe is its gamut, and you never have to worry about clipping, unless you routinely work with infinity. Even float16 is sufficient for conversion to int8 or int10, but since there's zero hardware or SIMD support for it, there's no point in saving a little memory for a lot of cpu time anymore. Likewise, int16 linear can uniquely represent all but maybe one or two of the bottom int10 gamma corrected values, so it is enough for any practical purpose (encoding). And if you get too hung up on roundoff errors, pretty soon you start wanting doubles or even quads.

Honestly, when it comes to filters directly associated with color control and grading, float32 support is damn handy, but I just can't see any point to it for destructive filters (like denoisers).
foxyshadis is offline   Reply With Quote
Old 18th June 2015, 07:56   #202  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
I'm not grading or filtering CGI, it's just, I found 16bpc can actually be, not enough in some cases, I had used to think 16bpc should be enough for almost everything, and that 3dsmax incident refreshed my mind, so a new level of "enough for everything", float32, replaced 16bits stuff in head
and naturally, I want all 16bits stuff degraded and take float32 instead, for, first, it can get the most hardass situation handled, and it's more mathematically correct, that's a bonus
should I just be okay with 16bpc, I'm in the middle of mvtools modification
feisty2 is offline   Reply With Quote
Old 18th June 2015, 14:41   #203  |  Link
Bloax
The speed of stupid
 
Bloax's Avatar
 
Join Date: Sep 2011
Posts: 317
?That 3ds Max incident where you did the equivalent of squishing a 0-255 range into 0-1 and then getting imperfections when stretching it back to 0-255 reminded you of the fact that you need more precision despite this never happening in the wild?
Bloax is offline   Reply With Quote
Old 18th June 2015, 15:56   #204  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
that kind of thing could happen, extreme like that, maybe rare, milder version, very common actually, say, every step introduces some errors, <= 0.5/65536 at int16, literally "very small" errors, but that's just ONE step, what happens if you got a process chain, right, errors will accumulate , they will grow fatter and fatter gradually, and finally, they won't just be there as "small" errors, they will become large errors, if you just got operations like +/-, errors might stay small, now imagine, what about operations like ^, exp, log

Last edited by feisty2; 18th June 2015 at 16:01.
feisty2 is offline   Reply With Quote
Old 18th June 2015, 16:08   #205  |  Link
captainadamo
Guest
 
Posts: n/a
Can you actually show us visual results of these "errors" you are spending huge amounts of time to get rid of? Maybe if you can actually demonstrate a need for this using a real-world sample, not something contrived in order to exaggerate, someone may be more willing to put in the work. All you've done so far is talk about theoreticals and vagaries but no actual examples.

Edited to add: It just seems that your effort would be better spent trying to mitigate these errors through other means rather than trying to hamfist in support for some other format into a codebase that is at best an opaque wall of code even for people who know C or C++ well. Let alone for someone who admits that they are basically a novice.

Last edited by captainadamo; 18th June 2015 at 16:29.
  Reply With Quote
Old 18th June 2015, 16:40   #206  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
I never had serious problems with int16 in regular cases so far, things are pretty extreme when int16 fails, then you might think they are "contrived in order to exaggerate", but I'm doing this like an insurance, it will surely save my ass whenever shit happens, maybe that's rare, but it won't bite in regular cases, and you even got some tiny bonus like "more mathematically correct"

Last edited by feisty2; 18th June 2015 at 16:43.
feisty2 is offline   Reply With Quote
Old 18th June 2015, 16:45   #207  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
I think you might be overreacting, look at QTGMC, it's all 8-bit processing and the result looks fantastic. I would be a little bit more understanding If your source was actually truly high bit depth. Anyways I think this discussion has gone off topic...
Reel.Deel is offline   Reply With Quote
Old 18th June 2015, 16:56   #208  |  Link
captainadamo
Guest
 
Posts: n/a
Quote:
Originally Posted by feisty2 View Post
I never had serious problems with int16 in regular cases so far, things are pretty extreme when int16 fails, then you might think they are "contrived in order to exaggerate", but I'm doing this like an insurance, it will surely save my ass whenever shit happens, maybe that's rare, but it won't bite in regular cases, and you even got some tiny bonus like "more mathematically correct"
Then have fun with that. You do realize that all you're going to do is make all your scripts with mvtools be even slower than they already are, right? Especially if you start removing asm code like you mention above. Is likely a 2 or 3 fold increase in processing time for the mvtools part of your script really going to be worth fixing rounding errors you admit you've never actually seen make visible problems? It's your time to waste as you want...

Last edited by captainadamo; 18th June 2015 at 17:01.
  Reply With Quote
Old 18th June 2015, 17:06   #209  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
I changed my mind hours ago (when Myrsloik confirmed he won't add float to rgvs/core soon) actually, I'm starting with simple filters like removegrain
feisty2 is offline   Reply With Quote
Old 23rd August 2015, 11:02   #210  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
so I'm floatizing MVTools and actually not that hard like I thought it would be
https://github.com/IFeelBloated/MVTools-Float
just floatized mv.Super and working on mv.Analyze
feisty2 is offline   Reply With Quote
Old 25th August 2015, 03:54   #211  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
where did you scale SAD values calculated from clips with different bit depths to the same level (like default thsad=400 works almost the same on uint8_t and uint16_t)?
I got float SADs returned by Sad_C and how do I make it that "400" kind of stuff?

edit: never mind, just found out they are scaled inside each individual client functions, not mv.Analyze...

edit2: Do I have to store the float SAD as bytes inside VECTOR struct and restore it back to float when I read it?

Last edited by feisty2; 25th August 2015 at 07:00.
feisty2 is offline   Reply With Quote
Old 26th August 2015, 16:24   #212  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@jackoneill
2 parameters (lambda and global) in mv.analyse are actually unusable, as they are internally occupied by Python, guess you'll have to rename them
feisty2 is offline   Reply With Quote
Old 26th August 2015, 17:30   #213  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by feisty2 View Post
@jackoneill
2 parameters (lambda and global) in mv.analyse are actually unusable, as they are internally occupied by Python, guess you'll have to rename them
https://github.com/vapoursynth/vapou...lter-arguments
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 27th August 2015, 03:12   #214  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
ah, didn't know that before, thx for the tip
feisty2 is offline   Reply With Quote
Old 29th August 2015, 06:07   #215  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
SATD(C version) for 16 * X blocks is broken I guess
feisty2 is offline   Reply With Quote
Old 29th August 2015, 09:25   #216  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by feisty2 View Post
SATD(C version) for 16 * X blocks is broken I guess
You guess? Or do you know for sure and can point out the problem?
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 29th August 2015, 09:31   #217  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
Quote:
Originally Posted by jackoneill View Post
You guess? Or do you know for sure and can point out the problem?
I'm sure, I can't point the problem out cuz I don't really get how that code works actually
but the result is very different from the avisynth mvtools, so it has to be broken
feisty2 is offline   Reply With Quote
Old 29th August 2015, 09:53   #218  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@jackoneill

Code:
xxx
clp=last
sup = MSuper(clp)
bv1 = MAnalyse(sup,isb=True,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
fv1 = MAnalyse(sup,isb=False,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
bv2 = MAnalyse(sup,isb=True,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
fv2 = MAnalyse(sup,isb=False,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
bv3 = MAnalyse(sup,isb=True,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
fv3 = MAnalyse(sup,isb=False,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
MDegrain3(clp, sup, bv1, fv1, bv2, fv2, bv3, fv3, thsad=2000, thscd1=10000)
converttoy8 ()


Code:
import vapoursynth as vs
core = vs.get_core()
clp = xxx
clp = core.std.ShufflePlanes(clp, planes=0, colorfamily=vs.GRAY)
clp = core.fmtc.bitdepth(clp, fulls=False, fulld=True, bits=32, flt=True)
clp2 = core.fmtc.bitdepth(clp, fulls=False, fulld=True, bits=16)

sup = core.mvsf.Super(clp)
bv1 = core.mvsf.Analyze(sup,isb=True,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
fv1 = core.mvsf.Analyze(sup,isb=False,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
bv2 = core.mvsf.Analyze(sup,isb=True,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
fv2 = core.mvsf.Analyze(sup,isb=False,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
bv3 = core.mvsf.Analyze(sup,isb=True,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
fv3 = core.mvsf.Analyze(sup,isb=False,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
clp = core.mvsf.Degrain3(clp, sup, bv1, fv1, bv2, fv2, bv3, fv3, thsad=2000, thscd1=10000)

sup2 = core.mv.Super(clp2)
bv12 = core.mv.Analyse(sup2,isb=True,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
fv12 = core.mv.Analyse(sup2,isb=False,blksize=16,overlap=8,delta=1,search=3,truemotion=True,dct=5)
bv22 = core.mv.Analyse(sup2,isb=True,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
fv22 = core.mv.Analyse(sup2,isb=False,blksize=16,overlap=8,delta=2,search=3,truemotion=True,dct=5)
bv32 = core.mv.Analyse(sup2,isb=True,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
fv32 = core.mv.Analyse(sup2,isb=False,blksize=16,overlap=8,delta=3,search=3,truemotion=True,dct=5)
clp2 = core.mv.Degrain3(clp2, sup2, bv12, fv12, bv22, fv22, bv32, fv32, thsad=2000, thscd1=10000)
clp2 = core.fmtc.bitdepth(clp2, fulls=True, fulld=True, bits=32, flt=True)

fnl = core.std.StackHorizontal([clp,clp2])
fnl.set_output ()
feisty2 is offline   Reply With Quote
Old 29th August 2015, 10:38   #219  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by feisty2 View Post
I'm sure, I can't point the problem out cuz I don't really get how that code works actually
but the result is very different from the avisynth mvtools, so it has to be broken
Then how did you fix it?
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 29th August 2015, 10:44   #220  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
Didn't fix it, more of "rewrote it"
The 4x4 SATD works so I just extended it to 8x8 16x16 and 32x32
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.