Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th July 2020, 16:31   #1  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,286
range_in_s settings on bitdepth conversion ?

What is math or generic explanation for this observation?

Start with YUV420P8 "bars"; Y=0,16,235,255 . Then convert to YUV420P10 , using either range_in_s full or limited

Y values read with vsedit picker, but double checked with planestats (crop to a single color)

Limited
Y=0,64,940,1020

Full
Y=0,64,943,1023




Code:
a = core.std.BlankClip(width=320, height=720, format=vs.YUV420P8, fpsnum=24, fpsden=1, length=24, color=[0, 128, 128])
b = core.std.BlankClip(width=320, height=720, format=vs.YUV420P8, fpsnum=24, fpsden=1, length=24, color=[16, 128, 128])
c = core.std.BlankClip(width=320, height=720, format=vs.YUV420P8, fpsnum=24, fpsden=1, length=24, color=[235, 128, 128])
d = core.std.BlankClip(width=320, height=720, format=vs.YUV420P8, fpsnum=24, fpsden=1, length=24, color=[255, 128, 128])

clip = core.std.StackHorizontal([a,b,c,d])

clip10 = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="limited")
#clip10 = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="full")

clip10.set_output()
poisondeathray is offline   Reply With Quote
Old 24th July 2020, 19:13   #2  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,360
I always use fmtc for this kinda stuff, I never encountered any problem.
you can specify "fulls" and "fulld" to enforce the assumed source range and the result range.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 24th July 2020, 20:25   #3  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,286
Quote:
Originally Posted by feisty2 View Post
I always use fmtc for this kinda stuff, I never encountered any problem.
you can specify "fulls" and "fulld" to enforce the assumed source range and the result range.
It's not a "problem" per se ; I'm asking about the underlying explanation

This is not a full=>limited or limited=>full discussion ( it's not clamping or expanding levels. )

To clarify - When you convert YUV 8bit to YUV 10bit - What are the expected values? Why should the range parameter change the results when the setting is same for input and output

range_in_s="limited", range_s="limited"
Y=0,64,940,1020

range_in_s="full", range_s="full"
Y=0,64,943,1023

core.fmtc.bitdepth (clip, bits=10, fulls=False, fulld=False)
Y=0,64,940,1020

core.fmtc.bitdepth (clip, bits=10, fulls=True, fulld=True)
Y=0,64,943,1023

Last edited by poisondeathray; 24th July 2020 at 20:31.
poisondeathray is offline   Reply With Quote
Old 24th July 2020, 20:35   #4  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,360
when the input range is "limited", the bit depth conversion is a bit shift operation, concretely
result = input << (output_depth - input_depth)
and as you can see, 235 << (10-8) = 235 * 4 = 940

for full range input, the pixel range is first normalized to [0, 1], then linearly scaled to the target range
235 / 255 * 1023 = 942.76470588235294117647058823529 ≈ 943
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 24th July 2020 at 20:39.
feisty2 is offline   Reply With Quote
Old 24th July 2020, 21:07   #5  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,286
Quote:
Originally Posted by feisty2 View Post
when the input range is "limited", the bit depth conversion is a bit shift operation, concretely
result = input << (output_depth - input_depth)
and as you can see, 235 << (10-8) = 235 * 4 = 940

for full range input, the pixel range is first normalized to [0, 1], then linearly scaled to the target range
235 / 255 * 1023 = 942.76470588235294117647058823529 ≈ 943
Ahhh!
Thanks f2!
poisondeathray is offline   Reply With Quote
Old 24th July 2020, 21:25   #6  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,360
as defined in BT601-7 (page 4), the upper and lower bounds of limited range for bitdepths > 8 themselves are the bit shifted values of 8-bit bounds, and since the conversion is a linear function, the bit shift operation must also apply to everything in between.

and since bit shift fails to map the full range bounds of different bitdepths (apparently 255 << (10-8) = 255 * 4 = 1020 ≠ 1023), we follow the regular "normalize then scale" routine for full range bitdepth conversions

now I dont usually worry about bullshit like this because I only work with normalized fp32 inputs (0.0 = black, 1.0=white), sample types other than fp32 or fp64 should just f**kin' die
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 24th July 2020 at 21:29.
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.