[Are_]

After reading your function and realizing you didn't even boder to read the documentation for any plugin (Levels had transposed arguments and many functions had nonexistent arguments/function_names) I lost part of my motivation to help you.

Anyway here is like it may be done:

import vapoursynth as vs

def scale(old_value, new_bd=16):
    return int((old_value * ((1 << new_bd) - 1)) / 255)


def brightdfttestmod(clip, sbright=0, smedium=0, sdark=0, th_low=20, th_med=40, th_high=100, tbsize=5):
    
    core = vs.get_core()
    
    if not isinstance(clip, vs.VideoNode):
        raise ValueError('This is not a clip')
    
    # This scales the values for Levels in case we are not working with 8-bit clips
    if clip.format.bits_per_sample != 8:
        th_low = scale(th_low, clip.format.bits_per_sample)
        th_med = scale(th_med, clip.format.bits_per_sample)
        th_high = scale(th_high, clip.format.bits_per_sample)
    
    # Bright
    if sbright > 0:
        bright = core.dfttest.DFTTest(clip, sigma=sbright, tbsize=tbsize, sbsize=18, sosize=9)
    else: 
        bright = clip
        
    # Medium
    if smedium > 0:
        medium = core.dfttest.DFTTest(clip, sigma=smedium, tbsize=tbsize, sbsize=18, sosize=9)
    else: 
        medium = clip
        
    # Dark
    if sdark > 0:
        dark = core.dfttest.DFTTest(clip, sigma=sdark, tbsize=tbsize, sbsize=18, sosize=9)
    else: 
        dark = clip

    mmask = core.std.Levels(medium, th_med, th_high, 1.0, (1 << clip.format.bits_per_sample) - 1, 0)
    dmask = core.std.Levels(medium, th_low, th_med, 1.0, (1 << clip.format.bits_per_sample) - 1, 0)

    output = core.std.MaskedMerge(bright, medium, mmask, first_plane=1)
    output = core.std.MaskedMerge(output, dark, dmask, first_plane=1)

    return output

Carefully read it and try to notice what you were doing wrong.
I also get ride of lsb because it makes no sense when all your entire pipeline can be at any bit depth.