Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th May 2018, 18:48   #1  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,164
Protecting lines from thinning when applying a LUT in anime?

I've got the Sailor Moon blu-ray, it was upscale from SD source and it has a red color cast. After I color corrected it with a LUT, some of the lines appeared to be thinned. You can see it here with this before and after, escpecially in the white areas:

source
color correct

The reason for that is in the source, because of the color cast, the outer part (softer part) of the lines appeared to be color casted as well (partially red). So when the LUT was applied with a mapping like "red->white", those outer parts were changed to white as well, resulting in lines to look thinned.

My question is how do I protect those outer parts?

This is my script:
PHP Code:
lut_file r'test.cube'

clip core.ffms2.Source(r"abc")

rgb_clip core.resize.Bicubic(clipmatrix_in_s="709"format=vs.RGBS)

match_clip core.timecube.Cube(rgb_clipcube=lut_file)

match_clip.set_output() 
lut file
lansing is offline   Reply With Quote
Old 14th May 2018, 19:16   #2  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,983
Did you try a line mask ? Maybe masktools, tcanny or some other method ?

Or merge back darker lines, or even line darkener ?

Or try adjusting the cast with other method ?
poisondeathray is offline   Reply With Quote
Old 14th May 2018, 20:18   #3  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,164
Quote:
Originally Posted by poisondeathray View Post
Did you try a line mask ? Maybe masktools, tcanny or some other method ?

Or merge back darker lines, or even line darkener ?

Or try adjusting the cast with other method ?
I'm thinking of line darkening only the edge of the lines after the LUT? But I don't see any edge masking filter that can select those.
lansing is offline   Reply With Quote
Old 14th May 2018, 20:32   #4  |  Link
fAy01
Registered User
 
Join Date: Jun 2010
Posts: 74
Quote:
Originally Posted by lansing View Post
I'm thinking of line darkening only the edge of the lines after the LUT? But I don't see any edge masking filter that can select those.
https://kageru.moe/blog/article/edgemasks/

fAy01 is offline   Reply With Quote
Old 15th May 2018, 04:19   #5  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,164
So I was trying to create a darker clip with levels for testing with maskmerge, but the Levels filter doesn't seem to work properly.

PHP Code:
lut_file r'test.cube' 

clip core.ffms2.Source(r"abc"

rgb_clip core.resize.Bicubic(clipmatrix_in_s="709"format=vs.RGBS

match_clip core.timecube.Cube(rgb_clipcube=lut_file

# this works properly as it brighten the highs
darken_clip core.std.Levels(match_clipmin_in=0max_in=225min_out=0max_out=255)

# this gives me a complete dark frame
darken_clip core.std.Levels(match_clipmin_in=50max_in=255min_out=0max_out=255)

darken_clip.set_output() 
lansing is offline   Reply With Quote
Old 15th May 2018, 04:24   #6  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,983
0 to 255 is for 8 bit code values . You probably need to change the range for RGBS

It should be 0.0 to 1.0 white to black for float
poisondeathray is offline   Reply With Quote
Old 15th May 2018, 05:04   #7  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,164
Quote:
Originally Posted by poisondeathray View Post
0 to 255 is for 8 bit code values . You probably need to change the range for RGBS

It should be 0.0 to 1.0 white to black for float
You are right, I change all values to 0-1 and it works.

Quote:
Originally Posted by fAy01 View Post
I don't see anything about selecting lines in that article.
lansing is offline   Reply With Quote
Old 15th May 2018, 05:10   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,983
Quote:
Originally Posted by poisondeathray View Post
0 to 255 is for 8 bit code values . You probably need to change the range for RGBS

It should be 0.0 to 1.0 white to black for float
oops I meant black to white





You might be able modify a line mask for parts of edges by thinning the selected lines (maybe awarpsharp1/2) if that' s what you meant . But it might be difficult to thin say, outer edge or inner edge selectively
poisondeathray is offline   Reply With Quote
Old 18th May 2018, 08:25   #9  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,164
I kind of get it working now. I created an average clip with convolution matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1]), and then compare the color-corrected clip to the average clip. If the value of the pixel in the cc is less than the average, change it to the value of the average. The line edge will darkened. Then I have copy the linemask code from fastlinedarkenmod and do a maskmerge between cc and the darken clip.

Code:
lut_file = r'test.cube' 

clip = core.ffms2.Source(r"abc") 

rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.RGBS) 

match_clip = core.timecube.Cube(rgb_clip, cube=lut_file) 

yuv_clip = core.resize.Bicubic(match_clip, matrix_s="709", format=vs.YUV420P8)

avg_clip = core.std.Convolution(yuv_clip, matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])

# compares clip to average, change value when below average
def darken_edge(x, y):         
	if x < y:
		return x
	else:
		if abs(x-y) < 30:
			return y 
		else:
			return x

darken_clip = core.std.Lut2(clipa=yuv_clip, clipb=avg_clip, function=darken_edge)

# linemask codes from fastlinedarkenmod
def scale(value, peak):
    return value * peak // 255

strength=48
protection=5
luma_cap=191
threshold=4
thinning=54


peak = (1 << yuv_clip.format.bits_per_sample) - 1 #255
Str = strength / 128
lum = scale(luma_cap, peak)
thr = scale(threshold, peak)
thn = thinning / 16

tmp = scale(127, peak)
     
exin = core.std.Maximum(yuv_clip, threshold=peak // (protection + 1)).std.Minimum()   

diff = core.std.Expr([yuv_clip, exin], ['y {lum} < y {lum} ? x {thr} + > x y {lum} < y {lum} ? - 0 ? {i} +'.format(lum=lum, thr=thr, i=tmp)])
    

expr = 'x {i} - {thn} * {peak} +'.format(i=tmp, thn=thn, peak=peak)
linemask = core.std.Expr([core.std.Minimum(diff)], [expr]).std.Convolution(matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])

# merge
result_clip = core.std.MaskedMerge(darken_clip, yuv_clip, linemask)

result_clip.set_output()
The problem I have now is that with the linemask, the edge darken effect is so subtle that it's unnoticeable. Without the mask it looks good.
lansing is offline   Reply With Quote
Old 20th May 2018, 04:54   #10  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,420
Quote:
Originally Posted by lansing View Post
I kind of get it working now. I created an average clip with convolution matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1]), and then compare the color-corrected clip to the average clip. If the value of the pixel in the cc is less than the average, change it to the value of the average. The line edge will darkened. Then I have copy the linemask code from fastlinedarkenmod and do a maskmerge between cc and the darken clip.

Code:
lut_file = r'test.cube' 

clip = core.ffms2.Source(r"abc") 

rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.RGBS) 

match_clip = core.timecube.Cube(rgb_clip, cube=lut_file) 

yuv_clip = core.resize.Bicubic(match_clip, matrix_s="709", format=vs.YUV420P8)

avg_clip = core.std.Convolution(yuv_clip, matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])

# compares clip to average, change value when below average
def darken_edge(x, y):         
	if x < y:
		return x
	else:
		if abs(x-y) < 30:
			return y 
		else:
			return x

darken_clip = core.std.Lut2(clipa=yuv_clip, clipb=avg_clip, function=darken_edge)

# linemask codes from fastlinedarkenmod
def scale(value, peak):
    return value * peak // 255

strength=48
protection=5
luma_cap=191
threshold=4
thinning=54


peak = (1 << yuv_clip.format.bits_per_sample) - 1 #255
Str = strength / 128
lum = scale(luma_cap, peak)
thr = scale(threshold, peak)
thn = thinning / 16

tmp = scale(127, peak)
     
exin = core.std.Maximum(yuv_clip, threshold=peak // (protection + 1)).std.Minimum()   

diff = core.std.Expr([yuv_clip, exin], ['y {lum} < y {lum} ? x {thr} + > x y {lum} < y {lum} ? - 0 ? {i} +'.format(lum=lum, thr=thr, i=tmp)])
    

expr = 'x {i} - {thn} * {peak} +'.format(i=tmp, thn=thn, peak=peak)
linemask = core.std.Expr([core.std.Minimum(diff)], [expr]).std.Convolution(matrix=[1, 1, 1, 1, 1, 1, 1, 1, 1])

# merge
result_clip = core.std.MaskedMerge(darken_clip, yuv_clip, linemask)

result_clip.set_output()
The problem I have now is that with the linemask, the edge darken effect is so subtle that it's unnoticeable. Without the mask it looks good.
Perhaps add .std.Maximum or .std.Inflate to the linemask? That'll widen its effective area.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Reply

Tags
lut

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.