Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th June 2004, 01:53   #101  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Code:

SetMemoryMax(256)

Import("C:\Programme\AviSynth 2.5\plugins\IIP.avs")

LoadPlugin("C:\Programme\AviSynth 2.5\plugins\undot.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\WarpSharp.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\LoadPluginEx.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\dustv5.dll")

mpeg2source("C:\Dokumente und Einstellungen\Soulhunter\Desktop\Temporal\test4\test4.d2v")

iip( dest_x=     1024,   dest_y=        576,
 \   ss1_x =    1.414,  ss1_y =      1.414,
 \   duststr =      2,  dustweight =  1.0,  antiflicker1= true, antiflicker2= true,
 \   detailcontr1=127,  detailcontr2 = 255,  contr_radius =  3, PixSharp=0.4,
 \   ss2_x =      3.5,  ss2_y =        3.5,
 \   Xstren =     255,  Xlimit =       255,
 \   subpelstren= 1.0,  flatweight =  64,
 \   protect_floor= 4,  protect_bias =  16,
 \   dering =    -60,  dering_weight= 1.0, dering_floor = 16, dering_bias=8,
 \   detail_floor= 1,  EQ = 2,
 \   warp_Y = true,     warp_UV = false,
 \   debug= "showall | compareH/V | protect | dering",  
 \   cropx=32, cropy=16
 \   )
Quote:

Script error: yv12subtract does not have a named argument "wideRange" -> IIP.avs, line 161...
__________________

Visit my IRC channel

Last edited by Soulhunter; 28th June 2004 at 01:59.
Soulhunter is offline   Reply With Quote
Old 28th June 2004, 02:38   #102  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
masktools-v1.4.16.zip
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th June 2004, 04:12   #103  |  Link
malkion
Registered User
 
Join Date: Feb 2004
Posts: 156
Didee, great job!

The new routine looks cool. Much softer than previous editions of iip.
Further testing on its way now. Thanks.
malkion is offline   Reply With Quote
Old 28th June 2004, 09:03   #104  |  Link
DarkNite
Almost Silent Member
 
DarkNite's Avatar
 
Join Date: Jun 2002
Location: Purgatory
Posts: 272
I agree, nice additions, and great script overall.

Oh, and I now present you with the The Most Visually Appealing Hyperlink of the Doom9 Forums Award for 2004. Kudos
__________________
Rethinking the "Why?" chromosome.
DarkNite is offline   Reply With Quote
Old 28th June 2004, 17:17   #105  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
iiP, v.0.5a

I re-arranged this post for better readability of the thread.


Full function call for iiP v.0.5a:
Code:
iip( dest_x=       XXX,  dest_y=        YYY,
 \   ss1_x =     1.414,  ss1_y =      1.414,
 \   duststr =       2,  dustweight =   1.0,  antiflicker1= true, antiflicker2= true,
 \   detailcontr1= 104,  detailcontr2 = 208,  contr_radius =    2, PixSharp=0.38,
 \   ss2_x =       3.5,  ss2_y =        3.5,
 \   Xstren =      255,  Xlimit =       255,
 \   subpelstren= 1.58,  flatweight =    0,
 \   protect_floor = 0,  protect_bias =  16,
 \   dering =      -80,  dering_weight= 1.0, dering_floor = 8, dering_bias=12,
 \   detail_floor = 10,  EQ = 2,
 \   warp_Y = false,     warp_UV = false,   exborder = false,
 \   debug= "protect | dering | compareH|V | showall", 
 \   cropx=64, cropy=32
 \   )
The script:
Code:
# iip :  Integrated Image Processor, v.0.5a
#
#
#  ---------------
# |  It is slow.  |
#  ---------------
#
# New in v.0.5 :
#
# EQ [int] : Edge quality. 0=deaktivated (don't do that!!!), 1=standard (as formerly), 2=better'n'slower, 3=evenbetter'n'evenslower
# Note: when using EQ = 2|3, then it's a good idea to *DEACTIVATE* deringing in iiP (don't hit the dog twice).
#
# warp_Y, warp_UV [bool] additional Luma & Chroma Warping. 
#
# Luma warping should make the picture more 'gracile' when upsizing to bigger resolution, e.g. DVD -> HDTV.
# For same resolution or downsizing, it may (?) help for sources with distorted detail. Or it may not - YMMV.
#
# Chroma warping helps on sources with bleeding colors (e.g. Enterprise: on the uniforms, the red stripes are bleeding into 
#   the blue cloth, which looks *very* ugly [like an aquarell that's gotten wet]. It looks much better with chroma warping.)
# On clean sources, chroma warping is a waste of CPU cycles.
#
# exborder [bool] : exclude outmost 8-pixel-border from sharpening (gradient masked)
#
# dering: old method: called with positive values for "dering"
#         new method: called with negative values for "dering" (better, but slower)
#
# detail_floor [int] : was formerly hardcoded, now user-definable. default 20.
#

function Ylevels(clip clp, int a, float gamma, int b, int c, int d)
       { wicked="x "+string(a)+" - "+string(b)+" "+string(a)+" - / 1 "+string(gamma)+" / ^ "+string(d)+" "+string(c)+" - * "+string(c)+" +"
        # Reminder: Yexpr = "x a - b a - / 1 gamma / ^ d c - * c +"
        # return( clp.subtitle(wicked) )
         return( clp.YV12LUT(Yexpr = wicked) )  
         }

function FineEdge( clip clp, int "div" )
       {
         logic(  clp.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", setdivisor=true, divisor=div)
          \     ,clp.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", setdivisor=true, divisor=div), "max", Y=3,U=1,V=1 )
         }

function iip( clip  clp, 
 \            int   "dest_x",       int   "dest_y", 
 \            int   "duststr",      float "dustweight", 
 \            float "ss1_x",        float "ss1_y", 
 \            int   "detailcontr1", int   "detailcontr2",
 \            int   "contr_radius", float "PixSharp", 
 \            float "ss2_x",        float "ss2_y", 
 \            int   "Xstren",       int   "Xlimit",  
 \            float "subpelstren",  int   "flatweight",
 \            bool  "antiflicker1", bool  "antiflicker2",
 \            int   "protect_floor",int   "protect_bias",
 \            int   "dering",       float "dering_weight",
 \            int   "dering_floor", float "dering_bias",
 \            int   "detail_floor",
 \            int   "EQ",           bool  "exborder",
 \            bool  "warp_Y",       bool  "warp_UV",
 \            string "debug", 
 \            int   "cropx",        int   "cropy"
 \            )
{
ox = clp.width
oy = clp.height
xcorr=(int(ox/16.0+.9999)*16)-ox
ycorr=(int(oy/ 8.0+.9999)* 8)-oy
dest_x =      default(dest_x, clp.width)
dest_y =      default(dest_y, clp.height)
ss1_x =       default(ss1_x, 1.414)
ss1_y =       default(ss1_y, 1.414)
ss2_x =       default(ss2_x, 3.5)
ss2_y =       default(ss2_y, 3.5)
duststr =     default(duststr, 2)
dustweight =  default(dustweight, 1.0)
antiflicker1= default(antiflicker1, true)
antiflicker2= default(antiflicker2, true)
Xlimit =      default(Xlimit, 255)  #  SS<3 ?  23 : 255
Xstren =      default(Xstren, 255)  #  SS<3 ? 216 : 255
detailcontr1= default(detailcontr1, 104)
detailcontr2= default(detailcontr2, 208)
contr_radius= default(contr_radius, 2)
PixSharp =    default(PixSharp, 0.4)
subpelstren = default(subpelstren, 1.58)
flatweight =  default(flatweight, 0)
protect_floor=default(protect_floor, 0)
protect_bias= default(protect_bias, 16)
dering =      default(dering, -80)
dering_floor= default(dering_floor, 8)
dering_floor= dering<0 ? dering_floor*2 : dering_floor
dering_bias = default(dering_bias, 16)
dering_weight=default(dering_weight, 1.0)
detail_floor= default(detail_floor, 20)
EQ =          default(EQ, 2)
EQ = (EQ>3) ? 3 : EQ
exborder =    default(exborder, false)
warp_Y =      default(warp_Y, false)
warp_UV =     default(warp_UV, false)
debug =       default(debug, "Mickey Mouse")
cropx =       default(cropx, 40)
cropy =       default(cropy, 20)

cropx=int(cropx/4)*4
cropy=int(cropy/4)*4
xx_ss1 = int(ox*ss1_x/16+.5)*16
yy_ss1 = int(oy*ss1_y/16+.5)*16
xx_ss2 = int(ox*ss2_x/16+.5)*16
yy_ss2 = int(oy*ss2_y/16+.5)*16
xx_small = int(ox/(abs(dering)/80+1.0)/16+.5)*16 + 32
yy_small = int(oy/(abs(dering)/80+1.0)/16+.5)*16 + 16

# Now I have the other idea - but it's too slow, yet
clp = antiflicker1 == true ? clp.temporalsoften(2,2,3,23,2) : clp

#---------------------------------------------------------------------------------------------------------------------
# Base denoising by PixieDust
#
dusted = clp.addborders(0,0,xcorr,ycorr).converttoyuy2().PixieDust(limit=duststr).converttoyv12().crop(0,0,-xcorr,-ycorr)
dusted = (dustweight==0.0) ? clp 
 \                         : ( dustweight<1.00 ? Overlay(clp, dusted, opacity=dustweight)
 \                                             : dusted )  

soft=dusted.bicubicresize(xx_small,yy_small,.2,.4).bicubicresize(xx_ss1,yy_ss1,1.0,.0)

#---------------------------------------------------------------------------------------------------------------------
# Build EdgeMask to protect already sharp detail from oversharpening
#
YV12subtract( dusted.sharpen(0.6).sharpen(0.6), dusted.blur(1.0).blur(1.0) )

edge00 = bicubicResize(xx_ss1,yy_ss1,-1.0,1.0)
edge0=edge00
 \     .YV12LUT(Yexpr="x 128 - abs "+string(protect_floor)+" - "+string(protect_bias)+" *")
 \     .deflate(Y=3,U=1,V=1).inflate(Y=3,U=1,V=1).blur(1.58)
edge00 = edge00.yv12lut(yexpr="x 128 - 2 *").expand(Y=3,U=1,V=1).blur(1.58)

#---------------------------------------------------------------------------------------------------------------------
# Build DeRing'ing Mask: +++ OLD ROUTINE +++
#
dering > 0 ? expand(Y=3,U=1,V=1) : NOP
edge1b = dering > 0
 \     ? yv12subtract( expand(Y=3,U=1,V=1).blur(1.58),
 \                     inpand(Y=3,U=1,V=1),tol=1,wideRange=true            )
 \      .bicubicResize(xx_ss1,yy_ss1,-1.0,1.0)
 \      .yv12LUT(Yexpr="x 128 - abs "+string(dering_floor)+" - "+string(dering_bias)+" *")
 \      .deflate(Y=3,U=1,V=1).blur(1.58)
 \     : last
                                                                                        # .Ylevels(0,3.0,255,0,255)
#---------------------------------------------------------------------------------------------------------------------
# 1st supersampling stage to perform sharpening at
#

(ss1_x > 1.0 || ss1_y > 1.0) ? dusted.Lanczosresize(xx_ss1,yy_ss1)
 \                           : dusted

#---------------------------------------------------------------------------------------------------------------------
# Build DeRing'ing Mask & perform dering'ing:   +++ NEW ROUTINE (Pre-processor) +++
#
tmpp = string(dering_floor)
edge1b = dering < 0
 \                   ?  yv12subtract( last, last.xsharpen(255,255),tol=1,wideRange=true )
 \                     .yv12lut(yexpr="x 128 - abs "+tmpp+" *")
 \                     .expand().inflate().expand().inflate().FineEdge(dering_bias).blur(1.58)
 \                     .greyscale.Ylevels(19,1.6,208,0,255)
 \                   : edge1b
dering < 0 ? maskedmerge( last, soft, edge1b, Y=3,U=1,V=1, useMMX=true ) : NOP

#-------- iterative sharpening, currently only 3-fold: its slow enough ... --------
 
temp = last
shrpcnt = 0
mskcnt  = 0
shrpcnt = (PixSharp     != 0) ? shrpcnt + 1 : shrpcnt
shrpcnt = (detailcontr2 != 0) ? shrpcnt + 1 : shrpcnt
shrpcnt = (detailcontr1 != 0) ? shrpcnt + 1 : shrpcnt
mskcnt  = (EQ>shrpcnt)        ? shrpcnt     : EQ

# Stage 1 :  UnsharpMasking, wide radius -----------------------
detailcontr1 != 0 
 \                ?   unsharpmask(detailcontr1,contr_radius,0)
 \                :   NOP
detailcontr1 != 0 && (shrpcnt==1 || mskcnt>1 && shrpcnt>=mskcnt) && (EQ>0) 
 \                ?   MaskedMerge(last,temp,edge0.expand.inflate.inflate, Y=3,U=1,V=1,useMMX=true)
 \                :   NOP
mskcnt = (detailcontr1 != 0 && (shrpcnt==1 || mskcnt>1 && shrpcnt>=mskcnt)) 
 \                ?   mskcnt - 1
 \                :   mskcnt
shrpcnt = detailcontr1 != 0 ? shrpcnt - 1 : shrpcnt

# Stage 2 :  UnsharpMasking, small radius ----------------------
detailcontr2 != 0 
 \                ?   unsharpmask(detailcontr2,           1,0)
 \                :   NOP
detailcontr2 != 0 && (shrpcnt==1 || mskcnt>1&&shrpcnt>=mskcnt) && (EQ>0)
 \                ?   MaskedMerge(last,temp,edge0.inflate.inflate,Y=3,U=1,V=1,useMMX=true)
 \                :   NOP
mskcnt = (detailcontr2 != 0 && (shrpcnt==1 || mskcnt>1&&shrpcnt>=mskcnt))
 \                ?   mskcnt - 1
 \                :   mskcnt
shrpcnt = detailcontr2 != 0 ? shrpcnt - 1 : shrpcnt

# Stage 3 :  per-pixel sharpening ------------------------------
PixSharp   != 0.0 
 \                ?   sharpen( PixSharp )
 \                :   NOP
PixSharp   != 0.0 && shrpcnt > 0 && EQ>0
 \                ?   MaskedMerge(last,temp,edge0, Y=3,U=1,V=1,useMMX=true)
 \                :   NOP
undot()

#---------------------------------------------------------------------------------------------------------------------
# DeRing'ing of sharpen'ed clip: +++ OLD ROUTINE (Post-processor) +++
#
dering > 0 ? MaskedMerge(last,last.unfilter(-dering,-dering),edge1b.bicubicresize(xx_ss1,yy_ss1),Y=3,U=1,V=1,useMMX=true)
 \          : NOP

# Now I have the other idea - but it's too slow, yet
antiflicker2 == true ? temporalsoften(1,2,3,23,2) : NOP

#---------------------------------------------------------------------------------------------------------------------
# 2nd supersampling stage to perform SubPel operation & XSharpening at
#
# Notes: Luma Warping  - may help for sources that are hard to get a "clean" picure from.
#                      - Should help making the picture a little more "gracile" when doing DVD -> HDTV upsizing 

xx_ss2 != ox || yy_ss2 != oy ? LanczosResize(xx_ss2,yy_ss2) : NOP

subpelstren != 0.0 ? blur(subpelstren) : NOP

warp_Y ? awarpsharp(depth=(ss2_x+ss2_y)/2*3,thresh=0.5,blurlevel=2,cm=0) : NOP

Xstren != 0 && Xlimit !=0 ? xsharpen(Xstren,Xlimit) : NOP

LanczosResize(dest_x,dest_y).undot()

#---------------------------------------------------------------------------------------------------------------------
# Build new EdgeMask of the enhanced clip
#
edge2 = bicubicresize(int(dest_x/1.75/16+.5)*16,int(dest_y/1.75/16+.5)*16,1.0,.0)
 \     .DEdgeMask(0,255,0,255,"-5 -7 -5  -7 48 -7  -5 -7 -5",Y=3,U=1,V=1,useMMX=true,setdivisor=true,divisor=1)
bordermask = blankclip(edge2,width=dest_x-16,height=dest_y-16,color=$FFFFFF)
 \          .addborders(2,2,2,2,color=$8F8F8F).addborders(2,2,2,2,color=$6F6F6F)
 \          .addborders(2,2,2,2,color=$202020).addborders(2,2,2,2,color=$000000)
edge2 = edge2 
 \     .levels( (detail_floor+int(detailcontr1/15+detailcontr2/30+PixSharp*15)), 4.0,128,flatweight,255,false)
 \     .bicubicResize(dest_x,dest_y,1.0,0)
edge2 = exborder ? edge2.logic(bordermask,"min") : edge2
edge2 = edge2 
 \     .inflate(Y=3,U=1,V=1).inflate(Y=3,U=1,V=1)
 \     .blur(1.58).greyscale()
 \     .inflate(Y=3,U=1,V=1).inflate(Y=3,U=1,V=1)
 \     .temporalsoften(1,63,63,255,2)

#---------------------------------------------------------------------------------------------------------------------
# Put original de-noised & not-sharpened clip into areas still appearing flat, after all that sharpening 
#
MaskedMerge(dusted.lanczosresize(dest_x,dest_y),last,edge2,Y=3,U=1,V=1,useMMX=true)

#---------------------------------------------------------------------------------------------------------------------
# Chroma Warping - often useful for sources where colors are "bleeding". Superfluid for clean sources.
#
warp_UV ? MergeChroma( aWarpSharp(depth = 8.5 * (dest_x/ox + dest_y/oy), thresh=0.5, cm=1, blurlevel=2) ) : NOP

#---------------------------------------------------------------------------------------------------------------------
# Visualizations
#
debug == "dering"   ? overlay(last,blankclip(last,color=$00A000),mask=edge1b.bicubicresize(dest_x,dest_y),mode="blend",opacity=dering_weight)
 \                  : NOP
debug == "protect"  ? overlay(last,blankclip(last,color=$00A000),mask=edge0.bicubicresize(dest_x,dest_y),mode="blend",opacity=0.75)
 \                  : NOP
debug == "detail"   ? overlay(last,blankclip(last,color=$00A000),mask=edge2.bicubicresize(dest_x,dest_y),mode="blend",opacity=0.75)
 \                  : NOP
debug == "compareH" ? stackhorizontal(clp.lanczosresize(dest_x,dest_y).crop(cropx,0,-cropx,-0).addborders(0,0,4,0,color=$800000).subtitle("Original"),
 \                                    last.crop(cropx,0,-cropx,-0).subtitle("iiP") )
 \                  : NOP
debug == "compareV" ? stackvertical(clp.lanczosresize(dest_x,dest_y).crop(0,cropy,-0,-cropy).addborders(0,0,0,4,color=$800000).subtitle("Original"),
 \                                  last .crop(0,cropy,-0,-cropy).subtitle("iiP") )
 \                  : NOP
debug == "showall"  ? stackvertical( stackhorizontal( edge0 .greyscale().lanczosresize(dest_x,dest_y).crop(cropx,cropy,-cropx,-cropy)
 \                                                          .addborders(0,0,4,4,color=$800000).subtitle("Protection of sharp edges"),
 \                                                    edge1b.greyscale().lanczosresize(dest_x,dest_y).crop(cropx,cropy,-cropx,-cropy)
 \                                                          .addborders(0,0,0,4,color=$800000).subtitle("DeRing'ing") ),
 \                                   stackhorizontal( edge2 .greyscale().lanczosresize(dest_x,dest_y).crop(cropx,cropy,-cropx,-cropy)
 \                                                          .addborders(0,0,4,0,color=$800000).subtitle("Detail areas"),
 \                                                    last  .crop(cropx,cropy,-cropx,-cropy).subtitle("iiP") ) )
 \                  : NOP

return(last)
}

#
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th June 2004, 17:18   #106  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Parameter description (only parameters new in iiP v.0.5)

EQ [int]

Edge "quality" parameter.
Specifies how many of the sharpening steps will be protected by the protection mask.

0 = deaktivated. Don't ever choose that for encoding!! - but it's nice for visualizing why edge protection is needed at all
1 = protect only the last sharpening step. "Standard", as in older versions.
2 = protect the first & the last sharpening step. Recommended.
3 = protect all three sharpening steps. Minimalizes any risk of producing oversharpening, but lessens the overall contrast gain.


warp_Y, warp_UV [bool], [bool]

These will activate additional luma warping, resp. chroma warping. (Uses MarcFD's aWarpSharp plugin.)

Luma warping - should help in making the picture a little more 'gracile' when upsizing to bigger resolution, e.g. 1:1 PAR -> anamorphic PAR, DVD -> 720p, or such. For keeping same resolution or downsizing, perhaps it may help on sources with distorted detail by "straightening" it. Or it may not help - YMMV.
Luma warping is performed in 2nd supersampling stage, prior to XSharpening.

Chroma warping - should help a lot on sources with bleeding colors. I put this in, since I had big problems on my Enterprise DVB captures: on the starfleet uniforms, the red stripes are badly bleeding into the blue cloth. This looks *very* ugly, like an aquarell that's gotten wet. It looks much better with chroma warping.
On clean sources, chroma warping mostly is a waste of CPU cycles.

Note: aWarpSharp's parameters are hard-coded. Depth is automatically scaled by the supersampling factors (Y) resp. by the [original:destination] size ratio (UV).
If someone thinks more configuration is needed, tell me. For me it works fine the way it is.


New deringing routine

Now you can choose between to routines for deringing:

The "old" routine can be used as formerly. Parameters have not changed. Note that this routine is a post-processor, after the sharpening, but prior to XSharpening.

The new routine uses another way to create the deringing mask (utilizes XSharpen to find areas-of-interest). The current processing method for the found areas is very sophisticated: "Downsize(softbicubic).Upsize(verysoftbicubic)".

It acts as a pre-processor, prior to the main sharpening steps, but already in the first supersampling stage.

To use the new deringing routine, assigning a negative value to the dering parameter.

The parameters dering_floor and dering_bias behave differently with this routine. It's a bit difficult to describe ...
dering_floor is sort of a sensitivity against edges: the bigger this parameter, the more areas around weaker edges will be considered.
dering_bias can be seen as some sort of multiplier, or normalizer ... how to tell? (Internally, it's the "divisor" parameter for MaskTool's "DEdgeMask" function.) The smaller dering_bias, the brighter the mask will get, and the more areas found by dering_floor will be kept. The bigger dering_bias, the more weaker areas will get cancelled from the mask again, and the darker the dering-mask will become.

To get a feeling for this, please play a little with these two parameters & the new routine, while having set debug="showall". A good starting point is floor=8, bias=12. It seems to me that bias should mostly be in the range bias = [floor, floor*2] - but who knows what different sources may require. This one is new to me, too

dering = -200 ~ -1 tells how strongly the affected areas will be blurred through the bicubic resizers. Recommended range is from "-160" for strong processing to "-20" for weak processing.
Attention: If you use a too big number here, you'll get artefacts near edges! Try "dering = -350" to see what I mean. You should not exceed -160, perhaps evev -200. Look out when artefacting starts to appear. "dering = -120" gives already pretty strong processing.

The dering_weight parameter is not used with the new deringing routine.


exborder [bool]

This is a switch to exclude the outmost border from sharpening.

It is not uncommon, even for otherwise HiQ DVD sources, to have artefacts in the border areas. You now these bright and/or dark lines that often appear near the top/bottom letterboxing, and on the frame's sides as well.
But, not everyone is willing to crop these additional 2 ~ 8 pixels from the sides, to get rid of that crap. Now, if you're cropping so tight that source's border artefacts are kept, then at least they shouldn't be sharpened
Set exborder=true to do so. The used masking draws a "sinus sweep" (sort of) from 100% exclusion to ~15% exclusion, over the outmost 8 pixels on all sides of the frame.
There is nothing to configure. Tell me if you want something.


detail_floor [int]

This is part of the controlling how weak a detail is still considered to be "detail", in order to be copied to the output frame from the sharpened clip. Remember: areas considered to be "no-detail" are copied from the PixieDust'ed, not-sharpened clip.
Look at the "Detail Mask" in debug mode to get a feeling for it. Former versions had this parameter hardcoded to "23". New default is "20". Negative values are possible, in case you consider them needed.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th June 2004, 17:19   #107  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
(Happy reading ;-) )

Explanations, thoughts, tips and (speed) considerations

- or -

Didée's usual, dreadfully boring blabla



Regarding the EQ parameter:

iiP performs up to three chained sharpening steps: UnsharpMasking-wide + UnsharpMasking-narrow + PixelSharpen.
Generally, any unprotected sharpening endangers oversharpening of high-contrast edges. With chained sharpening, it gets even worse: the first sharpening greates artefacts, the second one sharpens also those artefacts, the third one ... you get the idea. Without any protection, you'll get those ugly halos, and you get them even with additional borders, themselves emphasized:
first sharpener gives halos, second sharpener gives those halos a hard border, the third makes everything even more annoying.

Former iiP versions internally created exactly such an over-oversharpened image, and used only one MaskedMerge() operation after all sharpening, to exclude hard edges from getting copied from the sharpened clip into the working clip.
Although multi-protection was on my mind from the very start, I wanted to avoid the additionally needed processing time. And I thought the one-step protection would be sufficient to exclude all oversharpening artefacts. Well, it is mostly, but not always. Previous iiP versions let some artefacts sneak through, especially when strong sharpening values were used.

With v.0.5, multiple protection may be specified through the EQ parameter.

EQ=1 is exactly the same as formerly. Everything is sharpened-up without protection, and then copied to the working clip according to the protection mask.

The most "exact" way to do the image sharpening is to protect *every* step by the protection mask. This is done by specifying EQ=3. The result should be close to totally free of oversharpening artefacts produced by iiP itself. Oversharpening already present in the source is anther story -> DeRinging.
With EQ=3, the working chain is " 1.sharpening -> MaskedMerge -> 2.sharpening -> MaskedMerge -> 3.sharpening -> MaskedMerge ".

However, EQ=3 lessens the possible gain of detail contrast quite a bit. From my experience so far, EQ=2 is a very good compromise.
EQ=2 will protect the first and the last sharpening step - whichever those might be, in case you are using only two of the three possible sharpening steps. (Sounds simple, and basically it IS simple. However, to make this work correctly for all possibilities of choosing only two of the three sharpeners, required some more lines of code than I expected...)
EQ=2 should give the best trade-off between speed and edge quality. There should hardly be any artefacts coming through, and the contrast gain is definetly better than with EQ=3. Hence I recommend to use EQ=2 - unless you figure out that your particular source needs EQ=3, or can get away with EQ=1.

EQ=0 is a major bad idea for encoding, of course. However, by setting EQ to zero, it gets blataneously obvious why all this protection stuff is needed at all. Isn't it?

Needless to say: the higher EQ you use, the slower the script will run. Each EQ increment requires another instance of MaskedMerge()'ing the luma plane over - and this is on the 1st supersampling stage, so this operation is not for free.

OTOH: the higher EQ you use, the better compressability you'll get for the output.



(New) Deringing Routine

The biggest problem with self-made deringing is: to create a mask that "fits" really good onto the problematic areas.

Firstly, we want to catch as much of the problematic areas as possible, preferebly all of them, only them, and nothing else.

Secondly, while catching all of them, it must be avoided to mask *across* any edges in the image. We want the mask to be present in the "ultimate neighborhood" of hard edges, to catch halos and mosquito noise. In areas where the mask *covers* the edge, we would force the edge itself to get blurring (depending on the used processing mthod, that is). This is probably not what we want to do.

Thirdly, there is the question of which kind of processing should be actually performed on the masked areas.

(1) Is still unsolved. There's no distinction between "ah, here are edge artefacts, we must apply deringing here" and "yup, this area is clean, no deringing is needed". At least for edge halos, there are ideas of how to do indeed *that*. But it's not possible to do that in a script - if you can code plugins, and have a little spare time, just ask me. Ideas is what I have PLENTY.

(1)+(2) - The new deringing routine utilizes XSharpen to find areas-of-interest, basically by taking the clip, and subtracting an XSharpen'ed version from itself. On this difference image, a quasi-Sobel edge detector is performed. Finally, everything gets extended and feathered a little - there is the new deringing mask.
It has a definetly better respose to edge areas than the old method, which was based on an UnsharpMask - difference image. Still the new method's response to hard edges is greater than to weaker edges. The danger of masking-across-edges is magnitudes lower than with the old method. However, the "tightness" at which the new mask is "fitting" to either side of the edges is not perfectly distributed - but I consider it quite usable as it is. Further improvement might be possible.

(3) The currently used processor for deringing is the most lame one can think of:
"BicubicResize(small,soft).BicubicResize(orig,verysoft)" is what actually is done to the dering'ed areas - and this works surprisingly effective, handling both halos and mosquitos pretty well, without getting all too obvious. There is danger, of course, of loosing small detail that is present in the direct neighborhood of strong edges. (How do you want to distinguish small detail from mosquito noise - to a machine it looks the very same).
The strength of dering-processing defines at what scale the image will be downsized, before upsizing again. "dering=-150" will scale down by a factor of ca. 2.8, "-100" by ca. 2.2, "-50" by ca. 1.6. Values numerically greater than 160 might cause artefacts. Try "-350" to force those artefacts. Then, knowing how they look like, you know what to look for if you're forced to use very high settings.
Values up to "-150" should be safe, and strong enough for almost all cases. "-40" to "-80" should be a good range for rather clean sources - but let your own eyes decide.


As already stated, the old routine can be used just as formerly. The new routine is invoked by specifying negative values to the dering parameter.

It should be mentioned that the old routine acted (and still acts) as a *post* processor. That is, after the main sharpening, but before the XSharpening. Together with the fact that it partly masks *across* edges, it was used to fight against source-artefacts as well as artefacts created by iiP itself (through too little edge protection). The blurring of some edges then was (partly) restored by the following XSharpening.

The new routine now acts as *pre* processor, i.e. before any sharpening is applied (but after the base denoising). That's the preferred way to do it, now that EQ is available.


EQ and New Deringing

The new deringing is *noticeably* slower than the old one. That's just how it is, and there's little that can be done.
You should carefully check your source, if it needs deringing at all. Since selfmade artefacts now can be avoided through higher EQ values, perhaps you simply don't need any deringing!
In particular, I would not recommend to use the new deringing with strong values, together with EQ=3. This will result in very slow processing, and will give an output that is ... smooooth. Of course there might be sources out there that require exactly this. But generally, you should not go higher than EQ=2 when using somewhat strong deringing. With low or medium deringing, I'd say EQ=2 is the one to use. With strong deringing, and/or when using some lower sharpening values or less sharpening steps, you might be well off with EQ=1 as well.

All I can advise you to do is this:
Watch out for different problematic scenes, and then figure out which combination of features copes the best with all of them, on average.


Luma Warping

Well, not much to say here - exept that it uses MarcFD's "aWarpSharp" plugin, which you should have available. Try it, and see for yourself if you like it or not. Or if you can see any difference at all.
The effect should be minor, and that's how it is supposed to be. Assumingly, some people would like to hit me for the mere fact that I'm using WarpSharping on natural video soures. However, it is performed in the "big" 2nd supersampling stage, with a relative small depth value.
The intended effect is to get some minor thinning/straightening of image features, and it is mostly meant to be used when you actually *upscale* your clip through iiP. When you're doing the old "anamorphic 720*yy --> 1:1 640*yy" game, then there is probably not much to gain with lume warping. But try and see yourself.

Thanks to the fact that MarcFD managed to get his aWarpSharp pretty fast, the impact on speed is not so bad.


Chroma Warping

This one is not intended for general usage. But you might come across sources where it can do small wonders!
I implemented it for sources with bleeding colors, and there it generally achieves very noticeable improvements. One example: For the ones that know the (new) Enterprise series (Cpt. Archer, T'Pol, Dr. Phlox ...), you might have noticed that the blue uniforms with red stripes are particular difficult - the stripes are "bleeding" into the blue cloth like ink flows into ink paper For me, chroma warping does a brilliant job there.

UV warping is performed on the final output clip, on destination resolution. It use chroma mode 1 - "warp chroma with luma bump map".

This one also is not too expensive, performance-wise.

I think it doesn't make much sense on souces with clean colors (but mf might disagree...)

***

Ah, I'm out of time now.

If you have any questions, go ahead. I'm all ears.


- Didée
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th June 2004, 18:04   #108  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Wouldn't it be better to just link to a premade avs instead of posting the whole script?
Chainmax is offline   Reply With Quote
Old 28th June 2004, 18:38   #109  |  Link
malkion
Registered User
 
Join Date: Feb 2004
Posts: 156
Chainmax, I dont think Didee wants to setup linkage space.. I find it insignificant either way.

Didee, is there any possible hints you could provide to help us narrow the parameters for the various floors and bias settings with the debug view? Also if it helps to set the various other parameters, like the newer dering parm, I mean, what do you personally use the debug view for? I noticed prior to this version you had coded some yuv analyzing which you could call on to check out the video (or was it on limitedsharpen). Also, what are the usage for the current script's optional coding? such as the following:

1) the #ylevels call on the old dering routine.

2) edge00 = edge00.yv12lut(yexpr="x 128 - 2 *").expand(Y=3,U=1,V=1).blur(1.58)

this line is found in the build edgemask for protect floor routine. edge00 is no longer called in the debugview section.

these 2 are from the new 0.5a version. I guess I saw some more from a old version no longer worth mentioning.

there are some similar #calls in the limitedsharpen script, but thats for a different thread. I'll be doing further tests of course. These questions arent that important since it has nothing to do with the actual video encoding.
malkion is offline   Reply With Quote
Old 28th June 2004, 19:09   #110  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Quote:
Originally posted by Didée
masktools-v1.4.16.zip
Sorry, it was late n' there are way too much MaskTools versions on my HD !!!


By a first small preview, I was able to spot out some ugly blocks...

Think they are related to PixieDust !!!


Btw, the next eye riddle...


Bye
__________________

Visit my IRC channel

Last edited by Soulhunter; 28th June 2004 at 19:56.
Soulhunter is offline   Reply With Quote
Old 28th June 2004, 21:36   #111  |  Link
malkion
Registered User
 
Join Date: Feb 2004
Posts: 156
You can try adding .BlindPP(quant=8,cpu2="xxxxoo",moderate_h=40,moderate_v=60)

after dusted. may not help during preview, but according to some people
it can catch most blocks during encoding.
malkion is offline   Reply With Quote
Old 28th June 2004, 22:47   #112  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Yes, but this introduces too much quality degradation IMO...

Think Deen would be a good replacement for PixieDust !!!


Bye
__________________

Visit my IRC channel

Last edited by Soulhunter; 28th June 2004 at 22:51.
Soulhunter is offline   Reply With Quote
Old 29th June 2004, 09:32   #113  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Quote:
Originally posted by Soulhunter
[B]Sorry, it was late n' there are way too much MaskTools versions on my HD !!!
Heheh, yes, same on my side. That's why on my HD, *all* versions of MaskTools are named with version number: "MaskTools_v1.4.16.dll"

Quote:
By a first small preview, I was able to spot out some ugly blocks...
Think they are related to PixieDust !!!
Yes and yes. Old "problem" of PixieDust - I don't want to discuss this all over again. I'd say, it's a trade-off you have to pay for the (currently) unreached, unbeaten, unbelievable best overall denoiser+smoother we have available for usage. /*bites lips to not start lenghty elaborating*/

You should actually try something like malkion suggested:
Quote:
by malkion
You can try adding .BlindPP(quant=8,cpu2="xxxxoo",moderate_h=40,moderate_v=60)
Previously, I even had a similar line in the script, it was only outcommented.
Should really go in as a configurable option, so you can either use it or not, without having to mess with the script.

IMHO, s-th like BlindPP(Quant=12,cpu2="xx??oo",moderate_h=45|50|55, moderate_v=55|60|65" doesn't hurt the output noticeably (iiP still is to come after this line ), but helps a lot against the blocking. It won't vanish, but it gets much better.

Quote:
by Soulhunter
Think Deen would be a good replacement for PixieDust !!!
If you think so - go ahead and just try it! Very easy to do:

Just set "duststr=0", then PixieDust is completely disabled. As replacement, call your denoiser-of-choice immediately before iiP():

...
Deen(whatever)
iiP( 8064, 6048, duststr=0, ...)

I am curious what results you'll get, regarding quality, and not less regarding compressability
To have said that: I have close to zero experience with Deen. Why? Because its basic working principle is the same as Convolution3D. And the latter is one of the filters that disappeard (almost) completely out of my scripts, over time.

As said, right now I'm not in the mood to discuss PixieDust again ...
But I do have my certain reasons to use PixieDust exclusively, this you can believe.
Every heard something about "noise cutoff"? Tip: Think about the general principle of limiting thresholds ...


Preparing some more tips now.


- Didée
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 29th June 2004 at 09:41.
Didée is offline   Reply With Quote
Old 29th June 2004, 12:43   #114  |  Link
malkion
Registered User
 
Join Date: Feb 2004
Posts: 156
On my PC, I've noticed as well, that if you're encoding to anything greater than original DVD resolution (havent found the threshold yet) with pixiedust, the result clip is borked. I tried PixieDust on a HDTV source, and it didn't go too well. On normal DVD resolution source, it's fine. PixieDust doesn't like a source at 1280,720.
malkion is offline   Reply With Quote
Old 29th June 2004, 13:23   #115  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Chainmax:

Better it would be, yes.

In front of my mind's eye, I see a little website, holding all my scripts, nice colorful manuals for them, and preferably several dozens of MB's for offering demo clips to download.

You organize the webspace, and I'll upload.


malkion:

Indeed. All Dust flavours by Steady are limited by framesize. But remember, Steady appeared out of nowhere with his Dust package by a time when Conv3D was "the magic filter" - early 2003, that was. Times have changed a lot since then. By then, we thought not very much about processing HDTV sources. For sure we didn't think about *motion compensated* processing of such big framesizes ... not to speak about encoding
A big pity is that you can't call Dust more than once on the same clip ranges, because of some internal variable's stuff, as it seems. Otherwise you could crop with a little overlap, blow the dust over it, and re-stack it together.
If you definetly want to do it, you have to create four avisynth scripts, each calling the same source, each cropping one quadrant+overlap, dust it, crop overlap, return it. Then you create a script that calls these four scripts by *AviSource*, and stack them together.
With a little luck, it might work.

Hmmh ... time to check out Manao's mvtools - although I've the feeling they still have some way to go to reach Dust's level, as far as denoising is concerned. Actually I can't tell.

[edit]

still to malkion:

Excuse me, but ...
Code:
I noticed prior to this version you had coded some yuv analyzing which 
you could call on to check out the video (or was it on limitedsharpen)
I did ... WHAT ?

I have no idea what that should have been. Please explain closer. Do you by any chance mean an optional "colorYUV(analyze=true)" call to show the used level ranges?

Oh, and "edge00" is only a relict from developping, it is not used at all. Moreover, the usage of edge00 in the debug section was ... a li'l mistake


- Didée
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 29th June 2004 at 14:13.
Didée is offline   Reply With Quote
Old 29th June 2004, 13:25   #116  |  Link
Sirber
retired developer
 
Sirber's Avatar
 
Join Date: Oct 2002
Location: Canada
Posts: 8,978
Could you do a ZIP with all needed stuff? Thanks
__________________
Detritus Software
Sirber is offline   Reply With Quote
Old 29th June 2004, 13:42   #117  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,607
didée:

jep, multiple calls of ...dust doens'Ät work, but what about this? :


a= crop(upperleft)
b= crop(lowerleft)
c= crop(upperright)
d= crop(lowerright)

a+b+c+d

pixiedust(..)

a= trim(upperleftframerange)
b= trim(lowerleftframerange)
c= trim(upperrightframerange)
d= trim(lowerrightframerange)

stackhorizontal(stackvertical(a,b),stackvertical(c,d))
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 29th June 2004, 13:44   #118  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Quote:
Originally posted by Sirber
Could you do a ZIP with all needed stuff? Thanks
So that you can feed iiP's output into any sort of RealMedia encoder?

Do you think I'm CRAZY?



Sure, Sirber.


Scharfi:

Ah, interesting idea. Worth trying out. (But I put no money on it.)

Scharfi: The (failing) sample, please ...

- Didée
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 29th June 2004 at 13:49.
Didée is offline   Reply With Quote
Old 29th June 2004, 13:59   #119  |  Link
Sirber
retired developer
 
Sirber's Avatar
 
Join Date: Oct 2002
Location: Canada
Posts: 8,978
RealVideo, RealVideo, RealVideo!!!!!!!

Want me to host you?
__________________
Detritus Software
Sirber is offline   Reply With Quote
Old 29th June 2004, 20:24   #120  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Quote:
Originally posted by Didée

Deen(whatever)
iiP( 8064, 6048, duststr=0, ...)
Hehe, this is what I did before...

But I wasn't sure if your script doest need PixieDust's processing to do some other funky stuff !!!

Quote:
Originally posted by Didée
I am curious what results you'll get, regarding quality, and not less regarding compressability
When I find the time, I will post some samples...


Btw, progress with the new eye riddle ???


Bye
__________________

Visit my IRC channel
Soulhunter is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.