RoboCrop, v1.13 - 07 Feb 2020

[StainlessS]

RoboCrop v1.13, v2.58, avs/+ v2.60, x86 and x64 dll's.
Requires VS2008 CPP Runtimes.

V1.13 Docs overflow 16kb limit, so left at v1.0 docs below.
See post #37 for v1.13 Docs over two posts. Here: http://forum.doom9.org/showthread.ph...03#post1696703

RoboCrop()

To Protect And Serve - by StainlessS @ Doom9:- http://forum.doom9.org/showthread.php?t=168053

VIDEO: Planar, YUY2, RGB.
AUDIO: No change.

Plugin's for both Avisynth v2.5 and v2.6.

RoboCrop is an automatic cropping solution to crop black borders from video clips, loosely based on (but using no code from)
Autocrop by Glenn Bussell.

For the most part, you can just call it with RoboCrop(clip) and not bother with any other arguements, it is intended to be pretty
much autonomous in its decisions. You might however want to alter eg WMod and HMod if your encoder has special requirements, and
perhaps Laced if your source is likely to be fully progressive.

Borders are detected by sampling at Samples frames, at scanlines (h/v) using AverageLuma (RGB is converted to Luma Y at either TV or
PC levels, See Matrix). This sampling is done on all 4 sides of the frame.
If a scanline Average luma is below or equal to Thresh, then is considered possible black border, above considered possible image,
if Baffle [default=4] adjacent scanlines above Thresh, then it IS image.

Function RoboCrop(clip c,int "Samples"=32,Float "Thresh"=-32,bool "Laced"=true,int "WMod",int "HMod",int "RLBT"=15,bool "Debug"=false, \
    float "Ignore"=0.2,int "Matrix"=(c.width<=720?2:3), int "Baffle"=4, bool "ScaleAutoThreshRGB"=true,bool "ScaleAutoThreshYUV"=false, \
    int "CropMode"=2,bool "Blank"=false,bool "BlankPC"=false,bool "Align"=false,bool "Show"=false,String "LogFn"="",bool "LogAppend"=false, \
    Float "ATM"=4.0)


 Args:-

   Samples=32, Number of frames to sample from source clip c.
    As an observation, most clips have good border recogition within the 1st 2 sampled frames using the default -32.0 AUTOTHRESH
    although have noticed some dark clips that required ~8 samples (maybe more) for full recognition @ default Thresh = -32.0
    We use a default Samples=32, because we are intrinsically paranoid.
    If clip more than 250 frames, will ignore the first 5% and last 10% of frames when both auto-thresholding and crop sampling to
    try to negate effects of artificial black in titles and end credits. (Even without this feature, it always skips beginning and end
    where possible like this "00100100100" where 1 represents sampled frames when sampling 3 frames from 11).

   Thresh: Default= -32.0 (==DEFAULT AUTOTHRESH, any -ve Thresh is AUTOTHRESH where Thresh adjustments are automatic).

    Thresh > 0: (Explicit Threshold)
      A user supplied +ve Thresh should be at correct TV/PC levels for the the clip being worked on ie 16->235 for TV levels and
      0->255 for PC Levels (RGB, as appropriate for matrix being used).

    Thresh <= 0: (AUTOTHRESH)
      When Thresh <= 0, the clip will be sampled over Samples frames to find the minimum YPlaneMin (using Matrix if RGB) which
      we will call MINLUMA and an Explicit Threshold calculated by adding MINLUMA to abs(Thresh), after that it is processed
      as for Thresh > 0: (Explicit Threshold) as noted above, but, before adding MINLUMA, some AUTOTHRESH massaging occurs.

      Here AUTOTHRESH Thresh massaging occurs in sequence:-
      1 ) if (Thresh == DEFAULT AUTOTHRESH && ATM < 32.0) (DEFAULT AUTOTHRESH = -32.0, defaulted OR user supplied):
                Let FrameRange = FrameEnd-FrameStart+1 (after skipping titles and end Credits, also Samples limited to FrameRange).
                Samples_massage = (Samples>=16)        ? 1.0 : (Samples-1)    * (1.0/(16-1))
                Range_massage   =(Float(frmend-frmstart+1)/samples >= 250)  ? 1.0 :(Float(frmend-frmstart)/samples * (1.0/(250-1)))
                Thresh = -(((Samples_massage * Range_massage) * (32.0 -ATM)) + ATM)
                  The adjustment to Auto Thresh is to reduce the possibility of overcropping on eg a dark low 'Samples' clip, or where
                  source FrameRange (ie temporal frame set) is too small to take a reliable sample from.
                  Resulting massaged Thresh will range between -ATM and -32.0.
                  Although massaging is intended to reduce overcropping, it could result in not cropping enough border (less disastrous),
                  its a bit of a balancing act really. See also ATM.
      2 ) If RGB AND PC matrix(default) AND ScaleAutoThreshRGB==True(default) then
            Thresh= Thresh*(255.0/(235-16))
      3 ) If YUV AND ScaleAutoThreshYUV==True(default=false) then
            Thresh= Thresh*(255.0/(235-16))

      Steps 2) and 3) above, by default treat a -ve AUTOTHRESH as being @ TV Levels and so Scale RGB Thresh to PC levels but not YUV.
      If you want to supply a PC levels AUTOTHRESH Thresh for RGB, then simply set ScaleAutoThreshRGB=false to inhibit scaling.
      Note, if a TV levels Matrix is supplied for RGB, then scaling will always be inhibited.
       If your clip is YUV at PC levels and you want to use eg DEFAULT AUTOTHRESH (-32.0, which is considered to be @ TV levels),
      then set ScaleAutoThreshYUV=True to enable Thresh scaling.
       If your clip is YUV at PC levels and you want to use a PC levels AUTOTHRESH (-ve) then leave ScaleAutoThreshYUV at default false
      which does not scale Thresh.
      After any scaling, MINLUMA is then added to abs(Thresh) and processed as for +ve Explicit Threshold as noted above.
      NOTE, RoboCrop 'massages' DEFAULT THRESH (exactly -32.0) if low samples count or if short clip. Reason being to avoid
       overcropping when insufficient data available for reliable cropping. It is considered better to not crop enough or at all, than
       to overcrop. You can override by simply setting an explicit threshold (+ve) of eg 40.0, or setting a NON-DEFAULT auto thresh (-ve)
       eg -16.0 or -32.1, where YPlaneMin is established for the sampled frames and then abs(thresh) is added to that value which is
       then used as an explicit thresh.

   Laced:, Default=true, true for Interlaced.
          RoboCrop automatically deduces colorspace cropping restrictions and sets internal XMod and YMod,
          eg XMod and YMod would both be set to 2 for YV12, for YUY2 Xmod=2, YMod=1, etc.
          If Laced==true, then internal YMod is doubled so as not to destroy chroma.
          Below WMod and HMod are both defaulted to internal XMod and YMod respectively after the Laced hint is applied to YMod.

   WMod:, Default=The natural chroma cropping restriction of the colorspace concerned, eg 2 for YV12.
   HMod:, Default=The natural chroma cropping restriction of the colorspace concerned, BUT, doubled if laced=true.
    The above WMod,HMod sets rounding for legal cropping coords for colorspace concerned.
    WMod MUST be a multiple of internal XMod as described under Laced above, or it will throw an error.
    HMod MUST be a multiple of internal YMod as described under Laced above, or it will throw an error. If eg colorspace is
      YV12 then YMod would be set to 2, and if Laced, then YMod would be doubled to 4, so HMod MUST be a multiple of 4.
      Some encoders may require an WMod/HMod of eg 8 or 16, and if set thus, would crop off more or less depending upon
      which CropMode is set, if later resizing will be done, then encoder requirements can be satisfied during the resize.
      NOTE, Some players and VirtualDubMod (Helix YV12 decoder) dont like WMOD less than 4 (Vdub latest, OK, internal decoder).

   RLBT:=15=All Borders, Bitflags of edges to crop, 15 ($0F) crops all four. Each letter in the name 'RLBT' represents an edge and bit position
    starting with 'R' in bit 3 representing the value 8 (2^3=8). 'L' = bit 2=4 (2^2=4), 'B' = bit 1=2 (2^1=2), 'T' = bit 0=1 (2^0=1).
    To calculate the RLBT bitflags, for 'R'(Right) add 8, for 'L'(Left) add 4, for 'B'(Bottom) add 2, and for 'T'(Top) add 1.
    Add all of the bit flags together 8+4+2+1 (=15) crops all four edges, 8+4 crops only Right & Left, and 2+1 crops only Bottom & Top.

   DEBUG:=False=No Debug. Set True for debugging info, need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027

   Ignore:=0.2, Percentage of darkest pixels to ignore during AutoThresh scan to detect minimum luma pixel of all sampled frames.
     (ignores noise, as Threshold in YPlaneMin).

   Matrix:, RGB ONLY. For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
    Default for RGB is 2(PC601) if clip width <= 720 else 3(PC709) : YUV not used
    The defaults are for PC601 & PC709 range 0-255.
    So as to not require different AutoThresh for RGB, if clip c is RGB AND matrix is PC Levels AND Thresh < 0.0 and ScaleAutoThreshRGB=true,
    then Thresh will be scaled to RGB full range ie Thresh = Thresh * (255.0/(235.0-16.0)) ONLY when AutoThresh (ie Thresh < 0.0,
    YPlaneMin relative).
    When +ve Thresh is explicitly supplied (Thresh > 0.0) it is not scaled and assumed to be already correct range TV or PC levels.

   Baffle:=4, Number of scanlines (h/v) that must break threshold to be considered image (avoids noise). Does not usually need changing.

   ScaleAutoThreshRGB: bool default True. If true and RGB and Matrix at PC levels, and Thresh -ve (autothresh) then thresh will (after any low
    samples count scaling) be scaled to PC levels. By default, -ve Auto Thresh is considered to be at TV levels, and so will be scaled to
    to PC levels to match the matrix setting. If ScaleAutoThreshRGB is False Then autothresh is considered to be at PC levels and not scaled.

   ScaleAutoThreshYUV: bool default False. If true and YUV and Thresh -ve (autothresh) then thresh will (after any low samples count scaling)
    be scaled to PC levels. By default, -ve Auto Thresh is considered to be at TV levels, this allows you to change that assumption when
    source clips are at PC levels. If supplying PC levels clip and PC levels autothresh then leave this setting at false (no scaling).
    The above seemingly awkward  settings are due to having to deal with YUV @ TV levels and RGB @ PC levels with the possibility of
    YUV @ PC levels.

   Cropmode: default=2 == CropMore. 1 = CropLess. Range 1 -> 2.
     1 (CropLess) crops a little less border if the exact cropping coords do not comply with XMod/YMod/WMod/HMod as
       described earlier. May leave a little border edge.
     2 Default, (CropMore) crops a little extra where exact coords do not comply XMod/YMod/WMod/HMod requirements.
       You may lose a little of the image edges.

   Blank: Default False. If true, then Blanks the borders, ie sets border to RGB(0,0,0) or YUV(16,128,128) instead of cropping.
     Is overridden if Show == true.

   BlankPC: default false. If Blanking then blanks to YUV(0,128,128) instead of default YUV(16,128,128).

   Align: Default false. If true, then aligns frames in memory when cropping, for eg SSE2 16-byte alignment required by some
     filters, particularly smoothing filters (filters following after RoboCrop). See Docs for Crop().
     Only Applies if Cropping, ie Blank==false AND Show==false.

   Show: Default false. If true, then does no cropping, just shows a little info on frame. Overrides both Blank and Align.

   LogFn: Default "". Filename of Log file. Will output crop coords eg "10 16 700 544". Crop coords will be output whether
          cropping or not (Blank/Show).

   LogAppend: Default false. If true then appends log to any existing log file.

   ATM: Float default 4.0 (0.5 -> 32.0). Silently limited to valid range.
    ATM allows user to set the DEFAULT AUTOTHRESH massaging minimum. When eg samples = 1, then auto thresholding is of course quite
    unreliable and so auto thresh would be 'massaged' to -ve(ATM), other values of Samples below 16 will likewise have auto thresh
    massaged but to a lesser degree, linearly between -ve(ATM) for Samples == 1 and -ve(32.0) for Samples == 16.
    Previous (Fixed) default for ATM was 0.5, for maximum safety so that a single Samples scan would NOT overcrop. The new ATM arg default
    of 4.0 is a less paranoid safety setting which should in most cases work well but with a very short clip or eg single image then
    user might be better off giving the minimum ATM of 0.5. An ATM of 32.0 will switch OFF default auto thresh massaging.
    So long as your clips are about 6 mins or more (and Samples == default 32) then there will be no auto thresh massaging and current
    default is unlikely to need changed, with very short clips or reduced Samples count, then you might want to reduce ATM, for maximum
    paranoia, set ATM=0.5. See Thresh.


Cropping will likely fail on END CREDITS ONLY clips, where on black background, and will probably crop off the sides that are no
different in average luma to any letterbox borders, if you cannot see the borders, then neither can RoboCrop(), even setting the
auto Thresh to eg -1.0 or 0.0 is quite likely to fail. (See RLBT edge bitflags).

If cropping too much border, then increase Samples or reduce Thresh or lower ATM if short clip.
If not cropping enough border then border is being seen as image, increase Thresh.
To speed up, you may want to leave Auto-Thresh alone and reduce samples, but there is danger that it will not detect all borders (overcrop).
NOTE, The plugin AutoCrop() uses a Thresh of 40.0 and samples == 5 by default (No AutoThresh), but the base logic is not too dissimilar.

Final NOTE, the args to especially note are:- WMod, you might want to supply WMod=4 if using VDubMod (some players might try to download codec),
And Laced=False if you always process progressive or deinterlaced, And ATM set between 0.5 and 4.0 if very short clips/single-image.

StainlessS

Show=true v1.06

Show=true v1.10+

Bottom image above: Left edge, single dots line, other edges a bit wiggly ie cropping moved inwards [CROPMORE] to adhere to X/Y/W/H Mod requirements.

See Mediafire in sig.