ExBlend deblender - v1.04 - 18 Dec 2018

[StainlessS]

New Thread due to Forum restriction on editing thread title when over x number of years old.
Old version here:- https://forum.doom9.org/showthread.php?t=157337

Settings.txt Part 1 of 3 : Post #1 of 6

ExBlend() By Stainless Stephen [StainlessS on Doom9]

http://forum.doom9.org/showthread.php?t=157337
New thread:- https://forum.doom9.org/showthread.php?t=175948


ExBlend() v1.04 [Req VS2008 CPP runtimes]

YV12, YV16, YV24, YV8, YUY2 PROGRESSIVE source. Avisynth v2.5+ (YV12 & YUY2 only for v2.58 plugin)

ExBlend(), is a plugin originally written for telecined clips that were downsized from 720x480 to VCD 352x240.
The damage caused by this, looks similar to damage caused by blend Deinterlacing of telecined clips, which results
in a double blend, every five frames, GGGBBGGGBBGGGBB etc where 'G' is good and 'B' is blend.
Where damage is due to Deinterlacing of telecined clips, it is likely that a denoised detection clip is required
as although the damage looks quite similar, it is not exactly the same.

--------------------------------------------------------------------------------------------

Exblend(Clip,int "mode",int "pal",float "lockthresh",float "ithresh",int "show",string "ExBfile",bool "ver",bool "revip",
string "override",int "lv",int "dv",int "disp",clip "Dclip",int "CompUB",bool "Decimate",int "DecompIx",int "DecClpIx",
bool "Debug")


-------------------------------------
SOME DEFAULTS HAVE CHANGED IN V1.04 |
-------------------------------------



int mode Running mode Default 1

0 Free Running mode (Deprecated)
Mainly for those that might whinge about having to use modes 1 & 2.
In this mode it keeps a record in memory of the blend status of all visited
frames, if you eg scan forwards and then backwards you will get exactly the
same results as the 1st visit. Jumping about is also possible so long as
you have first scanned the entire file although some metrics will be wrong.
1 Scan and Store mode (Default)
As for Free Running mode but saves a copy of the scan status data when the
clip is closed ('ExBlend.XBD'). Use this prior to mode = 2.
Its pretty pointless having filters following ExBlend() in mode 1.
2 Pre-calculated mode
Loads in Scan Status Data saved by Scan and Store mode, this is particularly
useful to examine exactly which frames are unblended and you can jump about
without the possibilty of mismatched detections. You should still not jump
about if you are examining the BlendPair count and Ratio or other counts.
In mode 1, Scan & Store, Scan entire file (with or without metrics) and When
you have everything working right you can switch
to mode 2, (with metrics off). You now can add following filters to your hearts
content and ExBlend() should not take up too much CPU.
Switching from modes 1 to 2 may seems awkward but it is intended that
this filter be used in film restoration and so it is likely that many other
filters will follow this one, having a pre-calculated data file allows
Exblend() to use minimal CPU when adding further filters and checking the
resultant video. For the same reasons, Multidecimate() could also be a good
choice.
v1.03b, added native decimation in mode 2.

int pal -1, 0, 1 , Default 0, lets Exblend() know what the original frame rate was.

This sets the filters hints, after a blended pair is found, the detector will preset
3 (23.976/24 FPS) frames either side of the pair to HINTED meaning No Blend. If 'pal' (25 FPS)
is set to 1, it will preset 4 frames either side to HINTED. This helps it fall
into sync with sequence but may have slight delayed detection if the sequence goes
out of sync. If a frame has already been preset (visited) then the hint will not
be set, this allows hints to work when scanning both forward and backwards.
Not sure if PAL is actually useful or not, never seen any clip that exhibited this
problem with a good/blended signature of GGGGBBGGGGBBGGGG etc, where there are 4
good frames and 2 blends. As it did not add much difficulty to the plugin, PAL was
implemented just incase it was ever found to be useful.

-1 AutoPal mode. (MODE 0 ONLY)
If you are not sure what the original source was you can set PAL = -1 (MODE 0 Only).
If "Unblended:Blended Pairs" Ratio is between (4.0-0.01) and 5.0 at frame 1500 (1500=60*25
= 60 seconds @ 25 FPS), it will auto switch to PAL hints, the first 60 seconds
could contain the odd extra false blend pair detection.
If AUTO PAL is activated, the Metrics P (PAL) flag is hi-lited. It's
not a good idea to rely on this auto pal switch, you should set pal=1 when you
see it in orange. If autopal is activated and you later see the ratio rise above
5.0, then it would seem that the autopal detection was in error, this can happen
if the 1st 60 seconds contains a lot of static scenes with little movement, in this
case you should change to PAL=0 to disable the autopal detection and force it
to 23/24FPS original frame rate. Again, if the 1st 60 seconds contains a lot of static
scenes and it was originally PAL 25 FPS then the autopal may fail to detect and not
auto switch to PAL mode, in this case, ratio will later fall to above 4.0 and below
about 5.0. In this case you should change to PAL=1 to force PAL mode.

0, Forced 24 (or 23.976) FPS original frame rate (Default).
If the original source was 25 FPS and PAL is set
to 0, then this can reduce the effectiveness of ExBlend() due to it erroneously
detecting false blends where it should have been hinted, thus throwing it out of sync.

1, Forced PAL 25 FPS original frame rate. If the original source was 24 FPS and PAL is set
to 1, then this can DRAMATICALLY reduce the effectiveness of ExBlend() due to it hinting
out blends and overriding what should have been a blend detection. This will again, throw
it out of sync.

The R (RATIO) metrics flag will be displayed in orange at any time the ratio is not within expected
bounds, including during the AUTO PAL detection period (if it was a 25 FPS clip originally)
but will disappear once AUTO PAL has switched.


float lockthresh Range 0.0 (Off) to about 2.0, Default 1.0 (No longer considered EXPERIMENTAL, so default 1.0)

The lockthresh arg switches on the locking function of ExBlend().
Purpose being to try and latch on to the sequence when near static scene is encountered.
If the lock is activated, the 'L' Locked flag will be set in metrics.

NoLMN[f]=x.xxxx
L=y.yyyy LMX[f]=z.zzzz

I dont really wanna explain how locking works, it may
change if someone else can implement a better version. However, for locking
to take effect, some pre-conditions have to be satisfied. The frame must
be detected as non-blended, the previous frame should have been hinted, and at
least some of the conditions that would qualify as a blend detection must
be present, so not all frames qualify for the pre-conditions. If a frame does
qualify, then you may see something like L=y.yyyy which would indicate a
locking metric for the current frame, it is this metric that is compared to
LOCKTHRESH, if below or equal, then the frame is locked (converted to a blend
detection). If the frame is locked, the L flag will be set and the metric will
be Hi-lited. Once any frame has been locked, you may see LMX[f]=z.zzzz and
this shows the frame number and lock metric for the frame with the highest
locked metric, yet encountered. There may also be an indication for the frame
with the lowest lock metric that did NOT lock as in NoLMN[f]=x.xxxx
You may want to investigate all of the minimum and maximum flagged frames
both for locking and Ignore functions. Any frame that qualifies and would pass
the locking criteria will fail silently if the merge delta is above the
ithresh Ignore setting, it will not appear or be counted as an ignored frame.

If anyone would like to improve the functionality of lock, then please
be my guest.
This filter outputs a list of LOCKED frames which can be viewed using DebugView,
or may be output to a logging file.

v1.02, can now use eg 'disp=0' to view original blended clip along with metrics to
view frames flagged for investigation for either locking or ignore.


float ithresh 0.0 to 100.0, Default 3.0

There was a weird transition in the
original clip that prompted this filter to be written, it resulted in an even
more weird effect after unblending (looked sort of embossed). This is an
attempt to avoid future similar problems, but I dont have the original problem
video any more so cant really test it, also, the detection logic has changed
a little and so the it may no longer be a problem.
If merge Delta (MD) for a detected blend pair goes above this number, then the
blends will be ignored. I suppose that you could cut down on the number of false
blend detections by lowering this but it probably does not need it,
it works fairly well as it is. If MD exceeds ithresh, the 'I' Ignored flag will
be set and the detected blended pair will be ignored, and a little futher info
will be given when show=5. If there were any IGNORED blends, then it may be a good
idea to take a look at the offending frames both with ithresh at its setting
of 3.0 and with ithresh set to eg 100.0 where the suspect blend will not be ignored.

Written since above, the ithresh setting does seem to work quite well and it's
default setting of 3.0 may cut down on false detections but may IGNORE good blend
detections too, this is conservatively set so as not to allow any bad ones
through. A clip has been observed that went out of sequence every few minutes by
throwing in a triple blend, this ithresh setting seems to avoid the unpleasant
effects of bad detections at these points. You may need to raise this if many
IGNORED blends occur, perhaps in 'not so good clips'.
This filter outputs a list of IGNORED frames which can be viewed using DebugView,
or may be output to a logging file.

v1.02, can now use eg 'disp=0' to view original blended clip along with metrics to
view frames flagged for investigation for either locking or ignore.


dll's for Avs v2.58, v2.60/+ x86 and x64.
Zip includes 3 dll's + source + full vs 2008 project files (~214KB).
See MediaFire or SendSpace in Sig for Downloads

[StainlessS]

Settings.txt Part 2 of 3 : Post #2 of 6

int show Range 0-5
MODES 0 1, and 2 when Decimate == false:

0 Off (Default in MODE 2 only)

1 Show colored block in top left hand of clip when blend detected.

Blue block for 1st blended frame, Red for Second

2 Show blend detection only as in:-

FFF ExBlend a b ?1RPSHOLIMD

FFF shows the frame number

a & b show '-' No Blend, '1' 1st Blend, '2' 2nd Blend
a & b will usually be the same, 'a' is the used detection, which may
have been newly calculated or pre-calculated and stored in a file.
'b' is shown as calculated by the metrics, it is possible
that this is different from a pre-calculated detection.

? is the MODE flag which may be displayed as '¿' (an upside down question
mark) meaning Free running mode 0.
A '?', meaning Mode 1, scan and store mode, where ExBlend creates a precalculated
data file. '!' displayed means Mode 2, Pre-calculated mode, where it will use
data file saved in mode 1.

The '1' flag above, represents the DISP (Display) mode setting and can be
'0', '1', '2', or '3', (the missing mode '4' is not valid unless MODE 2 and Decimate==true
where metrics are different).
'0' represents, disp=0, the original blended clip is returned.
'1' represents, disp=1, ExBlend default returning an identical pair
of unblended frames replacing the blended ones.
'2' represents disp=2, which returns an unblended frame for the first of a blend pair and a
copy of the frame following the blend pair for the second blended frame. This is the SRestore
compatible mode.
'3' Represents disp=3, where the component frames which would be merged together to create the final
recovered frame are returned. In the first blend position, the left component frame is returned and the
right when in blend index 2 postion. You would not in most cases be interested in this.
This setting can be used to view the different return frames using metrics
mode to examine what a frame looks like and the metrics reported for that frame.

The 'R' (RATIO) flag will be displayed hilited at any time the ratio
is not within expected bounds, including during the AUTO PAL detection
period (if it was a 25 FPS clip originally) but will disappear once AUTO
PAL has switched.


The 'P' (PAL) flag indicates that you are in PAL mode. If orange,
then the 'P' flag indicates that the Autopal switch was activated
when the Exblend() argument was set to 'pal = -1' (MODE 0 ONLY, auto detect pal).
The 'F' (FILM) flag indicates Film, if hilited, means that the Autopal switch
was erroneously activated and indicates that you should restart with the argument
set to 'pal=0'. the P and F flags share the same position on screen and
cannot be set simultaneously.
The 'p' (lowercase) flag is shown when 'Auto Detect Pal' is in operation and will
switch to either 'P' or 'F' at frame 1500 where PAL is either detected, or not.


The 'S' (SEQUENCE) flag will display when ExBlend() falls out of
sequence. 1, 2 or HINTED is considered in sequence. 0 is considered out of sequence.
If hilited, means that the current frame is where it went out of sequence.


The 'H' (HINTED) flag indicates that the frame has been hinted.
When Exblend() finds a blended pair, it presets 3 frames on either
side of the blend pair to hinted (NOT-BLENDED), or 4 either side if in
PAL mode.

The 'O' (OVERRIDDEN) flag indicates that the used detection, differs
from the metrics blend detection, this is most likely to happen in static
scenes where false blend detections are overriden by the hints to be
unblended. Will be hi-lited in orange if override not due to hinting.
There is no need to be too concerned about a lot of overrides, the hints
are intended to allow the detector to be quite zealous in its detections.


The 'L' (LOCKED) flag indicates that the (previously EXPERIMENTAL) locking function
has locked onto the sequence dispite a blend detection not being found.
This filter outputs a list of LOCKED frames which can be viewed using DebugView,
or to the logging file.


The I (IGNORED) flag is displayed in the case where the 'merge delta'
exceeds ithresh. The 'I' flag is in Orange if the current frame is
ignored and normal if any frame was ignored (stays on).
Extra 'ignored' info is shown when show=5.
This filter outputs a list of IGNORED frames which can be viewed using DebugView,
or to the logging file.
New in 1.02, writes "New Sequence" and "Lost Sequence" debug messages, can view
in DebugView or in logging file. If you see several "lost sequence", "new sequence"
type messages (in close proximity) in the logs, this is probably down to a false
detection following a lost sequence, this is a prime component for the 'override'
functionality of v1.02.

The 'M' flag denotes a manual override is in effect at the current frame. These
are set using the "override" arg to use the override command file.

The 'D' flag shows if an unblended frame is reconstructed using the DClip (CompUB)
and is hi-lited if adjacent frame also comes from Dclip (CompUB==2).


Show == 2, will also show the colored block in the upper LHS of clip.

3 Show Metrics:-


<--Prev<---><CUR><--><Next->
MD x.xxxx x.xxxx x.xxxx x.xxxx
FD x.xxxx x.xxxx x.xxxx x.xxxx
^ ^
2 1

Right one Hi-lighted for 1st detected blend and Left for 2nd, if blend
pair detected, MD will be smaller than FD and also smaller than the MD
entries on either side. The next frame comes in from the right.
'FD' is Frame Delta (difference) and 'MD' Merge Delta. FD is the delta
between the frames in the clip, when hi-lited it is the difference
between the two frames of the DETECTED blended pair.
MD is the difference between the two copies of the unblended component
frames (that would be merged together to 'iron out' artifacts for the
final recovery frame).
The MD:FD comparison detects that it is a blend (MD should be smaller
than FD), and the MD to MD on either side comparison detects whether
it is in the 1st or 2nd blended pair position. (MD in the hi-lited column
should be smaller than those on either side).

Show == 3, includes show 1 & 2 metrics.

4 Show Blend count and ratio

BlendPairs =w (max=x)
Ratio =y (z%)

Gives a count of detected blend pairs and the ratio of
Unblended to BlendPairs. When you have scanned from start to end of the
clip (without jumping about or going backwards) 'Ratio' should give a
result of 3.xxxx for 24 FPS original and 4.xxxx for 25 FPS original,
meaning 3 good frames and 2 blended for 24 FPS or 4 good and 2 blended
for 25 FPS.
Should only scan forward (Otherwise, BlendPairs & Ratio will
be wrong). Following ratio, the percentage of detected blends is displayed,
based on the current pal mode, this may give a more humanistic idea as to
how well ExBlend() is doing.
Following BlendPairs (max=x) is displayed where x is the maximum number of
blend pairs that could exist in the clip using the current PAL mode, this
will change if AUTO PAL switch is activated. The max value displayed assumes
also that the clip stays in sequence throughout, if it does not, the true
max value could well vary from that shown and the true percentage of blends
found could well be higher or lower than shown, but not by much (small
fraction of a percent, probably).
Also note, ExBlend() may not detect the first or last blends in a clip due
to it needing 3 frames both before and after the current frame to do it's
detections.


Show == 4, includes show 1 to 3 metrics.

5 Shows Lock, Sequence, Overridden and Ignored counts as in:-
[Default when NOT [MODE==2 and Decimate ==true)]

LC=w SC=x OC=y IC=z

In addition, if IC is non-zero, then

IMN[f1]=x.xxxx
IMX[f2]=y.yyyy

Will be displayed where f1 and f2 indicate frame numbers where the minimum
and maximum merge deltas occured that exceeded 'ithresh' and so were
ignored. You might want to investigate these frames with current ithresh
setting and also with ithresh set to 100.0 (ithresh IGNORE switched OFF).
If particularly curious, you might want to set ithresh to 0.0
(Ignore everything), do a complete scan and take a look to see what the blend
with the worst merge delta looks like, if bad, then lower ithresh a little
(tiny weeny bit below y.yyyy). Dont leave ithresh set to 0.0 for the final
scan as this will avoid almost all detections.

New to v1.02, the 'disp' arg allows return of the original blended frames,
or the de-blended frames and so you could say create
a stacked display to show both before and after with metrics, also better than
the above suggested, changing of ithresh.

Seeking to frame zero will reset the statistics. NOTE, if clip Trimmed, then
may not ever go to frame zero, and so frame 0 statistics reset may not work.
(an offhand example of where this might occur [have not checked this] might be
where you script eg "Last.Trim(100,0).Exblend()" so that a new clip never was
produced, ExBlend was invoked on a part of a clip, and so frame zero will not
be encountered within ExBlend)

NoLMN[f]=x.xxxx
L=y.yyyy LMX[f]=z.zzzz

The above is connected to the lockthresh argument of Exblend().
See Lockthresh.

MODE 2, when Decimate==true: (Defaults OFF, for MODE==2, Decimate==true).

show == 0, Off

show > 0, show different metrics.

as In:-

'39 ] Using 48 (* SDDS)'

where '39' is the decimated frame number, 48 is the source clip frame, and '(* SDDS)'
indicates an unblended frame using the source clip for adjacent frames and the DClip
frames for the blended frames.


string ExBfile Default "ExBlend"
ExBlend Project file base name. Used to create any required logging file eg
'ExBlend_1.LOG' when used in MODE==1.
Also used in creating filename of file to store 'mode' 1 Scan and Store data and to also load data file
for mode 2 Pre-calculated mode.
Creates (by default) in local directory where AVS file is located.
In Mode 1, also creates a Decimated Frames command file, by default, 'ExBlend_Decimated.TXT'
for use as a command file giving frame numbers of unblended frames after decimating in MODE==2.

bool ver true or false, Default False
Just shows the version number of ExBlend(), Perhaps other filter writers should
consider whether showing version and copyright stuff serves any positive purpose
(other than ego boosting) when showing metrics. When ver=true, it will show
version info (Also a little ego boosting stuff), without interfering with any
important metrics. ALL args are ignored when 'ver==true'.

bool revip Range false (Off) to true (On), Default false (Off)

Reverses the metrics display as in:-

<--Next<---><CUR><--><Prev->
MD x.xxxx x.xxxx x.xxxx x.xxxx
FD x.xxxx x.xxxx x.xxxx x.xxxx
^ ^
1 2

It may seem more intuitive to have the first blend appear on the left
and the second on the right, but the frames come in from the left.

[StainlessS]

Settings.txt Part 3 of 3 : Post #3 of 6



string Override Default "", (not set), eg "Override.txt" (New to v1.01)
Filename of file to force manual override the blend detections.
In v1.03, is only used in MODE=1. Neither Mode 0 nor Mode 2
allows a manual override, you must force the override's in mode
1, v1.03 extends the override ability and greatly increases it's effectivness.
Also, logging will assist in setting up the overrides file.

Required command format:-

444=h1
555
666=*4

In the above case, frame 444 is set to HINTED, ie Not A Blend. The number
after the 'h' denotes the hinted count, in this case 1, eg 444=h2 would
set two consecutive hinted frames.

555 will be set to a SHORT BLEND PAIR ie (H12H), frame 555 (in this case) will
be set to blend index 1, 556 to index 2, and the frames before and after the
blend pair to HINTED. NOTE, in this case frame 555 is the 1st BLEND index and
NOT the first hinted frame.

Frame 666=*4 will set an override at frame 666 to 4 LONG BLEND PAIRS. A long
blend is shown (in logs) as (H12HHH) for FILM or (H12HHHH) for PAL. The frame
(in this case 666) is the position of the 1st blend index, it will (on only
the first LONG BLEND PAIR) be preceded by a hint, and the blend pair followed
by 3 or four hints depending upon whether FILM or PAL mode is in effect. As in
above case, '666=*4' would yield a result of (H12HHH12HHH12HHH12HHH) if in
film mode, or (H12HHHH12HHHH12HHHH12HHHH) in PAL.
With metrics shown, the 'M' flag will show where individual manual override 'H'
hints were set, or blend index 1 postions of either short or long blend pair
overrides (in the 666 example, it will flag the positions of all the 1's
within the parenthesis['(' and ')'] ).

ExBlend is intended to correct clips that do not stay in sequence, if you have
a clip that does stay in sequence (and you are sure of that), you can force
ExBlend to stay in sequence too using the manual override command file. If say
the first, 1st blend index is at frame 2, you could create a text file containing
the manual override command:-

2=*1000000

Thats it. (you would also need to set correct pal/film mode and such). Here
we gave a command to set 1 million LONG BLEND PAIR's, it will cease when
it comes to end of clip, and in the logs will tell you how many it actually set.
This excessive repetition count will not be considered an error by the override
command parser, however, trying to start a new command outside of clip range will
be considered an error. Also, NOTE, all manual override commands MUST be in ascending
order of frame number, you are NOT permitted to override previously set overrides,
some sort of order has to be maintained. As an aid (and byproduct of this strict
ordering) the parser can tell you (in logs) where the next permissable command
may be set. If trying to set up a manual override command file, use the logs, they
are your friend when trying to do this.

int lv Log file Verbosity. (Default 0, OFF)
0=OFF, 1=ERRORS, 2=WARNINGS & ERRORS, 3 INFO & WARNINGS & ERRORS,
4=VERBOSE INFO & etc, 5 = DEBUG + VERBOSE etc.

lv=4, to show all manual overrides in logs (maybe large logs).

int dv Log to DebugView window Verbosity. (Default 0, OFF)
0=OFF, 1=ERRORS, 2=WARNINGS & ERRORS, 3 INFO & WARNINGS & ERRORS,
4=VERBOSE INFO & etc, 5 = DEBUG + VERBOSE etc.

Can use DebugView program to view this log, New in v1.02

DebugView available here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx

int disp Display mode, 0 to 4, Default 1 (DISP_EXBLEND).

0=Original Blended input clip.

1=ExBlend (default) ie returns two identical copies of deblended frames.

2=SRestore, ExBlend Returns unblended frame for 1st blend index, and a duplicate of the frame following
the blend pair in the 2nd blend index postion.
This is the compatible with what srestore returns, and implemented to better compare ExBlend with same.
INVALID in mode 2 when decimate==true.

3=Component frames. Returns both of the frames that would be merged to give the final unblended frames,
blend index 1 and 2 will return the two component frames.

4=Adjacent frames. Only valid in MODE==2 and when Decimate==true.
Returns frames adjacent to blends, using CompUB and Decompix args.

clip DClip Detection clip, (Default None)

Can use eg a denoised clip for detection instead of source clip.
IF eg a telecined clip exhibiting combing has been vertically downsized a little, it will still appear
combed but cannot be inverse telecined, in this case you might like to try a DClip with the Didee suggested

DClip=blur(0,1.0) # Vertical Blur lines

And use that for the detection clip, you might also like to set CompUB to 2.

int CompUB Component unblend mode (Default 2, [experimental, may change]).
Component Unblend Mode for DISP display mode.
Unblend-ing is achieved in two stages, firstly, two component frames are recoverd, and then the average
of these two components is used to finally recover the unblended frame. Each of the component frames is
unblended using a blended frame and also the good frame adjacent to the blended frame.
CompUB controls which clips are used in reconstruction of the component frames where both Source clip
and Dclip are provided. If no Dclip is supplied then the source clip is always used (Obviously).

CompUB ranges from 0 to 2, where, (below assumes a Dclip)

0 = No frames of Dclip, both blended And blend adjacent frames comes from source clip.
1 = One Frame comes from Dclip ie the blended frame. The adjacent frame comes from source clip.
2 = Both blended and adjacent frames come from Dclip.

An easy way of remembering it is as the number of Dclip frames used in the reconstruction of the Component frames.
(Info:- The Blend detector always uses the Dclip if supplied ie "DDDD").
0 == "SSSS", 1 == "SDDS", 2 == "DDDD", where the two middle letters are the blends and outer are adjacent.

bool Decimate (Default true in MODE==2, INVALID in MODE==0 or MODE==1)
False, off, No decimation
True, Decimate output in mode 2, MODE 1 already calculated which frames would be decimated and creates
a file (default "ExBlend_Decimated.TXT") holding the frame numbers of the unblended frames in the
MODE 2 decimated clip.


int Decompix Decimated Component Index, (Only Valid in MODE=2 when Decimate==true, when Default is 1)

When Decimate is true, (MODE 2 only) the only valid DISP Display modes are 0_DISP_BLENDED, 1_DISP_EXBLEND, 3_DISP_COMPONENT
and DISP_ADJACENT.
Decompix Is used in the DISP==0_DISP_BLENDED, DISP==3_DISP_COMPONENT and DISP==4_DISP_ADJACENT, to select which
frames to return in place of decimated unblended frames.

0_DISP_BLENDED, using both CompUB and Decompix, you can pull out the blended frame from either blend index 1 or 2,
and from either the source clip (most usual) or the Dclip if required, using the CompUB arg.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

1 DISP_EXBLEND is the usual disp mode and returns a fully unblended frame for each pair of blends. Decompix, not used.

3 DISP_COMPONENT, returns the component frame for either blend index 1 or blend index 2 (1st or 2nd blend),
Decompix specifies which of the 2 blend component frames are returned. Can be used to 'pull out' component
frames (1 & 2) from a decimated clip for alternative processing. eg, you could do some kind of denoising on the
component frames before blending them together yourself with eg Merge(). You could then replace your denoised
and merged frames back into the decimated clip, using eg ClipClop() plugin. ExBlend() does no denoising itself.

4 DISP_ADJACENT, returns the source 'S' frame adjacent to the Decompix blend index frame.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.


int DecClpIx Decimated Clip index (MODE==1, default 1: MODE 0 & 1, IGNORED)

This setting (-ve to 255 [default 1]), sets the clip index used in the MODE==1 creation of (default) "ExBlend_Decimated.Txt"
file. This command file is a command file intended for the ClipClop() frame replacement plugin. For ClipClop(), the format
would be like so:-

"1 666"

Where '1' is the clip index number for the replacement clip, and '666' is the frame to replace.
This could be used like this,

FixedClip = ClipClop(ExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

DecClpIx can be used to change the clip index (above=='1') from range 0 to 255 with <0 denoting NONE.


bool Debug Debug logging (false/true, default false)
If true, switches on logging to file or DebugView to lv & dv, 1=Errors, 2=Warnings and 5=Debug, this would only
be used to avoid perhaps the info and Verbose logging. To omit only Verbose, set eg dv=3 (INFO) and Debug=true.
Will not effect log to file unless lv is non zero, ie just DebugView window.

-----------------------------------------------------------------------------------

DebugView here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx


ClipClop() Plugin here:-
http://forum.doom9.org/showthread.php?t=162266


Prune() Plugin here:-
http://forum.doom9.org/showthread.php?t=162446
-----------------------------------------------------------------------------------

Change log :
2/Dec/2009 - ver 1.0 - 1st Public Release

26/Feb/2010 - ver 1.01

Changed blend recovery mode to be identical to srestore, Instead of using recovered frame
at blend index 2, we now use frame following blended pair so the duplicate is of the frame
following blends. Better for compare with srestore.
Switched to using DDigit text renderer.

8/Oct/2010 - ver 1.02b
Lockthresh, no longer considered EXPERIMENTAL, defaults to 1.0.
Added new arg eg Log="Logfile.Log", default ExBlend.Log.
Override commands extended and parser enhanced, assisting messages
added to logs.
Added lv and dv and disp, args.
Added "New Sequence" and "Lost Sequence" and other messages to messages log.
Moved flags around.

Recompiled using the updated DDigit text renderer with Toolkit 2003 optimizing compiler.

15/Feb/2012 - ver 1.03b

Added Dclip, Decimate, CompUB, Dcompix, DecClpIx, and Debug args. Added DISP_ADJACENT.
Lots of changes. Default back to ExBlend instead of sRestore.

18/Dec/2018 - ver 1.04.
Moved to VS 2008, added x64. Added v2.60 standard Planar Colorspaces (excl YV411).

[StainlessS]

ExBlend_QwikRef.txt Part 1 of 2 : Post #4 of 6

EVERYTHING HERE OVERRULES OLD DOCS (See old docs Also)

ExBlend() v1.04 QuickRef

YV12, YUY2, PROGRESSIVE Only. EDIT: Plus YV16 YV24 Y8.

Quick OverView, ExBlend() is a plugin to repair damage caused by blend Deinterlacing of telecined clips, which results
in a double blend, every five frames, GGGBBGGGBBGGGBB etc where 'G' is good and 'B' is blend.
Also used for telecined clips that were downsized from 720x480 to VCD 352x240 which look similar to the above mentioned damage.

Exblend(Clip,int "mode",int "pal",float "lockthresh",float "ithresh",int "show",string "ExBfile",bool "ver",bool "revip",
string "override",int "lv",int "dv",int "disp",clip "Dclip",int "CompUB",bool "Decimate",int "DecompIx",int "DecClpIx",bool "Debug")


Mode 0: Free Running Mode (Deprecated) --------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int -1 1 0 PAL = -1, ONLY allowed HERE in MODE 0
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL, either of these two most usual)
Exbfile string - - "ExBlend" Project: (logging Only), 'exbfile'.XBD Not created.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - - IGNORED (MODE 1 ONLY)
Lv int 0 5 0 Logfile: 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG
Dv int 0 5 0 DebugView: 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)

Mode 1: (ExBlend v1.03+ Scan & Store Data, default, Need to do this before using MODE==2) ------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int 0 1 0 PAL -1 Not Allowed (Error)
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL)
Exbfile string - - "ExBlend" Project: logs,'Exbfile'.XBD & Decimated (Mode 2) Frames Cmd file.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - "" Input Override Command file
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED (MODE 2 ONLY)
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 Clip index used in eg Exblend_Decimated.Txt, <0 == NONE.
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)


Mode 2: (Final Output mode, needs MODE 1 pass 1st.) -------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int - - - Ignored, Set from 'Exbfile'.XBD
LockThresh float - - - Ignored, Set from 'Exbfile'.XBD
IThresh float - - - Ignored, Set from 'Exbfile'.XBD
Show int 0 5 0 Metrics (NON-DECIMATED:0=OFF,5=FULL | DECIMATED:0=OFF,>0 ON) [DEFAULT different to modes 0 & 1]
Exbfile string - - "ExBlend" Project: (logging, Inputs Project Mode 1 XBD file)
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Not used when Decimate (Metrics are different).)
Override string - - - IGNORED, Set in Mode 1
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 4 1 If Decimate, 0_BLENDED, 1_EXBLEND, 3_COMPONENT & 4_ADJACENT valid (2_SRESTORE converted to EXBLEND)
DClip clip - - UNSET Usually as Mode 1 (or NONE), Same colorspace as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool false True True
Decompix int 1 2 1 Decimated Component Index. Valid if DECIMATE & disp=0_BLENDED/3_COMPONENT/3_ADJACENT else Ignored.
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)

[StainlessS]

ExBlend_QwikRef.txt Part 2 of 2 : Post #5 of 6



To use ExBlend(), you need to do a scan of the clip first using eg:

ExBlend(clip, Mode=1) # AssumeFPS(250.0)

or

ExBlend(clip, Mode=1,dclip=DCLIP) # AssumeFPS(250.0)

Where DCLIP is a denoised detection clip if required.

Scan through the file and it will create an ExBlend.XBD file for Mode 2 as in below example.

ExblendedClip=ExBlend(clip, Mode=2,dclip=DCLIP)



DISP: (Display Mode)
Controls which frames are returned instead of Blended Frames, using CompUB & Decompix as additional controls.
Non-Blended frames will ALWAYS return frames from source clip.

Where 'S' is Source Clip Frame, 'D' is DCLIP Detection clip frame (if supplied Else 'S')

In 'mode' 0, 1, & 2 (mode 2 when Decimate == False):

0 DISP_BLENDED If (CompUB !=0), Returns blended frame from D, else blended frame from S.
1 DISP_EXBLEND Blended frames are unblended as for CompUB.
2 DISP_SRESTORE 1st Blended frame is unblended as for CompUB, 2nd replaced with frame after 2nd blend.
3 DISP_COMPONENT Blended frames replaced with COMPONENT frame as for CompUB.

In 'mode' 2 with Decimate==True: (only DISP 0, 1,3 and 4 valid)

0 DISP_BLENDED If (CompUB !=0), Returns BLENDED frame from D, else blended frame from S,
further control via Decompix to select 1st or 2nd blend frame of blend pair.
You might want to set CompUB to 0 here to return the original blended source frame and
not from the DClip.
1 DISP_EXBLEND Blended frames are unblended as for CompUB (Same as NOT Decimated).
3 DISP_COMPONENT Returns COMPONENT frame as for CompUB, Decompix==1 = 1st Component frame, 2 =2nd.
4 DISP_ADJACENT Returns source frame 'S' adjacent to Decompix blend index.


CompUB: (Component Unblend Mode)

Unblend-ing is achieved in two stages, firstly, two component frames are recoverd, and then the average
of these two components is used to finally recover the unblended frame. Each of the component frames is
unblended using a blended frame and also the good frame adjacent to the blended frame.
CompUB controls which clips are used in reconstruction of the component frames where both Source clip
and Dclip are provided. If no Dclip is supplied then the source clip is always used (Obviously).

CompUB ranges from 0 to 2, where, (below assumes a Dclip)

0 = No frames of Dclip, both blended And blend adjacent frames comes from source clip.
1 = One Frame comes from Dclip ie the blended frame. The adjacent frame comes from source clip.
2 = Both blended and adjacent frames come from Dclip (Default [experimental, may change]).

An easy way of remembering it is as the number of Dclip frames used in the reconstruction of the Component frames.
(Info:- The Blend detector always uses the Dclip if supplied ie "DDDD").
0 == "SSSS", 1 == "SDDS", 2 == "DDDD", where the two middle letters are the blends and outer are adjacent.


Decompix: (Decimated Component Index)
When Decimate is true, (MODE 2 only) the only valid DISP Display modes are 0_DISP_BLENDED, 1_DISP_EXBLEND, 3_DISP_COMPONENT
and DISP_ADJACENT (only valid in mode==2 and when Decimate==true).

0_DISP_BLENDED, using both CompUB and Decompix, you can pull out the blended frame from either blend index 1 or 2,
and from either the source clip (most usual) or the Dclip if required, using the CompUB arg.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

1 DISP_EXBLEND is the usual disp mode and returns a fully unblended frame for each pair of blends.

3 DISP_COMPONENT, returns the component frame for either blend index 1 or blend index 2 (1st or 2nd blend),
Decompix specifies which of the 2 blend component frames are returned. Can be used to 'pull out' component
frames (1 & 2) from a decimated clip for alternative processing. eg, you could do some kind of denoising on the
component frames before blending them together yourself with eg Merge(). You could then replace your denoised
and merged frames back into the decimated clip, using eg ClipClop() plugin. ExBlend() does no denoising itself.

4 DISP_ADJACENT, returns the source 'S' frame adjacent to the Decompix blend index frame.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

ExbFile: (ExBlend Project File, [default "ExBlend"], Changed function a little in v1.03)
The Exbfile project name is used to create log file names (eg "ExBlend_1.LOG", where '_1' represents the active
MODE at the time).
In MODE 1 only, it is also used to create eg an "ExBlend.XBD" file which totally replaces
any detection stage in MODE 2 and all of the info required to Decimate in Mode 2 is contained in the XBD file.
Mode 1, also creates an eg "ExBlend_Decimated.TXT", which is a command file for use in a range/frame replacement
plugin. [eg ClipClop()], This "ExBlend_Decimated.TXT" is a file containing frame numbers that can be used to select
MODE 2 decimated output blend recovered frames.
In Mode 2, you can choose to not Decimate, in which case it would be as previous versions of ExBlend and require
later Decimation using your favorite Decimator. Or, you can Decimate if you choose, you can process the
Decimated output of ExBlend 1_DISP_EXBLEND using your favorite Motion Compensated denoiser, and feed the original
ExBlend output and the denoised output into ClipClop, using the ExBlend_Decimated.TXT command file to replace the
ExBlend recovered un-blended frames with the denoised ones, and NOT requiring yet another pass. Pass 1 to generate
the XBD and decimated frames file and Pass 2 to unblend, decimate, motion compensated denoise and replace back
into the ExBlended clip again.



DecClpIx: (Decimated Clip Index, MODE 1 Only)
This setting (-ve to 255 [default 1]), sets the clip index used in the MODE==1 creation of (default) "ExBlend_Decimated.Txt"
file. This command file is a command file intended for the ClipClop() frame replacement plugin. For ClipClop(), the format
would be like so:-

"1 666"

Where '1' is the clip index number for the replacement clip, and '666' is the frame to replace.
This could be used like this,

FixedClip = ClipClop(ExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

In the above, 'ExblendedClip' would be clip 0, and 'DenoisedExblendedClip' clip 1, it would replace the cleaned up frames
back into the deblended clip after eg post Exblend recovery denoising (eg MCDegrain).
You could also use the Prune() plugin to view ONLY the unblended Denoised frames, Prune requires clip index's starting at 0
BUT, the below should work OK.

FixedFramesOnlyClip = Prune(FixedClip,FixedClip,CMD="ExBlend_Decimated.Txt")

OR

FixedFramesOnlyClip = Prune(DenoisedExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

without putting them back into the ExblendedClip clip first.


As Prune()'s clips index starts at zero, we used the input clip name twice so we can use clip 1 (2nd FixedClip) to match
the command file.


To view just the ExBlended frames (without denoise) eg:

Prune(ExblendedClip,ExblendedClip,CMD="ExBlend_Decimated.Txt")


To extract the two component clips recovered frames only, and view them side by side

COMPUB=2 # you choose
C1=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=1)
C2=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=2)
C1_2=(Prune(C1,C1,CMD="ExBlend_Decimated.Txt")
C2_2=(Prune(C2,C2,CMD="ExBlend_Decimated.Txt")

StackHorizontal(C1_2,C2_2)


To extract the two component recovered frames, denoise them, merge together put back into ExBlended decimated clip.

COMPUB=2 # you choose
C1=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=1).MCDegrain(3)
C2=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=2).MCDegrain(3)

#Return StackHorizontal(C1,C2)

Combine = Merge(C1,C2) # Repaired. Average of the two denoised component frame clips.

FixedClip=ClipClop(C1,Combine,CMD="ExBlend_Decimated.Txt")


As an alternative to using double clip names in Prune, you can change the default clip index in the "ExBlend_Decimated.Txt" (Decimated Frames file)
so that Prune or some other alternative could be used with another clip index, a -ve "DecClpIx" will miss out the clip index altogether,
but you would then have to use another frame replacement plugin alternative to ClipCLop() (Whose index needs to be 1).

Together, Disp, Decimate, CompUB, Decompix, and eg ExBlend_Decimated.Txt allows ExBlend() to expose both
of the component frames (Decompix) fashioned with the desired CompUB method, after eg Motion compensated
processing, merged together to create the finally recovered frame, this result would then be fed back into
either one of the two ExBlend unblended component clips via ClipClop() and the ExBlend_Decimated.Txt command file.

Advanced scriptors may wish to extract the clips containing the blended frames and adjacent frames for both blend index
1 and blend index 2 and do some clever masking and such to recover the unblended frames themselves.

-----------------------------------------------------------------------------------

DebugView here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx


ClipClop() Plugin here:-
http://forum.doom9.org/showthread.php?t=162266


Prune() Plugin here:-
http://forum.doom9.org/showthread.php?t=162446
-----------------------------------------------------------------------------------

[StainlessS]

Post #6 of 6.

Here from an example clip courtesy of JohnMeyer from his thread here:-
http://forum.doom9.org/showthread.php?t=162599


https://s20.postimg.cc/jxhxf8wot/Ex_Blend-_Dog.jpg (https://postimg.cc/image/51je7nla1/)

The clip in question is 3:2 pulldown that has been resized as if progressive.

See Here for extended use demo to fix above(in old thread):- https://forum.doom9.org/showthread.php?p=1558846#post1558846

[Boulder]

I just finished the test encode of my screwed up Dream Theater Blu-ray and you know what? It's perfect! I muxed it to mkv and checked many places and everything seems fine. I really thought that there was no saving that one, but I guess it was tricky enough to challenge the scripters :D

So, thanks a lot and have a Merry Christmas :)

[StainlessS]

Boulder problem here:- https://forum.doom9.org/showthread.php?p=1859719#post1859719

Boulder,
Just curious, did you need blur?
Also, were there left over blends [EDIT: I mean dupes] as in SRestore, and if so did you need DropDeadGorgeous ?

Glad you got it sorted, and a very merry xmas to you too sir :)


EDIT: Had connection problem, double post removed.

[Boulder]

Just curious, did you need blur?
Also, were there left over blends [EDIT: I mean dupes] as in SRestore, and if so did you need DropDeadGorgeous ?

I didn't check the result frame-by-frame, I just watched it play with the normal framerate. The difficult sections like some crossfades seemed ok and longer pans also didn't have any jerkiness. I used the basic script you gave me to try so blur was being used.

[StainlessS]

Ok, thanx.
The script with SHARP=0.333 was just demo to show SHARP use and more a wild guess than anything, default sharp in McDegrainSharp is I think about 0.6,
thought that might be too much.

I can feel Hanfruns itching to post some time soon, have a guddun.

[Boulder]

I have a new clip with some restoration needed:

https://drive.google.com/open?id=1O1uKGsvamTMP33vkNodNAplNFP5oUVnw

I tried figuring out a pattern and it looks like 4 good frames, 2 blended ones. Then if you look closer, it looks more like 3 good frames and 3 blended ones where the middle one is a real blended mess and the surrounding frames much less so :( The standard settings don't seem to work properly because of that.

[StainlessS]

Sorry, cant help with triple blends, good luck in your endeavour.

[Boulder]

Thanks for the confirmation so I won't try hitting my head against the wall with ExDeblend :) I'll post in the SRestore thread, maybe someone has some ideas.

[StainlessS]

Thought I might include this, its in the source folder for coders, but might assist a little for those trying to figure out what it does.

Findblend_Method.txt
[EDIT: Below MD_??_?? is the difference between two potentially unblended 'component' pixels when merged together for final unblended pixel.
eg MD_10_23 and FD_12 etc are the names of actual variables in the source code. FD_12 etc are the actual differences between adjacent frame pixels in blended clip.]
We use the same names pre-prended with "W_" as per frame weighted [0.0 -> 100.0%] variable names.


|-----------------------------------------------|
ExBlend [ FindBlend() ]

Rough description of ExBlend() blend detection
|-----------------------------------------------|


-------- -------- -------- --------
|MD_10_23| |MD_21_34| |MD_32_45| |MD_43_56|
| | | | | | | |
| | | | | | | |
|--------| |--------| |--------| |--------|
| | | | | | | |
_| |\ _| |____ | |____ | |
| \ | \ /| |\ | | | /| |
| \ \ \ / | | \ /| | | / | |
| \ \ \ / | | \/ | / | / | |
| \ \ \_ /__ |_________ | /\/________/_____|____/ __| /|
| \ \__ /_ | ___\___|_/ // / ____|_____/ ________/ |
-------- \ --------/ \--|-----/ -\--|--// \-----/--/ -|------/ --------/
|Frame_0 | |Frame_1 | |Frame_2 | |Frame_3 | |Frame_4 | |Frame_5 | |Frame_6 |
| | | | | | | Current| | | | | | |
| in_n-3 | | in_n-2 | | in_n-1 | | in_n | | in_n+1 | | in_n+2 | | in_n+3 |
| p0 | | p1 | | p2 | | p3 | | p4 | | p5 | | p6 |
-------- -------- -------- -------- -------- -------- --------
| | | | | | | |
| | | | | | | |
-------- -------- -------- --------
Metrics display->| FD_12 | | FD_23 | | FD_34 | | FD_45 |<-Metrics display
ONLY | | | | | | | | ONLY
| | | | | | | |
-------- -------- -------- --------

-------------------------------------------------------------------------------
PER PIXEL VARIABLES:- (at current x,y postion in frame)
----
px = Pixel Luma from Frame_x (x = 0 to 6).
----
FD_ab = abs(pa - pb) Delta between adj frames[a & b] pixels
----
UP_ab = (pa * 2) - pb;
Pixel, unblended from 2 frames, two of these are later merged to recover
original frame. These are the inputs to MD_ab_cd and are not shown above
as it just makes it look confusing (I tried it), but each pair of lines
into MD_ab_cd can be thought of as UP_ab.
The ordering of the frame numbers (a,b) IS IMPORTANT and can be gotten from
the description of MD_ab_cd
----
MD_ab_cd = abs(UP_ab - UP_cd)
Merge Delta, between component pixels of the potentially recovered
(unblended) pixel. If we find a blended pair of frames and at the current
frame position we have the first blended frame, then current frame in_n
(frame_3) will be replaced with a recovered frame the equivalent unblended
pixels of MD_32_45 and the second blended frame in_n+1 (frame_4) also will
be filled with the same content when the next frame is requested,
(after doing all this again), however, next time around, in_n will be the
second blended frame as the contents of everything would have shifted left
by 1 frame, and so, MD_32_45 will NO LONGER be the recovered frame, it will
now be MD_21_34. When the recovered frame is finally rendered it will go
sort of like:-
RecoveredPixel = (UP_ab + UP_cd + 1) / 2; (for luma)
-------------------------------------------------------------------------------
PER FRAME VARIABLES
---
WEIGHTINGS:- (float)
Accumulated weighting for pixels; Obviously clear to zero at beginning.
----
W_MD_ab_cd += MD_ab_cd;
W_FD_ab += FD_ab;
----
METRIC:-
Final scaling of measurements for whole frames before deciding if blended.
----
float mx = (vi.width * vi.height) * 255.0;
For scaling weighting into more managable & meaningful numbers.

W_MD_ab_cd = W_MD_ab_cd * 100.0 / mx;
W_UP_ab = W_UP_ab * 100.0 / mx;
W_FD_ab = W_FD_ab * 100.0 / mx;

If ((W_MD_32_45 < W_MD_21_34) && (W_MD_32_45 < W_MD_43_56))
{
// then, is possible 1st of pair of blends
if (W_MD_32_45 < W_FD_34)
{
// then IS 1st blend index
}
}
else If ((W_MD_21_34 < W_MD_10_23) && (W_MD_21_34 < W_MD_32_45))
{
// then is possible 2nd of pair of blends
if (W_MD_21_34 < W_FD_23)
{
// then IS 2nd blend index
}
}
else
{
// Is NOT a blend
}


Also this in there too (identities.txt).
EDIT: After staring at below gobble-de-gook for a while, added the stuff in blue [will be modded as below in next release].

Below written to assist understanding of how the screwed up clip was created, and is from point of view of the pre-blended clip.

The 'IN' (ie W,X,Y,Z) are frames before the bad blends were created, and 'OUT' is the resultant bad blending sequence that need be fixed,
C and D are the blended pair, and Y is the missing frame that is partially present in both C and D blended pair.
[Missing Y frame is the one that needs recovering].



IN W X Y Z # Pre blended frames.
| | \ /\ /|
OUT A B C D E # Bad frames that we have to deal with.


A = W
B = X
C = (X + Y) / 2 : X = 2C - Y : B = 2C - Y : Y = 2C - X : Y = 2C - B
D = (Y + Z) / 2 : Y = 2D - Z : Y = 2D - E : Z = 2D - Y : E = 2D - Y
E = Z

FD_CD = | ((X + Y) / 2) - ((Y + Z) / 2) | == | ((X + Y - Y - Z) / 2) | == | (X - Z) / 2 |
== | (B - E) / 2 |
== | B - E | / 2

FD_BC = | X - ((X + Y) / 2) | == | (2X - (X + Y)) /2 | == |(2X - X - Y)/2| == | (X - Y) / 2 |
== | (B - Y) / 2 |
== | B - Y | / 2


FD_DE = | ((Y + Z) / 2) - Z | == | (Y + Z - 2Z) / 2 | == | (Y - Z) / 2 |
== | (Y - E) / 2 |
== | Y - E | / 2
# Above FD_nm is expected Frame Difference between frames n and m.

[domator]

Hi StainlessS,
I think your plugin could help me fix a problem with Sliders DVD. I tried using it but I didn't get good results. Maybe you could take a look?
This is the problem:
https://forum.doom9.org/showthread.php?p=1869424#post1869424
And this is original file:
http://5.196.74.194/domek/VTS_02_1.demuxed.m2v
And this is my AviSynth script which didn't fix the problem:
LoadPlugin("Exblend_25.dll")
LoadPlugin("DGDecode.dll")
LoadPlugin("TIVTC.dll")
MPEG2Source("VTS_02_1.d2v")
TFM()
ExBlend()

[StainlessS]

ExBlend not intended for your kind of blending, sorry.

[Blackout_Out]

Hey! I'm getting an error that says "ExBlend ERROR, Mode 2 XBD, Cannot open XBD file," but how do I scan the clip using StaxRip?

Thanks!

[StainlessS]

Maybe best to ask in staxRip thread, I aint ever used it.

[Blackout_Out]

Maybe best to ask in staxRip thread, I aint ever used it.
Fair enough, thank you!

[StainlessS]

Blackout_Out,
Not sure but I seem to remember that XBD path is a bit broken on Vista+ (or maybe W7+),
default XBD directory is I think "C:\", but OS dont allow to be written there so does not exist, need give
either full path or eg ".\MyFile.xbd" for current directory relative, for both Mode=1 and 2.
Hope that helps.
[Above from memory, could be wrong]
EDIT: Also, you need do Mode=1 scan before Mode=2 pass.

[CruJones]

Thanks for the ExBlend plugin StainlessS. I recently taught myself QTGMC deinterlacing with AviSynth to work on a some DVD's I have. When I stumbled upon your plugin, I recognized that telecine pattern of double blend every 5 frames in an old 2 min clip from 1996 that I was hoping to fix. In mode 2 after the XBD scan it definitely improved the footage with 25 fps to get rid of the ghosting from the original but it's not perfect. There's a number of fades from one sequence to another in the editing and in those moments the frames seem a little off. I'd like to try and see if I can correct any of this further but without scripting experience I'm probably way in over my head with this. Any advice is much appreciated though. Cheers!

[StainlessS]

I guess that you're lucky that it worked at all, was really only intended for when NTSC telecine clip was downsized to half height via eg verticalReduceBy2(), producing 2 out of 5 blending.
Fades will/may cause ExBlend to lose sync, and take time to get back in-sync, ExBlend is a bit particular in the exact cause of blending, some have used SRestore() instead for same [likely that fades will prove problem there too].
Where loses sync [at eg fades] might want to look at the Overrides arg [string, filename, MODE=1 only].
Set Override at first blend pair following 'de-sync'.
Good Luck.

EDIT: From Settings.txt

string Override Default "", (not set), eg "Override.txt" (New to v1.01)
Filename of file to force manual override the blend detections.
In v1.03, is only used in MODE=1. Neither Mode 0 nor Mode 2
allows a manual override, you must force the override's in mode
1, v1.03 extends the override ability and greatly increases it's effectivness.
Also, logging will assist in setting up the overrides file.

Required command format:-

444=h1
555
666=*4

In the above case, frame 444 is set to HINTED, ie Not A Blend. The number
after the 'h' denotes the hinted count, in this case 1, eg 444=h2 would
set two consecutive hinted frames.

555 will be set to a SHORT BLEND PAIR ie (H12H), frame 555 (in this case) will
be set to blend index 1, 556 to index 2, and the frames before and after the
blend pair to HINTED. NOTE, in this case frame 555 is the 1st BLEND index and
NOT the first hinted frame.

Frame 666=*4 will set an override at frame 666 to 4 LONG BLEND PAIRS. A long
blend is shown (in logs) as (H12HHH) for FILM or (H12HHHH) for PAL. The frame
(in this case 666) is the position of the 1st blend index, it will (on only
the first LONG BLEND PAIR) be preceded by a hint, and the blend pair followed
by 3 or four hints depending upon whether FILM or PAL mode is in effect. As in
above case, '666=*4' would yield a result of (H12HHH12HHH12HHH12HHH) if in
film mode, or (H12HHHH12HHHH12HHHH12HHHH) in PAL.
With metrics shown, the 'M' flag will show where individual manual override 'H'
hints were set, or blend index 1 postions of either short or long blend pair
overrides (in the 666 example, it will flag the positions of all the 1's
within the parenthesis['(' and ')'] ).

ExBlend is intended to correct clips that do not stay in sequence, if you have
a clip that does stay in sequence (and you are sure of that), you can force
ExBlend to stay in sequence too using the manual override command file. If say
the first, 1st blend index is at frame 2, you could create a text file containing
the manual override command:-

2=*1000000

Thats it. (you would also need to set correct pal/film mode and such). Here
we gave a command to set 1 million LONG BLEND PAIR's, it will cease when
it comes to end of clip, and in the logs will tell you how many it actually set.
This excessive repetition count will not be considered an error by the override
command parser, however, trying to start a new command outside of clip range will
be considered an error. Also, NOTE, all manual override commands MUST be in ascending
order of frame number, you are NOT permitted to override previously set overrides,
some sort of order has to be maintained. As an aid (and byproduct of this strict
ordering) the parser can tell you (in logs) where the next permissable command
may be set. If trying to set up a manual override command file, use the logs, they
are your friend when trying to do this.

[CruJones]

Thanks for the response and info StainlessS. The clip that I was using the ExBlend on was not from a DVD source by the way but rather a TV capture from a Cinemax/HBO program which originally aired in 1996. (I was probably unclear in my first post and see why you were surprised it worked). I’m not sure of the source of the original capture but ExBlend has definitely improved a lot of the footage. Thanks again!

[Julek]

The Canaan blu-ray suffers from a chroma blend, I was trying to fix it via vapoursynth but was getting a worse result than Yousei-Raws.
So I went to see if AVS had something better and ended up discovering ExBlend which has the same result as my script. :D

My deblend function: https://github.com/dnjulek/jvsfunc/blob/main/jvsfunc.py#L148
In some scenes the result was really good: https://slow.pics/c/zz4RFF80
But in others the Yousei-Raws result was cleaner: https://slow.pics/c/Ff8LeIjy
I was wondering what Yousei used to fix it.
The blend probably happened because the video was upscaled from 720p to 1080p, destripe to 360p fixes the luma: https://pastebin.com/2eVi3ijq
Video samples: https://drive.google.com/drive/folders/1IXnaJiE0IFk82vwow9J3MS3ZVBfOx8VM

[Zarxrax]

Something doesn't seem to be working right when I try to use Exblend.
When I use mode=1, it actually seems to detect the blends, and doesn't display the blended frames in the output. But when I use mode=2, it just seems to leave them in and create new blends!

This is just a basic script loading a video through FFMS2 and then using Exblend(mode=1), doing a scan of the whole video, and then changing to mode=2.

Am I doing something wrong?
https://i.postimg.cc/tZXC9RNJ/exblend-mode2.jpg (https://postimg.cc/tZXC9RNJ)

[StainlessS]

Attachments can take forever, and show script probably more informative.
See Here for future Image post:- https://forum.doom9.org/showthread.php?p=1959414#post1959414

IIRC, VISTA+, broke the current directory relative file path stuff, and it might not work properley without
full path [or at least have the ".\" part for current dir relative eg ".\somefile.txt", a plain "somefile.txt" can fail].

So, dont rely on default paths for old Exblend, specify full path or proper current path relative preceding filename with ".\"

EDIT: I think W7 [or maybe W10] defaulted to saving any files without ''.\" in VirtualStore (whatever that is),
again. specifiy full path and ideally not in root of C: drive.

[StainlessS]

Zarxrax, Did you ever find a solution for this ?
Curious if it is possible for avisynth to do anything like this:
Take a frame or short clip from one video, and search for it's location in another video. While the 2 videos contain the same general footage, they may differ greatly in quality, resolution, cropping, and so on.

I have hundreds of clips that I am trying to locate within tens of hours of footage, so some semi-automated method would be much simpler than manually searching through all of the footage for every single clip.

If not avisynth, perhaps it might be possible using ffmpeg or some other tool?

And if not, are you still interested in a possible [maybe - sometimes] solution ?
Once prepped (make SEARCH clip, maybe 320x240 or bit bigger copy of big file, stretching to Max dynamic range PC_Levels, then make fingerprint file),
search on your single FIND Frame should take maybe max 1 second [find or fail].
Could even auto make FIND Clip same as SEARCH clip size etc, pick out say 10 frames from find clip, and show result SEARCH clip frame that matches first found
frame of FIND clip, with a small number of FIND frames (maybe 10) should nearly always find at least one of them.

Would take time to make small dimension SEARCH clip, and then maybe eg 16 mins [on my old XP machine core duo for 2 hours Jurassic Park DVD at DVD rez]
to do the fingerprint file [need keep both small SEARCH clip and fingerprint file]. After that, real fast to find/fail_find frames.
(When first written, was for EXACT same frame to search for, had it doing something like 3 Million frames/ second on the XP core duo, sadly, frames turned out not to be
exactly the same, and I spent 9 months trying to get it working with hit/miss results and quite a bit slower due to match tolerance).
https://forum.doom9.org/showthread.php?t=170332&highlight=RT_QwikScan

EDIT: The first posts in linked thread were of script functions, was later implemented as RT_Stats plugin.

[Zarxrax]

Attachments can take forever, and show script probably more informative.
See Here for future Image post:- https://forum.doom9.org/showthread.php?p=1959414#post1959414

IIRC, VISTA+, broke the current directory relative file path stuff, and it might not work properley without
full path [or at least have the "." part for current dir relative eg ".\somefile.txt", a plain "somefile.txt" can fail].

So, dont rely on default paths for old Exblend, specify full path or proper current path relative preceding filename with "."

EDIT: I think W7 [or maybe W10] defaulted to saving any files without ''." in VirtualStore (whatever that is),
again. specifiy full path and ideally not in root of C: drive.

I updated my post with the image link. It does not seem to be an issue with the file paths. The plugin correctly outputs the file in the same folder as the .avs without me having to specify anything. But I did try manually specifying the path just to be sure.

Regarding your qwickscan function, thanks, I'll hold onto that in case I need something like that again in the future.

[StainlessS]

Zarx,
OK, let me know if you want assistance with the QwikScan thingy. [is some basic functionality to search for frame somewhere in RT_stats thread (needs same frame size - try LumaTol=0.0 for identical frame)].

Can you post your scripts, also which version EXBlend (I aint used original for long time, I last used the Beta, think I fixed the path thing in Beta).

[Zarxrax]

Zarx,
OK, let me know if you want assistance with the QwikScan thingy. [is some basic functionality to search for frame somewhere in RT_stats thread (needs same frame size - try LumaTol=0.0 for identical frame)].

Can you post your scripts, also which version EXBlend (I aint used original for long time, I last used the Beta, think I fixed the path thing in Beta).

Script is simply just FFMS2("input.mov").ExBlend(mode=1)
I am using the 20181218 x64 version (I assume its the latest).

[StainlessS]

OK, it is the latest.

Well I dont know what to say, You script shows that you did not set the ExBFile to anything eg "D:\ExBlend.XBD", suggest that you do give specific path (not C:\) and filename at very least.
Also, the image shown suggests that it only found 2 BlendPairs at frame 419, methinks maybe you just jumped there after clip open, and so metrics mean nothing at all.
Also, unless you did COMPLETE scan from start of clip to end, then XBD file will NOT be created.
Metrics show PAL = -1, (default, ie unknown/auto detect mode), once you figure out what clip you have, then you need set PAL=0 (not PAL) or 1=PAL [Better detect when not guessing].
After FULL scan at correct PAL and Mode=1, then Ratio% should be ideally green and about 95%+, 98% a lot better, less than about 95% forget it, it aint the right detector plugin.
You have HIGH Ratio (too high actually due to not knowing whether should be PAL=0 or 1, and only 2 blends detected).

Suggest do as above, full scan, and check that on last frame that Metrics are at least 95+%, then close script and that ExBlend.XBD exists when you closed. [not written until closed].

Then can switch to Mode=2 (and can also then set Decimate=True otherwise there will be dupes to be decimated in result of mode 2 pass).

EDIT: I think thats about right, long long time since i used it.
EDIT: Seems that in Mode=2, Decimate default is also true, so you dont need set that. [Quickref has table of defaults and settings for each mode].
EDIT: Not sure, maybe you dont need supply ExBFile="Exblend", think should be ok without doing anything, but not sure if path is needed,
I dont have test clip, I shall try with any old clip and see what happens.

[StainlessS]

OK, just tried it with any old clip, did not of course have proper detections, but still created ExBlend.EXB file in current directory [I must have fixed directory stuff on last update],
and no longer needs path, indeed you must just give "ExBFile" or similar, the function of ExBFile has changed a little since older version,
it used to be a filename [and had path probs on Vista+], but has changed to be a Project Name, several files are created using the Project name, and the ".XBD"
part is auto appended to the correct one.

Also, pass 1 can just use

Avisource("...")
ExBlend() # Default is now ExBlend(Mode=1)

And can use VDub2() Video Analasis Pass for fast scan, will stop on last frame to see metrics [It used to jump back to frame 0 until I asked to be changed].
Close file and ExBlend.XBD file and also ExBlend_Decimated.TXT will be created, [ClipClop command file with frame list of frames that will be UnBlended on pass 2,
can use for selective denoise of only Unblended frames of the decimated result].


Pass 2

Avisource("...")
ExBlend(Mode=2) # Default Where Mode=2, is now ExBlend(Mode=2, Decimate=True)


Latest version was big change and many defaults changed with mode. [needed to be changed because of massive possibilites added by decimation and other stuff].
From the QuickRef [default for both mode=1 and mode=2]

Mode 1: (ExBlend v1.03+ Scan & Store Data, default, Need to do this before using MODE==2) ------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int 0 1 0 PAL -1 Not Allowed (Error)
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL)
Exbfile string - - "ExBlend" Project: logs,'Exbfile'.XBD & Decimated (Mode 2) Frames Cmd file.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - "" Input Override Command file
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED (MODE 2 ONLY)
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 Clip index used in eg Exblend_Decimated.Txt, <0 == NONE.
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)


Mode 2: (Final Output mode, needs MODE 1 pass 1st.) -------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int - - - Ignored, Set from 'Exbfile'.XBD
LockThresh float - - - Ignored, Set from 'Exbfile'.XBD
IThresh float - - - Ignored, Set from 'Exbfile'.XBD
Show int 0 5 0 Metrics (NON-DECIMATED:0=OFF,5=FULL | DECIMATED:0=OFF,>0 ON) [DEFAULT different to modes 0 & 1]
Exbfile string - - "ExBlend" Project: (logging, Inputs Project Mode 1 XBD file)
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Not used when Decimate (Metrics are different).)
Override string - - - IGNORED, Set in Mode 1
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 4 1 If Decimate, 0_BLENDED, 1_EXBLEND, 3_COMPONENT & 4_ADJACENT valid (2_SRESTORE converted to EXBLEND)
DClip clip - - UNSET Usually as Mode 1 (or NONE), Same colorspace as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool false True True
Decompix int 1 2 1 Decimated Component Index. Valid if DECIMATE & disp=0_BLENDED/3_COMPONENT/3_ADJACENT else Ignored.
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)



To use ExBlend(), you need to do a scan of the clip first using eg:

ExBlend(clip, Mode=1) # AssumeFPS(250.0)

or

ExBlend(clip, Mode=1,dclip=DCLIP) # AssumeFPS(250.0)

Where DCLIP is a denoised detection clip if required.

Scan through the file and it will create an ExBlend.XBD file for Mode 2 as in below example.

ExblendedClip=ExBlend(clip, Mode=2,dclip=DCLIP)


EDIT: Also, its v2.60 colorspace cabable [docs need update].
and Default on mode=1 is PAL=0, not pal, so thats ok.
So really, I presume that you just did not do complete scan of the clip in mode = 1 [but need also check the Ratio % on last frame].

[Zarxrax]

Hmmm, well I know it is NTSC content that was re-edited on an NTSC vhs deck, so I suppose that might have messed with the fields in a way that made it incompatible with this plugin. Also with it being anime, it was mostly animated on the 2's so most of the time there is only 1 visible blended frame per group of 5 frames. I tried the pal=0 setting with no improvement.

You are right that for my mode=1 screenshot, I did directly seek to that area. After some more investigation, if I have it on mode 1 and play from the beginning (or encode the output of the full scan and watch it back) I can see that it doesn't actually get fixed in mode 1. Only if I directly seek there, it will fix that section, I suppose by sheer chance. I just saw that it seemed like the plugin was actually working there and I guess I got my hopes up.

[StainlessS]

NTSC content that was re-edited on an NTSC vhs deck
Well that there would make it fail at [EDIT: unless extreme care taken] every edit and need re-acquire lock on the blend sequences.
most of the time there is only 1 visible blended frame per group of 5 frames
The plug is specifically for 3 good 2 blend sequences, I did not know what caused it when i wrote the plugin,
apparently ActionMan33 [EDIT: ActionMan133] wrote a script to fix similar stuff [and theres at least one more as well as SRestore].
It turns out that [I think] most clips fixable with Exblend are result of
VerticalReduceBy2 (clip), HorizontalReduceBy2 (clip) OR, ReduceBy2 (clip) on Interlaced source [EDIT: 3:2 pulldown source].
However, a few changes made 2 versions ago allowed it to be used to fix a JohnMeyer clip that was not
a result of that downsizing by 2, but had been an NTSC interlaced resize [EDIT: 3:2 pulldown source].

Also, I made mistake in saying was in AutoPal mode, I saw '?' flag and mistook it for the upside down '?' which flags AutoPal mode.

So, sorry, this plug is not for you, this time. :(

EDIT: Sorry, NOT NTSC interlaced resize, NTSC 3:2 pulldown telecine film resize.

EDIT: The PAL mode was to detect 4:2 [good/blend] sequences, I had no idea at the time what it was that caused the problems,
and was easy to add just incase they existed in the wild.

There were at one time a number of "Betty Page" clips available [maybe downsized for Video CD] which were easily available via Google,
[type in her name a a bunch of them were there] these all seemed to demo the problem. [those available last time I tried were much lesser
quality [flv maybe] from sites such as YouTube, but I think still un-exblend perfectly.

EDIT: A quick look and the BP videos are not at top of google list any more [mostly stuff about the Movie - The Notorious Bettie Page].

EDIT: I was actually doing a working version that fixed 3 (or 4) good, 3 blend, but dont know if I've even still got the source.
(the test clips were all artificially created)

EDIT: The original sorta reasonable quality betty page clips were actually mpeg (mpg1 so VCD I think).

[Zarxrax]

Well thanks. I previously wasn't able to obtain a good result with srestore, but I revisited it now and tried a few different settings, and I am getting fairly decent results from it now. Deblending is such a complex topic to wrap my head around.

[Selur]

@StainlessS: Any plans for an Vapoursynth port of Exblend?

[Selur]

I used:
ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LoadDll.dll")
LoadDLL("I:\Hybrid\64bit\Avisynth\avisynthPlugins\libfftw3f-3.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\TIVTC.dll")
LoadPlugin("C:\Users\Selur\Desktop\ExBlend_x64.dll")
Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
# loading source: C:\Users\Selur\Desktop\Courage the Cowardly Dog S02E17E18 SOURCE.mkv
# color sampling YV12@8, matrix: bt709, scantyp: telecine, luminance scale: limited
LWLibavVideoSource("C:\Users\Selur\Desktop\COURAG~1.MKV",cache=false,dr=true,format="YUV420P8", prefer_hw=0)
# current resolution: 720x480
# deinterlacing


TFM() # field match back to a film frames
TDecimate() #remove duplicates
AssumeFrameBased()
org=last
Exblend(mode=1, Exbfile="e:\Exbfile")
#Exblend(mode=0,revip=true, show=0)
StackHorizontal(last, org)


# filtering
# Converting from 8 to 10bit for encoder
ConvertBits(10)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
PreFetch(16)
# output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last
and called
avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL
Assuming that this would create "e:\Exbfile.XBD" which it does not and thus changing the script to
Exblend(mode=2, Exbfile="e:\Exbfile")
aborts with:
ExBlend: ERROR, Mode 2 XBD, Cannot open BXD file "e:\Exbfile.XBD".

-> How to use this properly?

[StainlessS]

Any plans for an Vapoursynth port of Exblend?
Nope, I dont do the vapourSynth thingy, yet. If somebody else wants to do conversion, I have no problems with that.


TFM() # field match back to a film frames
TDecimate() #remove duplicates
AssumeFrameBased()
org=last
Exblend(mode=1, Exbfile="e:\Exbfile")

Above in red tastes a bit off.
ExBlend is intended for clip that should have been de-telecined prior to resize down to VCD/SVCD size, where would create 3/2 blending [3good/2blend].

Stuff in red would seem to provide exblend with NTSC film rate progressive, not NTSC telecined(29.976), badly resized downwards.

aborts with:
ExBlend: ERROR, Mode 2 XBD, Cannot open BXD file "e:\Exbfile.XBD".

I can only imagine that ExBlend did not find any blended pairs, and so the XBD file is empty/non existing.
[EDIT: If indeed it was the type of badly resized clip that Exblend is intended for, then the stuff in red would likely bugger that up and make ExBlend fail to find anything]
[EDIT: You would best look at Exblend Metrics in Avisynth, and detect %]

[Selur]

Background:
A user of Hybrid wrote to me:
I've got an American NTSC VHS, which is an NTSC 29.98 conversion of a PAL shot on video movie. The original master is lost. I have it captured via S-Video.
and that no matter what he did in Hybrid, he couldn't get rid of the ghosting. Looking at the fields I noticed that some of the fields already had the ghosting.
Since I rembered I got rid of similar ghosting a few years back which ExBlend, I wanted to try with the latest version.

If I run ExBlend in mode=0 it properly finds some of the existing blended frames and 'fixes' them.
https://i.ibb.co/K54n9q8/exblend.png (https://ibb.co/rQ89Z6X)
But seeing that mode=0 is depricated and not the way one should use the filter I wanted to try it using 2passes.
Problem is:
Exblend(mode=1, Exbfile="e:\Exbfile")
doesn't create a 'e:\Exbfile.xbd' file or any other file and thus mode=2 aborts with the above message.
-> So I don't know if this is a bug in ExBlend or whether I don't know how to properly use it.

Cu Selur

Ps.: not sure whether ExBlend was meant for this, but it helps with at least some of the blends which is why I would like to use it. I can stick with mode=0, but thought that since it's depricated it would be wise to figure out how to use the 2pass processing. :)

[StainlessS]

See this thingy here:- https://forum.doom9.org/showthread.php?p=1860483&highlight=exblend#post1860483
where was used with a bit of a Didee kludge, and it worked ok.
Also has in post problem that seemed to come into existence with W7 (maybe Vista), for paths.

Also, just in case you are using older Ver$, here is the last 2018 ver[v1.04]:- https://www.mediafire.com/file/ejqfgiccrpmyj2o/ExBlend_25%252626_x86_x64_dll_v1-04_20181218.zip/file

Also, I seem to remember that no XBD file will be generated unless a complete scan is done {ie, it does not write a partially complete file}
[also, Might possibly abort write if detections were really not good, flagged in metrics in RED].
I think also writes message to DebugView when writing XBD, or tells if skipping write. [check it out].

[Selur]

I'm on Windows 11.

Also, I seem to remember that no XBD file will be generated unless a complete scan is done {ie, it does not write a partially complete file}
I called my clip with "avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL" which runs through the whole thing.

I now opened the script in avsmod, changed the call to
Exblend(mode=1, ExBfile="e:\Exbfile", debug=true, lv=5, dv=5)
downloaded and started Debugview and scrolled through the clip in avspmod.

Tried to save the content of Debugview, but then Debugview, simple froze.
-> took a screenshot and killed Debugview.
https://i.ibb.co/Ph3KRdx/Debugview.png (https://ibb.co/VSrRdsV)
Now E:\Exbfile_1.LOG was created, when I closed avspmod which contains:
ExBlend: INFO,
ExBlend: INFO, ExBlend() v1.04 - 18 Dec 2018 - (c) StainlessS.
ExBlend: INFO,
ExBlend: DEBUG, Limiting args to 'Mode 1'
ExBlend: DEBUG, Checking Colorspace
ExBlend: DEBUG, Checking valid 'mode'
ExBlend: DEBUG, Checking valid 'pal' range
ExBlend: DEBUG, Allocating fdat[]
ExBlend: DEBUG, Allocating MetricsDat[]
ExBlend: INFO, ------------------------------------
ExBlend: INFO, MODE = 1
ExBlend: INFO, PAL = 0
ExBlend: INFO, LOCKTHRESH = 1,00
ExBlend: INFO, ITHRESH = 3,00
ExBlend: INFO, SHOW = 5
ExBlend: INFO, EXBFILE = 'e:\Exbfile'
ExBlend: INFO, VER = False
ExBlend: INFO, REVIP = False
ExBlend: INFO, OVERRIDE = NOT SET
ExBlend: INFO, LV = 5 DEBUG
ExBlend: INFO, DV = 5 DEBUG
ExBlend: INFO, DISP = 1 EXBLEND [Default]
ExBlend: INFO, DCLIP = NO
ExBlend: INFO, COMPUB = 2 0=SSSS, 1=SDDS 2=DDDD [S=Source,D=DClip]
ExBlend: INFO, DECIMATE = False
ExBlend: INFO, DECOMPIX = 1 UNUSED
ExBlend: INFO, DecClpIx = 1
ExBlend: INFO, DEBUG = True
ExBlend: INFO, ------------------------------------
ExBlend: DEBUG, LOG Filename = e:\Exbfile_1.LOG
ExBlend: DEBUG, LOG file Verbosity = 5
ExBlend: DEBUG, LOG DebugView Verbosity = 5
ExBlend: DEBUG, XBD Filename = e:\Exbfile.XBD
ExBlend: DEBUG, FRAMES Filename = e:\Exbfile_Decimated.TXT
ExBlend: DEBUG, ------------------------------------
ExBlend: DEBUG, Constructor, no Errors encountered.
ExBlend: DEBUG, Constructor OUT >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
ExBlend: INFO, [569] Lost Sequence: Detected=2
ExBlend: DEBUG, Destructor IN <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ExBlend: DEBUG, Mode 1, checking if clip fully scanned.
ExBlend: DEBUG, Mode 1, Buffer Incomplete ( 0,3 Percent)
ExBlend: DEBUG, delete[] MetricsDat
ExBlend: DEBUG, delete[] fdat
ExBlend: DEBUG, Destructor OUT (CLOSING LOG) >>>>>>>>>>>>>>>>>>>>
>>>
No e:\Exbfile.XBD or e:\Exbfile_Decimated.TXT file was created.

Cu Selur

[StainlessS]

OK, I think I twigged whats happening,
I called my clip with "avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL" which runs through the whole thing.
think that avs2yuv64 is returning to frame 0 after scan, which I've a sneaky feeling will start afresh ie reset, as if nothing has been scanned.
I'm not on my compiler machine, and not even on windows at present, I'll see what I can do when I can.

EDIT: I've never actually used avs2yuv. [or cant remember it if I have done]

EDIT:
I still doubt that it would do any good if it is to follow this

TFM() # field match back to a film frames
TDecimate() #remove duplicates


but that's a different matter.

[StainlessS]

Selur,
https://forum.doom9.org/showthread.php?p=1846853#post1846853
above linked RT_ForceProcess() will scan entire file and NOT return to frame 0 on completion. [if that is any good]

[Selur]

Thanks will look whether I get RT_ForceProcess working.
a. Okay, the link in that thread doesn't work, but I got the RT_Stats_25&26_x86_x64_dll_v2.00Beta13_20201229.zip from your MediaFire site.
b. Looked at the source the TFM&Decimate isn't needed, source is wrongly flagged as interlaced while it's progressive (+ blended fields).
-> okay, I got no idea how to use that properly.
using:
ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
LoadPlugin("C:\Users\Selur\Desktop\ExBlend_x64.dll")
LoadPlugin("C:\Users\Selur\Desktop\RT_Stats_x64.dll")
LWLibavVideoSource("e:\clips\Courage the Cowardly Dog S02E17E18 SOURCE.mkv",cache=false,dr=true,format="YUV420P8", prefer_hw=0)
org=last
#Exblend(mode=1, ExBfile="e:\Exbfile", debug=true, lv=5, dv=5)
RT_ForceProcess()
#Exblend(mode=0,revip=true, show=0)
#StackHorizontal(last, org)
PreFetch(16)
# output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last
also doesn't output a file (I could find), but it shows some stuff in the debugviewer.
https://i.ibb.co/DkjR1sv/dv.png (https://ibb.co/d6H5f1V)

Cu Selur
Ps.: Also uploaded the source to my GoogleDrive (https://drive.google.com/file/d/1vKS7UMcCW0sQmiQ42eSCYG3FTHqM1ZEp/view?usp=sharing)

[StainlessS]

Selur,
OK, Exblend only writes XBD file on exblend input clip [EDIT: ExBlend Filter] destructor, so we need to do a little hack for RT_ForceProcess thingy to work.


VideoFileName ="E:\SELUR\Courage the Cowardly Dog S02E17E18 SOURCE.dgi"
DGSource(VideoFileName)
ORG=Last

TMP=ORG
TMP.Exblend(mode=1, ExBfile=".\Exbfile", debug=true, lv=5, dv=5)
RT_ForceProcess() # Force PASS1
TMP=0 # Force Close ExBlend I/P clip [and so also destroy ExBlend Filter clip]. XBD file NOT written until ExBlend destructor is called.

ORG
Exblend(mode=2,ExBfile=".\Exbfile") # PASS2

Return last

EDIT: Above we had a bit of an 'Interoperability' style problem, as RT_ForceProcess returns only a 0 [TwriteAVI::ForceProcess() does return a clip and would probably be more suitable].
So, we had to kill input clip to Exblend instead of output clip. Making Last=0 [instead of "TMP=0"] after RT_ForceProcess() does not work, not sure why.

EDIT: Above PASS2 is decimated result, so any StackHorizontal whatsit will be out-of-sync frames.

EDIT: Metrics on last frame [single pass only] show 92.53% of maximum possible blend pair detections,
I would only use ExBlend() if above about 95% or 96%. [EDIT: Although it is a short clip and does fluctuate about 92% -> 96%, longer clip would be better accuracy %]

[StainlessS]

With stackhorizontal


VideoFileName ="E:\SELUR\Courage the Cowardly Dog S02E17E18 SOURCE.dgi"
DGSource(VideoFileName)
ORG=Last

TMP=ORG
TMP.Exblend(mode=1, ExBfile=".\Exbfile", debug=true, lv=5, dv=5)
#return last Return Pass 1

RT_ForceProcess()
TMP=0 # kill TMP clip and call ExBlend filter destructor

ORG

SHOW=1
Exblend(mode=2,ExBfile=".\Exbfile",Decimate=False,Disp=1,Show=SHOW) # Non decimated, Blend pair replaced with 2 x copies of ex-blended frame. Little colored block flags ex-Blended frame pair.
#Exblend(mode=2,ExBfile=".\Exbfile") # Deblended & Decimated

# Both below lines for Non Decimated ONLY.
StackHorizontal(ORG,Last)
#Subtract(ORG,Last)

Return last

[Julek]

Ps.: Also uploaded the source to my GoogleDrive (https://drive.google.com/file/d/1vKS7UMcCW0sQmiQ42eSCYG3FTHqM1ZEp/view?usp=sharing)

In this case there is no new information in the combed frame, it's just the prev blended with the next, the correct way here is to just drop it like this:


import vapoursynth as vs
core = vs.core

src = core.lsmas.LWLibavSource(r"C:\Users\conta\Videos\src\dog.mkv")
decimate = src.std.SelectEvery(cycle=5, offsets=[0, 1, 2, 4])
decimate.set_output()

But since the pattern is not fixed, you would have to adjust the offsets for each scene.

[Julek]

In this case there is no new information in the combed frame, it's just the prev blended with the next, the correct way here is to just drop it like this:


import vapoursynth as vs
core = vs.core

src = core.lsmas.LWLibavSource(r"C:\Users\conta\Videos\src\dog.mkv")
decimate = src.std.SelectEvery(cycle=5, offsets=[0, 1, 2, 4])
decimate.set_output()

But since the pattern is not fixed, you would have to adjust the offsets for each scene.
I just noticed that the background is true 30 fps, so decimate would not be ideal.
I will try to make a script to replace the damaged part using a mask.

[Selur]

But since the pattern is not fixed, you would have to adjust the offsets for each scene.
I know I could go scene by scene and manually drop fields, but that really is a pain to do for a whole file and not just a snipped.
I'm looking for some way to automate this. :)

[Selur]

@StainLess: Finally found the time to test it, and with this 2pass works fine: Thanks for the help!