ExBlend deblender - v1.04 - 18 Dec 2018

[StainlessS]

New Thread due to Forum restriction on editing thread title when over x number of years old.
Old version here:- https://forum.doom9.org/showthread.php?t=157337

Settings.txt Part 1 of 3 : Post #1 of 6

ExBlend() By Stainless Stephen [StainlessS on Doom9]

http://forum.doom9.org/showthread.php?t=157337
New thread:- https://forum.doom9.org/showthread.php?t=175948


ExBlend() v1.04 [Req VS2008 CPP runtimes]

YV12, YV16, YV24, YV8, YUY2 PROGRESSIVE source. Avisynth v2.5+ (YV12 & YUY2 only for v2.58 plugin)

ExBlend(), is a plugin originally written for telecined clips that were downsized from 720x480 to VCD 352x240.
The damage caused by this, looks similar to damage caused by blend Deinterlacing of telecined clips, which results
in a double blend, every five frames, GGGBBGGGBBGGGBB etc where 'G' is good and 'B' is blend.
Where damage is due to Deinterlacing of telecined clips, it is likely that a denoised detection clip is required
as although the damage looks quite similar, it is not exactly the same.

--------------------------------------------------------------------------------------------

Exblend(Clip,int "mode",int "pal",float "lockthresh",float "ithresh",int "show",string "ExBfile",bool "ver",bool "revip",
string "override",int "lv",int "dv",int "disp",clip "Dclip",int "CompUB",bool "Decimate",int "DecompIx",int "DecClpIx",
bool "Debug")


-------------------------------------
SOME DEFAULTS HAVE CHANGED IN V1.04 |
-------------------------------------



int mode Running mode Default 1

0 Free Running mode (Deprecated)
Mainly for those that might whinge about having to use modes 1 & 2.
In this mode it keeps a record in memory of the blend status of all visited
frames, if you eg scan forwards and then backwards you will get exactly the
same results as the 1st visit. Jumping about is also possible so long as
you have first scanned the entire file although some metrics will be wrong.
1 Scan and Store mode (Default)
As for Free Running mode but saves a copy of the scan status data when the
clip is closed ('ExBlend.XBD'). Use this prior to mode = 2.
Its pretty pointless having filters following ExBlend() in mode 1.
2 Pre-calculated mode
Loads in Scan Status Data saved by Scan and Store mode, this is particularly
useful to examine exactly which frames are unblended and you can jump about
without the possibilty of mismatched detections. You should still not jump
about if you are examining the BlendPair count and Ratio or other counts.
In mode 1, Scan & Store, Scan entire file (with or without metrics) and When
you have everything working right you can switch
to mode 2, (with metrics off). You now can add following filters to your hearts
content and ExBlend() should not take up too much CPU.
Switching from modes 1 to 2 may seems awkward but it is intended that
this filter be used in film restoration and so it is likely that many other
filters will follow this one, having a pre-calculated data file allows
Exblend() to use minimal CPU when adding further filters and checking the
resultant video. For the same reasons, Multidecimate() could also be a good
choice.
v1.03b, added native decimation in mode 2.

int pal -1, 0, 1 , Default 0, lets Exblend() know what the original frame rate was.

This sets the filters hints, after a blended pair is found, the detector will preset
3 (23.976/24 FPS) frames either side of the pair to HINTED meaning No Blend. If 'pal' (25 FPS)
is set to 1, it will preset 4 frames either side to HINTED. This helps it fall
into sync with sequence but may have slight delayed detection if the sequence goes
out of sync. If a frame has already been preset (visited) then the hint will not
be set, this allows hints to work when scanning both forward and backwards.
Not sure if PAL is actually useful or not, never seen any clip that exhibited this
problem with a good/blended signature of GGGGBBGGGGBBGGGG etc, where there are 4
good frames and 2 blends. As it did not add much difficulty to the plugin, PAL was
implemented just incase it was ever found to be useful.

-1 AutoPal mode. (MODE 0 ONLY)
If you are not sure what the original source was you can set PAL = -1 (MODE 0 Only).
If "Unblended:Blended Pairs" Ratio is between (4.0-0.01) and 5.0 at frame 1500 (1500=60*25
= 60 seconds @ 25 FPS), it will auto switch to PAL hints, the first 60 seconds
could contain the odd extra false blend pair detection.
If AUTO PAL is activated, the Metrics P (PAL) flag is hi-lited. It's
not a good idea to rely on this auto pal switch, you should set pal=1 when you
see it in orange. If autopal is activated and you later see the ratio rise above
5.0, then it would seem that the autopal detection was in error, this can happen
if the 1st 60 seconds contains a lot of static scenes with little movement, in this
case you should change to PAL=0 to disable the autopal detection and force it
to 23/24FPS original frame rate. Again, if the 1st 60 seconds contains a lot of static
scenes and it was originally PAL 25 FPS then the autopal may fail to detect and not
auto switch to PAL mode, in this case, ratio will later fall to above 4.0 and below
about 5.0. In this case you should change to PAL=1 to force PAL mode.

0, Forced 24 (or 23.976) FPS original frame rate (Default).
If the original source was 25 FPS and PAL is set
to 0, then this can reduce the effectiveness of ExBlend() due to it erroneously
detecting false blends where it should have been hinted, thus throwing it out of sync.

1, Forced PAL 25 FPS original frame rate. If the original source was 24 FPS and PAL is set
to 1, then this can DRAMATICALLY reduce the effectiveness of ExBlend() due to it hinting
out blends and overriding what should have been a blend detection. This will again, throw
it out of sync.

The R (RATIO) metrics flag will be displayed in orange at any time the ratio is not within expected
bounds, including during the AUTO PAL detection period (if it was a 25 FPS clip originally)
but will disappear once AUTO PAL has switched.


float lockthresh Range 0.0 (Off) to about 2.0, Default 1.0 (No longer considered EXPERIMENTAL, so default 1.0)

The lockthresh arg switches on the locking function of ExBlend().
Purpose being to try and latch on to the sequence when near static scene is encountered.
If the lock is activated, the 'L' Locked flag will be set in metrics.

NoLMN[f]=x.xxxx
L=y.yyyy LMX[f]=z.zzzz

I dont really wanna explain how locking works, it may
change if someone else can implement a better version. However, for locking
to take effect, some pre-conditions have to be satisfied. The frame must
be detected as non-blended, the previous frame should have been hinted, and at
least some of the conditions that would qualify as a blend detection must
be present, so not all frames qualify for the pre-conditions. If a frame does
qualify, then you may see something like L=y.yyyy which would indicate a
locking metric for the current frame, it is this metric that is compared to
LOCKTHRESH, if below or equal, then the frame is locked (converted to a blend
detection). If the frame is locked, the L flag will be set and the metric will
be Hi-lited. Once any frame has been locked, you may see LMX[f]=z.zzzz and
this shows the frame number and lock metric for the frame with the highest
locked metric, yet encountered. There may also be an indication for the frame
with the lowest lock metric that did NOT lock as in NoLMN[f]=x.xxxx
You may want to investigate all of the minimum and maximum flagged frames
both for locking and Ignore functions. Any frame that qualifies and would pass
the locking criteria will fail silently if the merge delta is above the
ithresh Ignore setting, it will not appear or be counted as an ignored frame.

If anyone would like to improve the functionality of lock, then please
be my guest.
This filter outputs a list of LOCKED frames which can be viewed using DebugView,
or may be output to a logging file.

v1.02, can now use eg 'disp=0' to view original blended clip along with metrics to
view frames flagged for investigation for either locking or ignore.


float ithresh 0.0 to 100.0, Default 3.0

There was a weird transition in the
original clip that prompted this filter to be written, it resulted in an even
more weird effect after unblending (looked sort of embossed). This is an
attempt to avoid future similar problems, but I dont have the original problem
video any more so cant really test it, also, the detection logic has changed
a little and so the it may no longer be a problem.
If merge Delta (MD) for a detected blend pair goes above this number, then the
blends will be ignored. I suppose that you could cut down on the number of false
blend detections by lowering this but it probably does not need it,
it works fairly well as it is. If MD exceeds ithresh, the 'I' Ignored flag will
be set and the detected blended pair will be ignored, and a little futher info
will be given when show=5. If there were any IGNORED blends, then it may be a good
idea to take a look at the offending frames both with ithresh at its setting
of 3.0 and with ithresh set to eg 100.0 where the suspect blend will not be ignored.

Written since above, the ithresh setting does seem to work quite well and it's
default setting of 3.0 may cut down on false detections but may IGNORE good blend
detections too, this is conservatively set so as not to allow any bad ones
through. A clip has been observed that went out of sequence every few minutes by
throwing in a triple blend, this ithresh setting seems to avoid the unpleasant
effects of bad detections at these points. You may need to raise this if many
IGNORED blends occur, perhaps in 'not so good clips'.
This filter outputs a list of IGNORED frames which can be viewed using DebugView,
or may be output to a logging file.

v1.02, can now use eg 'disp=0' to view original blended clip along with metrics to
view frames flagged for investigation for either locking or ignore.


dll's for Avs v2.58, v2.60/+ x86 and x64.
Zip includes 3 dll's + source + full vs 2008 project files (~214KB).
See MediaFire or SendSpace in Sig for Downloads

[StainlessS]

Settings.txt Part 2 of 3 : Post #2 of 6

int show Range 0-5
MODES 0 1, and 2 when Decimate == false:

0 Off (Default in MODE 2 only)

1 Show colored block in top left hand of clip when blend detected.

Blue block for 1st blended frame, Red for Second

2 Show blend detection only as in:-

FFF ExBlend a b ?1RPSHOLIMD

FFF shows the frame number

a & b show '-' No Blend, '1' 1st Blend, '2' 2nd Blend
a & b will usually be the same, 'a' is the used detection, which may
have been newly calculated or pre-calculated and stored in a file.
'b' is shown as calculated by the metrics, it is possible
that this is different from a pre-calculated detection.

? is the MODE flag which may be displayed as '¿' (an upside down question
mark) meaning Free running mode 0.
A '?', meaning Mode 1, scan and store mode, where ExBlend creates a precalculated
data file. '!' displayed means Mode 2, Pre-calculated mode, where it will use
data file saved in mode 1.

The '1' flag above, represents the DISP (Display) mode setting and can be
'0', '1', '2', or '3', (the missing mode '4' is not valid unless MODE 2 and Decimate==true
where metrics are different).
'0' represents, disp=0, the original blended clip is returned.
'1' represents, disp=1, ExBlend default returning an identical pair
of unblended frames replacing the blended ones.
'2' represents disp=2, which returns an unblended frame for the first of a blend pair and a
copy of the frame following the blend pair for the second blended frame. This is the SRestore
compatible mode.
'3' Represents disp=3, where the component frames which would be merged together to create the final
recovered frame are returned. In the first blend position, the left component frame is returned and the
right when in blend index 2 postion. You would not in most cases be interested in this.
This setting can be used to view the different return frames using metrics
mode to examine what a frame looks like and the metrics reported for that frame.

The 'R' (RATIO) flag will be displayed hilited at any time the ratio
is not within expected bounds, including during the AUTO PAL detection
period (if it was a 25 FPS clip originally) but will disappear once AUTO
PAL has switched.


The 'P' (PAL) flag indicates that you are in PAL mode. If orange,
then the 'P' flag indicates that the Autopal switch was activated
when the Exblend() argument was set to 'pal = -1' (MODE 0 ONLY, auto detect pal).
The 'F' (FILM) flag indicates Film, if hilited, means that the Autopal switch
was erroneously activated and indicates that you should restart with the argument
set to 'pal=0'. the P and F flags share the same position on screen and
cannot be set simultaneously.
The 'p' (lowercase) flag is shown when 'Auto Detect Pal' is in operation and will
switch to either 'P' or 'F' at frame 1500 where PAL is either detected, or not.


The 'S' (SEQUENCE) flag will display when ExBlend() falls out of
sequence. 1, 2 or HINTED is considered in sequence. 0 is considered out of sequence.
If hilited, means that the current frame is where it went out of sequence.


The 'H' (HINTED) flag indicates that the frame has been hinted.
When Exblend() finds a blended pair, it presets 3 frames on either
side of the blend pair to hinted (NOT-BLENDED), or 4 either side if in
PAL mode.

The 'O' (OVERRIDDEN) flag indicates that the used detection, differs
from the metrics blend detection, this is most likely to happen in static
scenes where false blend detections are overriden by the hints to be
unblended. Will be hi-lited in orange if override not due to hinting.
There is no need to be too concerned about a lot of overrides, the hints
are intended to allow the detector to be quite zealous in its detections.


The 'L' (LOCKED) flag indicates that the (previously EXPERIMENTAL) locking function
has locked onto the sequence dispite a blend detection not being found.
This filter outputs a list of LOCKED frames which can be viewed using DebugView,
or to the logging file.


The I (IGNORED) flag is displayed in the case where the 'merge delta'
exceeds ithresh. The 'I' flag is in Orange if the current frame is
ignored and normal if any frame was ignored (stays on).
Extra 'ignored' info is shown when show=5.
This filter outputs a list of IGNORED frames which can be viewed using DebugView,
or to the logging file.
New in 1.02, writes "New Sequence" and "Lost Sequence" debug messages, can view
in DebugView or in logging file. If you see several "lost sequence", "new sequence"
type messages (in close proximity) in the logs, this is probably down to a false
detection following a lost sequence, this is a prime component for the 'override'
functionality of v1.02.

The 'M' flag denotes a manual override is in effect at the current frame. These
are set using the "override" arg to use the override command file.

The 'D' flag shows if an unblended frame is reconstructed using the DClip (CompUB)
and is hi-lited if adjacent frame also comes from Dclip (CompUB==2).


Show == 2, will also show the colored block in the upper LHS of clip.

3 Show Metrics:-


<--Prev<---><CUR><--><Next->
MD x.xxxx x.xxxx x.xxxx x.xxxx
FD x.xxxx x.xxxx x.xxxx x.xxxx
^ ^
2 1

Right one Hi-lighted for 1st detected blend and Left for 2nd, if blend
pair detected, MD will be smaller than FD and also smaller than the MD
entries on either side. The next frame comes in from the right.
'FD' is Frame Delta (difference) and 'MD' Merge Delta. FD is the delta
between the frames in the clip, when hi-lited it is the difference
between the two frames of the DETECTED blended pair.
MD is the difference between the two copies of the unblended component
frames (that would be merged together to 'iron out' artifacts for the
final recovery frame).
The MD:FD comparison detects that it is a blend (MD should be smaller
than FD), and the MD to MD on either side comparison detects whether
it is in the 1st or 2nd blended pair position. (MD in the hi-lited column
should be smaller than those on either side).

Show == 3, includes show 1 & 2 metrics.

4 Show Blend count and ratio

BlendPairs =w (max=x)
Ratio =y (z%)

Gives a count of detected blend pairs and the ratio of
Unblended to BlendPairs. When you have scanned from start to end of the
clip (without jumping about or going backwards) 'Ratio' should give a
result of 3.xxxx for 24 FPS original and 4.xxxx for 25 FPS original,
meaning 3 good frames and 2 blended for 24 FPS or 4 good and 2 blended
for 25 FPS.
Should only scan forward (Otherwise, BlendPairs & Ratio will
be wrong). Following ratio, the percentage of detected blends is displayed,
based on the current pal mode, this may give a more humanistic idea as to
how well ExBlend() is doing.
Following BlendPairs (max=x) is displayed where x is the maximum number of
blend pairs that could exist in the clip using the current PAL mode, this
will change if AUTO PAL switch is activated. The max value displayed assumes
also that the clip stays in sequence throughout, if it does not, the true
max value could well vary from that shown and the true percentage of blends
found could well be higher or lower than shown, but not by much (small
fraction of a percent, probably).
Also note, ExBlend() may not detect the first or last blends in a clip due
to it needing 3 frames both before and after the current frame to do it's
detections.


Show == 4, includes show 1 to 3 metrics.

5 Shows Lock, Sequence, Overridden and Ignored counts as in:-
[Default when NOT [MODE==2 and Decimate ==true)]

LC=w SC=x OC=y IC=z

In addition, if IC is non-zero, then

IMN[f1]=x.xxxx
IMX[f2]=y.yyyy

Will be displayed where f1 and f2 indicate frame numbers where the minimum
and maximum merge deltas occured that exceeded 'ithresh' and so were
ignored. You might want to investigate these frames with current ithresh
setting and also with ithresh set to 100.0 (ithresh IGNORE switched OFF).
If particularly curious, you might want to set ithresh to 0.0
(Ignore everything), do a complete scan and take a look to see what the blend
with the worst merge delta looks like, if bad, then lower ithresh a little
(tiny weeny bit below y.yyyy). Dont leave ithresh set to 0.0 for the final
scan as this will avoid almost all detections.

New to v1.02, the 'disp' arg allows return of the original blended frames,
or the de-blended frames and so you could say create
a stacked display to show both before and after with metrics, also better than
the above suggested, changing of ithresh.

Seeking to frame zero will reset the statistics. NOTE, if clip Trimmed, then
may not ever go to frame zero, and so frame 0 statistics reset may not work.
(an offhand example of where this might occur [have not checked this] might be
where you script eg "Last.Trim(100,0).Exblend()" so that a new clip never was
produced, ExBlend was invoked on a part of a clip, and so frame zero will not
be encountered within ExBlend)

NoLMN[f]=x.xxxx
L=y.yyyy LMX[f]=z.zzzz

The above is connected to the lockthresh argument of Exblend().
See Lockthresh.

MODE 2, when Decimate==true: (Defaults OFF, for MODE==2, Decimate==true).

show == 0, Off

show > 0, show different metrics.

as In:-

'39 ] Using 48 (* SDDS)'

where '39' is the decimated frame number, 48 is the source clip frame, and '(* SDDS)'
indicates an unblended frame using the source clip for adjacent frames and the DClip
frames for the blended frames.


string ExBfile Default "ExBlend"
ExBlend Project file base name. Used to create any required logging file eg
'ExBlend_1.LOG' when used in MODE==1.
Also used in creating filename of file to store 'mode' 1 Scan and Store data and to also load data file
for mode 2 Pre-calculated mode.
Creates (by default) in local directory where AVS file is located.
In Mode 1, also creates a Decimated Frames command file, by default, 'ExBlend_Decimated.TXT'
for use as a command file giving frame numbers of unblended frames after decimating in MODE==2.

bool ver true or false, Default False
Just shows the version number of ExBlend(), Perhaps other filter writers should
consider whether showing version and copyright stuff serves any positive purpose
(other than ego boosting) when showing metrics. When ver=true, it will show
version info (Also a little ego boosting stuff), without interfering with any
important metrics. ALL args are ignored when 'ver==true'.

bool revip Range false (Off) to true (On), Default false (Off)

Reverses the metrics display as in:-

<--Next<---><CUR><--><Prev->
MD x.xxxx x.xxxx x.xxxx x.xxxx
FD x.xxxx x.xxxx x.xxxx x.xxxx
^ ^
1 2

It may seem more intuitive to have the first blend appear on the left
and the second on the right, but the frames come in from the left.

[StainlessS]

Settings.txt Part 3 of 3 : Post #3 of 6



string Override Default "", (not set), eg "Override.txt" (New to v1.01)
Filename of file to force manual override the blend detections.
In v1.03, is only used in MODE=1. Neither Mode 0 nor Mode 2
allows a manual override, you must force the override's in mode
1, v1.03 extends the override ability and greatly increases it's effectivness.
Also, logging will assist in setting up the overrides file.

Required command format:-

444=h1
555
666=*4

In the above case, frame 444 is set to HINTED, ie Not A Blend. The number
after the 'h' denotes the hinted count, in this case 1, eg 444=h2 would
set two consecutive hinted frames.

555 will be set to a SHORT BLEND PAIR ie (H12H), frame 555 (in this case) will
be set to blend index 1, 556 to index 2, and the frames before and after the
blend pair to HINTED. NOTE, in this case frame 555 is the 1st BLEND index and
NOT the first hinted frame.

Frame 666=*4 will set an override at frame 666 to 4 LONG BLEND PAIRS. A long
blend is shown (in logs) as (H12HHH) for FILM or (H12HHHH) for PAL. The frame
(in this case 666) is the position of the 1st blend index, it will (on only
the first LONG BLEND PAIR) be preceded by a hint, and the blend pair followed
by 3 or four hints depending upon whether FILM or PAL mode is in effect. As in
above case, '666=*4' would yield a result of (H12HHH12HHH12HHH12HHH) if in
film mode, or (H12HHHH12HHHH12HHHH12HHHH) in PAL.
With metrics shown, the 'M' flag will show where individual manual override 'H'
hints were set, or blend index 1 postions of either short or long blend pair
overrides (in the 666 example, it will flag the positions of all the 1's
within the parenthesis['(' and ')'] ).

ExBlend is intended to correct clips that do not stay in sequence, if you have
a clip that does stay in sequence (and you are sure of that), you can force
ExBlend to stay in sequence too using the manual override command file. If say
the first, 1st blend index is at frame 2, you could create a text file containing
the manual override command:-

2=*1000000

Thats it. (you would also need to set correct pal/film mode and such). Here
we gave a command to set 1 million LONG BLEND PAIR's, it will cease when
it comes to end of clip, and in the logs will tell you how many it actually set.
This excessive repetition count will not be considered an error by the override
command parser, however, trying to start a new command outside of clip range will
be considered an error. Also, NOTE, all manual override commands MUST be in ascending
order of frame number, you are NOT permitted to override previously set overrides,
some sort of order has to be maintained. As an aid (and byproduct of this strict
ordering) the parser can tell you (in logs) where the next permissable command
may be set. If trying to set up a manual override command file, use the logs, they
are your friend when trying to do this.

int lv Log file Verbosity. (Default 0, OFF)
0=OFF, 1=ERRORS, 2=WARNINGS & ERRORS, 3 INFO & WARNINGS & ERRORS,
4=VERBOSE INFO & etc, 5 = DEBUG + VERBOSE etc.

lv=4, to show all manual overrides in logs (maybe large logs).

int dv Log to DebugView window Verbosity. (Default 0, OFF)
0=OFF, 1=ERRORS, 2=WARNINGS & ERRORS, 3 INFO & WARNINGS & ERRORS,
4=VERBOSE INFO & etc, 5 = DEBUG + VERBOSE etc.

Can use DebugView program to view this log, New in v1.02

DebugView available here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx

int disp Display mode, 0 to 4, Default 1 (DISP_EXBLEND).

0=Original Blended input clip.

1=ExBlend (default) ie returns two identical copies of deblended frames.

2=SRestore, ExBlend Returns unblended frame for 1st blend index, and a duplicate of the frame following
the blend pair in the 2nd blend index postion.
This is the compatible with what srestore returns, and implemented to better compare ExBlend with same.
INVALID in mode 2 when decimate==true.

3=Component frames. Returns both of the frames that would be merged to give the final unblended frames,
blend index 1 and 2 will return the two component frames.

4=Adjacent frames. Only valid in MODE==2 and when Decimate==true.
Returns frames adjacent to blends, using CompUB and Decompix args.

clip DClip Detection clip, (Default None)

Can use eg a denoised clip for detection instead of source clip.
IF eg a telecined clip exhibiting combing has been vertically downsized a little, it will still appear
combed but cannot be inverse telecined, in this case you might like to try a DClip with the Didee suggested

DClip=blur(0,1.0) # Vertical Blur lines

And use that for the detection clip, you might also like to set CompUB to 2.

int CompUB Component unblend mode (Default 2, [experimental, may change]).
Component Unblend Mode for DISP display mode.
Unblend-ing is achieved in two stages, firstly, two component frames are recoverd, and then the average
of these two components is used to finally recover the unblended frame. Each of the component frames is
unblended using a blended frame and also the good frame adjacent to the blended frame.
CompUB controls which clips are used in reconstruction of the component frames where both Source clip
and Dclip are provided. If no Dclip is supplied then the source clip is always used (Obviously).

CompUB ranges from 0 to 2, where, (below assumes a Dclip)

0 = No frames of Dclip, both blended And blend adjacent frames comes from source clip.
1 = One Frame comes from Dclip ie the blended frame. The adjacent frame comes from source clip.
2 = Both blended and adjacent frames come from Dclip.

An easy way of remembering it is as the number of Dclip frames used in the reconstruction of the Component frames.
(Info:- The Blend detector always uses the Dclip if supplied ie "DDDD").
0 == "SSSS", 1 == "SDDS", 2 == "DDDD", where the two middle letters are the blends and outer are adjacent.

bool Decimate (Default true in MODE==2, INVALID in MODE==0 or MODE==1)
False, off, No decimation
True, Decimate output in mode 2, MODE 1 already calculated which frames would be decimated and creates
a file (default "ExBlend_Decimated.TXT") holding the frame numbers of the unblended frames in the
MODE 2 decimated clip.


int Decompix Decimated Component Index, (Only Valid in MODE=2 when Decimate==true, when Default is 1)

When Decimate is true, (MODE 2 only) the only valid DISP Display modes are 0_DISP_BLENDED, 1_DISP_EXBLEND, 3_DISP_COMPONENT
and DISP_ADJACENT.
Decompix Is used in the DISP==0_DISP_BLENDED, DISP==3_DISP_COMPONENT and DISP==4_DISP_ADJACENT, to select which
frames to return in place of decimated unblended frames.

0_DISP_BLENDED, using both CompUB and Decompix, you can pull out the blended frame from either blend index 1 or 2,
and from either the source clip (most usual) or the Dclip if required, using the CompUB arg.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

1 DISP_EXBLEND is the usual disp mode and returns a fully unblended frame for each pair of blends. Decompix, not used.

3 DISP_COMPONENT, returns the component frame for either blend index 1 or blend index 2 (1st or 2nd blend),
Decompix specifies which of the 2 blend component frames are returned. Can be used to 'pull out' component
frames (1 & 2) from a decimated clip for alternative processing. eg, you could do some kind of denoising on the
component frames before blending them together yourself with eg Merge(). You could then replace your denoised
and merged frames back into the decimated clip, using eg ClipClop() plugin. ExBlend() does no denoising itself.

4 DISP_ADJACENT, returns the source 'S' frame adjacent to the Decompix blend index frame.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.


int DecClpIx Decimated Clip index (MODE==1, default 1: MODE 0 & 1, IGNORED)

This setting (-ve to 255 [default 1]), sets the clip index used in the MODE==1 creation of (default) "ExBlend_Decimated.Txt"
file. This command file is a command file intended for the ClipClop() frame replacement plugin. For ClipClop(), the format
would be like so:-

"1 666"

Where '1' is the clip index number for the replacement clip, and '666' is the frame to replace.
This could be used like this,

FixedClip = ClipClop(ExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

DecClpIx can be used to change the clip index (above=='1') from range 0 to 255 with <0 denoting NONE.


bool Debug Debug logging (false/true, default false)
If true, switches on logging to file or DebugView to lv & dv, 1=Errors, 2=Warnings and 5=Debug, this would only
be used to avoid perhaps the info and Verbose logging. To omit only Verbose, set eg dv=3 (INFO) and Debug=true.
Will not effect log to file unless lv is non zero, ie just DebugView window.

-----------------------------------------------------------------------------------

DebugView here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx


ClipClop() Plugin here:-
http://forum.doom9.org/showthread.php?t=162266


Prune() Plugin here:-
http://forum.doom9.org/showthread.php?t=162446
-----------------------------------------------------------------------------------

Change log :
2/Dec/2009 - ver 1.0 - 1st Public Release

26/Feb/2010 - ver 1.01

Changed blend recovery mode to be identical to srestore, Instead of using recovered frame
at blend index 2, we now use frame following blended pair so the duplicate is of the frame
following blends. Better for compare with srestore.
Switched to using DDigit text renderer.

8/Oct/2010 - ver 1.02b
Lockthresh, no longer considered EXPERIMENTAL, defaults to 1.0.
Added new arg eg Log="Logfile.Log", default ExBlend.Log.
Override commands extended and parser enhanced, assisting messages
added to logs.
Added lv and dv and disp, args.
Added "New Sequence" and "Lost Sequence" and other messages to messages log.
Moved flags around.

Recompiled using the updated DDigit text renderer with Toolkit 2003 optimizing compiler.

15/Feb/2012 - ver 1.03b

Added Dclip, Decimate, CompUB, Dcompix, DecClpIx, and Debug args. Added DISP_ADJACENT.
Lots of changes. Default back to ExBlend instead of sRestore.

18/Dec/2018 - ver 1.04.
Moved to VS 2008, added x64. Added v2.60 standard Planar Colorspaces (excl YV411).

[StainlessS]

ExBlend_QwikRef.txt Part 1 of 2 : Post #4 of 6

EVERYTHING HERE OVERRULES OLD DOCS (See old docs Also)

ExBlend() v1.04 QuickRef

YV12, YUY2, PROGRESSIVE Only. EDIT: Plus YV16 YV24 Y8.

Quick OverView, ExBlend() is a plugin to repair damage caused by blend Deinterlacing of telecined clips, which results
in a double blend, every five frames, GGGBBGGGBBGGGBB etc where 'G' is good and 'B' is blend.
Also used for telecined clips that were downsized from 720x480 to VCD 352x240 which look similar to the above mentioned damage.

Exblend(Clip,int "mode",int "pal",float "lockthresh",float "ithresh",int "show",string "ExBfile",bool "ver",bool "revip",
string "override",int "lv",int "dv",int "disp",clip "Dclip",int "CompUB",bool "Decimate",int "DecompIx",int "DecClpIx",bool "Debug")


Mode 0: Free Running Mode (Deprecated) --------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int -1 1 0 PAL = -1, ONLY allowed HERE in MODE 0
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL, either of these two most usual)
Exbfile string - - "ExBlend" Project: (logging Only), 'exbfile'.XBD Not created.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - - IGNORED (MODE 1 ONLY)
Lv int 0 5 0 Logfile: 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG
Dv int 0 5 0 DebugView: 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)

Mode 1: (ExBlend v1.03+ Scan & Store Data, default, Need to do this before using MODE==2) ------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int 0 1 0 PAL -1 Not Allowed (Error)
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL)
Exbfile string - - "ExBlend" Project: logs,'Exbfile'.XBD & Decimated (Mode 2) Frames Cmd file.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - "" Input Override Command file
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED (MODE 2 ONLY)
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 Clip index used in eg Exblend_Decimated.Txt, <0 == NONE.
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)


Mode 2: (Final Output mode, needs MODE 1 pass 1st.) -------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int - - - Ignored, Set from 'Exbfile'.XBD
LockThresh float - - - Ignored, Set from 'Exbfile'.XBD
IThresh float - - - Ignored, Set from 'Exbfile'.XBD
Show int 0 5 0 Metrics (NON-DECIMATED:0=OFF,5=FULL | DECIMATED:0=OFF,>0 ON) [DEFAULT different to modes 0 & 1]
Exbfile string - - "ExBlend" Project: (logging, Inputs Project Mode 1 XBD file)
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Not used when Decimate (Metrics are different).)
Override string - - - IGNORED, Set in Mode 1
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 4 1 If Decimate, 0_BLENDED, 1_EXBLEND, 3_COMPONENT & 4_ADJACENT valid (2_SRESTORE converted to EXBLEND)
DClip clip - - UNSET Usually as Mode 1 (or NONE), Same colorspace as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool false True True
Decompix int 1 2 1 Decimated Component Index. Valid if DECIMATE & disp=0_BLENDED/3_COMPONENT/3_ADJACENT else Ignored.
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)

[StainlessS]

ExBlend_QwikRef.txt Part 2 of 2 : Post #5 of 6



To use ExBlend(), you need to do a scan of the clip first using eg:

ExBlend(clip, Mode=1) # AssumeFPS(250.0)

or

ExBlend(clip, Mode=1,dclip=DCLIP) # AssumeFPS(250.0)

Where DCLIP is a denoised detection clip if required.

Scan through the file and it will create an ExBlend.XBD file for Mode 2 as in below example.

ExblendedClip=ExBlend(clip, Mode=2,dclip=DCLIP)



DISP: (Display Mode)
Controls which frames are returned instead of Blended Frames, using CompUB & Decompix as additional controls.
Non-Blended frames will ALWAYS return frames from source clip.

Where 'S' is Source Clip Frame, 'D' is DCLIP Detection clip frame (if supplied Else 'S')

In 'mode' 0, 1, & 2 (mode 2 when Decimate == False):

0 DISP_BLENDED If (CompUB !=0), Returns blended frame from D, else blended frame from S.
1 DISP_EXBLEND Blended frames are unblended as for CompUB.
2 DISP_SRESTORE 1st Blended frame is unblended as for CompUB, 2nd replaced with frame after 2nd blend.
3 DISP_COMPONENT Blended frames replaced with COMPONENT frame as for CompUB.

In 'mode' 2 with Decimate==True: (only DISP 0, 1,3 and 4 valid)

0 DISP_BLENDED If (CompUB !=0), Returns BLENDED frame from D, else blended frame from S,
further control via Decompix to select 1st or 2nd blend frame of blend pair.
You might want to set CompUB to 0 here to return the original blended source frame and
not from the DClip.
1 DISP_EXBLEND Blended frames are unblended as for CompUB (Same as NOT Decimated).
3 DISP_COMPONENT Returns COMPONENT frame as for CompUB, Decompix==1 = 1st Component frame, 2 =2nd.
4 DISP_ADJACENT Returns source frame 'S' adjacent to Decompix blend index.


CompUB: (Component Unblend Mode)

Unblend-ing is achieved in two stages, firstly, two component frames are recoverd, and then the average
of these two components is used to finally recover the unblended frame. Each of the component frames is
unblended using a blended frame and also the good frame adjacent to the blended frame.
CompUB controls which clips are used in reconstruction of the component frames where both Source clip
and Dclip are provided. If no Dclip is supplied then the source clip is always used (Obviously).

CompUB ranges from 0 to 2, where, (below assumes a Dclip)

0 = No frames of Dclip, both blended And blend adjacent frames comes from source clip.
1 = One Frame comes from Dclip ie the blended frame. The adjacent frame comes from source clip.
2 = Both blended and adjacent frames come from Dclip (Default [experimental, may change]).

An easy way of remembering it is as the number of Dclip frames used in the reconstruction of the Component frames.
(Info:- The Blend detector always uses the Dclip if supplied ie "DDDD").
0 == "SSSS", 1 == "SDDS", 2 == "DDDD", where the two middle letters are the blends and outer are adjacent.


Decompix: (Decimated Component Index)
When Decimate is true, (MODE 2 only) the only valid DISP Display modes are 0_DISP_BLENDED, 1_DISP_EXBLEND, 3_DISP_COMPONENT
and DISP_ADJACENT (only valid in mode==2 and when Decimate==true).

0_DISP_BLENDED, using both CompUB and Decompix, you can pull out the blended frame from either blend index 1 or 2,
and from either the source clip (most usual) or the Dclip if required, using the CompUB arg.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

1 DISP_EXBLEND is the usual disp mode and returns a fully unblended frame for each pair of blends.

3 DISP_COMPONENT, returns the component frame for either blend index 1 or blend index 2 (1st or 2nd blend),
Decompix specifies which of the 2 blend component frames are returned. Can be used to 'pull out' component
frames (1 & 2) from a decimated clip for alternative processing. eg, you could do some kind of denoising on the
component frames before blending them together yourself with eg Merge(). You could then replace your denoised
and merged frames back into the decimated clip, using eg ClipClop() plugin. ExBlend() does no denoising itself.

4 DISP_ADJACENT, returns the source 'S' frame adjacent to the Decompix blend index frame.
Might be useful to anyone wanting to do their own advanced un-blending in script, using ExBlend() only as the detector.

ExbFile: (ExBlend Project File, [default "ExBlend"], Changed function a little in v1.03)
The Exbfile project name is used to create log file names (eg "ExBlend_1.LOG", where '_1' represents the active
MODE at the time).
In MODE 1 only, it is also used to create eg an "ExBlend.XBD" file which totally replaces
any detection stage in MODE 2 and all of the info required to Decimate in Mode 2 is contained in the XBD file.
Mode 1, also creates an eg "ExBlend_Decimated.TXT", which is a command file for use in a range/frame replacement
plugin. [eg ClipClop()], This "ExBlend_Decimated.TXT" is a file containing frame numbers that can be used to select
MODE 2 decimated output blend recovered frames.
In Mode 2, you can choose to not Decimate, in which case it would be as previous versions of ExBlend and require
later Decimation using your favorite Decimator. Or, you can Decimate if you choose, you can process the
Decimated output of ExBlend 1_DISP_EXBLEND using your favorite Motion Compensated denoiser, and feed the original
ExBlend output and the denoised output into ClipClop, using the ExBlend_Decimated.TXT command file to replace the
ExBlend recovered un-blended frames with the denoised ones, and NOT requiring yet another pass. Pass 1 to generate
the XBD and decimated frames file and Pass 2 to unblend, decimate, motion compensated denoise and replace back
into the ExBlended clip again.



DecClpIx: (Decimated Clip Index, MODE 1 Only)
This setting (-ve to 255 [default 1]), sets the clip index used in the MODE==1 creation of (default) "ExBlend_Decimated.Txt"
file. This command file is a command file intended for the ClipClop() frame replacement plugin. For ClipClop(), the format
would be like so:-

"1 666"

Where '1' is the clip index number for the replacement clip, and '666' is the frame to replace.
This could be used like this,

FixedClip = ClipClop(ExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

In the above, 'ExblendedClip' would be clip 0, and 'DenoisedExblendedClip' clip 1, it would replace the cleaned up frames
back into the deblended clip after eg post Exblend recovery denoising (eg MCDegrain).
You could also use the Prune() plugin to view ONLY the unblended Denoised frames, Prune requires clip index's starting at 0
BUT, the below should work OK.

FixedFramesOnlyClip = Prune(FixedClip,FixedClip,CMD="ExBlend_Decimated.Txt")

OR

FixedFramesOnlyClip = Prune(DenoisedExblendedClip,DenoisedExblendedClip,CMD="ExBlend_Decimated.Txt")

without putting them back into the ExblendedClip clip first.


As Prune()'s clips index starts at zero, we used the input clip name twice so we can use clip 1 (2nd FixedClip) to match
the command file.


To view just the ExBlended frames (without denoise) eg:

Prune(ExblendedClip,ExblendedClip,CMD="ExBlend_Decimated.Txt")


To extract the two component clips recovered frames only, and view them side by side

COMPUB=2 # you choose
C1=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=1)
C2=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=2)
C1_2=(Prune(C1,C1,CMD="ExBlend_Decimated.Txt")
C2_2=(Prune(C2,C2,CMD="ExBlend_Decimated.Txt")

StackHorizontal(C1_2,C2_2)


To extract the two component recovered frames, denoise them, merge together put back into ExBlended decimated clip.

COMPUB=2 # you choose
C1=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=1).MCDegrain(3)
C2=ExBlend(clip,Mode=2,disp=3,CompUB=COMPUB,Decompix=2).MCDegrain(3)

#Return StackHorizontal(C1,C2)

Combine = Merge(C1,C2) # Repaired. Average of the two denoised component frame clips.

FixedClip=ClipClop(C1,Combine,CMD="ExBlend_Decimated.Txt")


As an alternative to using double clip names in Prune, you can change the default clip index in the "ExBlend_Decimated.Txt" (Decimated Frames file)
so that Prune or some other alternative could be used with another clip index, a -ve "DecClpIx" will miss out the clip index altogether,
but you would then have to use another frame replacement plugin alternative to ClipCLop() (Whose index needs to be 1).

Together, Disp, Decimate, CompUB, Decompix, and eg ExBlend_Decimated.Txt allows ExBlend() to expose both
of the component frames (Decompix) fashioned with the desired CompUB method, after eg Motion compensated
processing, merged together to create the finally recovered frame, this result would then be fed back into
either one of the two ExBlend unblended component clips via ClipClop() and the ExBlend_Decimated.Txt command file.

Advanced scriptors may wish to extract the clips containing the blended frames and adjacent frames for both blend index
1 and blend index 2 and do some clever masking and such to recover the unblended frames themselves.

-----------------------------------------------------------------------------------

DebugView here:-
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx


ClipClop() Plugin here:-
http://forum.doom9.org/showthread.php?t=162266


Prune() Plugin here:-
http://forum.doom9.org/showthread.php?t=162446
-----------------------------------------------------------------------------------

[StainlessS]

Post #6 of 6.

Here from an example clip courtesy of JohnMeyer from his thread here:-
http://forum.doom9.org/showthread.php?t=162599


https://s20.postimg.cc/jxhxf8wot/Ex_Blend-_Dog.jpg (https://postimg.cc/image/51je7nla1/)

The clip in question is 3:2 pulldown that has been resized as if progressive.

See Here for extended use demo to fix above(in old thread):- https://forum.doom9.org/showthread.php?p=1558846#post1558846

[Boulder]

I just finished the test encode of my screwed up Dream Theater Blu-ray and you know what? It's perfect! I muxed it to mkv and checked many places and everything seems fine. I really thought that there was no saving that one, but I guess it was tricky enough to challenge the scripters :D

So, thanks a lot and have a Merry Christmas :)

[StainlessS]

Boulder problem here:- https://forum.doom9.org/showthread.php?p=1859719#post1859719

Boulder,
Just curious, did you need blur?
Also, were there left over blends [EDIT: I mean dupes] as in SRestore, and if so did you need DropDeadGorgeous ?

Glad you got it sorted, and a very merry xmas to you too sir :)


EDIT: Had connection problem, double post removed.

[Boulder]

Just curious, did you need blur?
Also, were there left over blends [EDIT: I mean dupes] as in SRestore, and if so did you need DropDeadGorgeous ?

I didn't check the result frame-by-frame, I just watched it play with the normal framerate. The difficult sections like some crossfades seemed ok and longer pans also didn't have any jerkiness. I used the basic script you gave me to try so blur was being used.

[StainlessS]

Ok, thanx.
The script with SHARP=0.333 was just demo to show SHARP use and more a wild guess than anything, default sharp in McDegrainSharp is I think about 0.6,
thought that might be too much.

I can feel Hanfruns itching to post some time soon, have a guddun.

[Boulder]

I have a new clip with some restoration needed:

https://drive.google.com/open?id=1O1uKGsvamTMP33vkNodNAplNFP5oUVnw

I tried figuring out a pattern and it looks like 4 good frames, 2 blended ones. Then if you look closer, it looks more like 3 good frames and 3 blended ones where the middle one is a real blended mess and the surrounding frames much less so :( The standard settings don't seem to work properly because of that.

[StainlessS]

Sorry, cant help with triple blends, good luck in your endeavour.

[Boulder]

Thanks for the confirmation so I won't try hitting my head against the wall with ExDeblend :) I'll post in the SRestore thread, maybe someone has some ideas.

[StainlessS]

Thought I might include this, its in the source folder for coders, but might assist a little for those trying to figure out what it does.

Findblend_Method.txt
[EDIT: Below MD_??_?? is the difference between two potentially unblended 'component' pixels when merged together for final unblended pixel.
eg MD_10_23 and FD_12 etc are the names of actual variables in the source code. FD_12 etc are the actual differences between adjacent frame pixels in blended clip.]
We use the same names pre-prended with "W_" as per frame weighted [0.0 -> 100.0%] variable names.


|-----------------------------------------------|
ExBlend [ FindBlend() ]

Rough description of ExBlend() blend detection
|-----------------------------------------------|


-------- -------- -------- --------
|MD_10_23| |MD_21_34| |MD_32_45| |MD_43_56|
| | | | | | | |
| | | | | | | |
|--------| |--------| |--------| |--------|
| | | | | | | |
_| |\ _| |____ | |____ | |
| \ | \ /| |\ | | | /| |
| \ \ \ / | | \ /| | | / | |
| \ \ \ / | | \/ | / | / | |
| \ \ \_ /__ |_________ | /\/________/_____|____/ __| /|
| \ \__ /_ | ___\___|_/ // / ____|_____/ ________/ |
-------- \ --------/ \--|-----/ -\--|--// \-----/--/ -|------/ --------/
|Frame_0 | |Frame_1 | |Frame_2 | |Frame_3 | |Frame_4 | |Frame_5 | |Frame_6 |
| | | | | | | Current| | | | | | |
| in_n-3 | | in_n-2 | | in_n-1 | | in_n | | in_n+1 | | in_n+2 | | in_n+3 |
| p0 | | p1 | | p2 | | p3 | | p4 | | p5 | | p6 |
-------- -------- -------- -------- -------- -------- --------
| | | | | | | |
| | | | | | | |
-------- -------- -------- --------
Metrics display->| FD_12 | | FD_23 | | FD_34 | | FD_45 |<-Metrics display
ONLY | | | | | | | | ONLY
| | | | | | | |
-------- -------- -------- --------

-------------------------------------------------------------------------------
PER PIXEL VARIABLES:- (at current x,y postion in frame)
----
px = Pixel Luma from Frame_x (x = 0 to 6).
----
FD_ab = abs(pa - pb) Delta between adj frames[a & b] pixels
----
UP_ab = (pa * 2) - pb;
Pixel, unblended from 2 frames, two of these are later merged to recover
original frame. These are the inputs to MD_ab_cd and are not shown above
as it just makes it look confusing (I tried it), but each pair of lines
into MD_ab_cd can be thought of as UP_ab.
The ordering of the frame numbers (a,b) IS IMPORTANT and can be gotten from
the description of MD_ab_cd
----
MD_ab_cd = abs(UP_ab - UP_cd)
Merge Delta, between component pixels of the potentially recovered
(unblended) pixel. If we find a blended pair of frames and at the current
frame position we have the first blended frame, then current frame in_n
(frame_3) will be replaced with a recovered frame the equivalent unblended
pixels of MD_32_45 and the second blended frame in_n+1 (frame_4) also will
be filled with the same content when the next frame is requested,
(after doing all this again), however, next time around, in_n will be the
second blended frame as the contents of everything would have shifted left
by 1 frame, and so, MD_32_45 will NO LONGER be the recovered frame, it will
now be MD_21_34. When the recovered frame is finally rendered it will go
sort of like:-
RecoveredPixel = (UP_ab + UP_cd + 1) / 2; (for luma)
-------------------------------------------------------------------------------
PER FRAME VARIABLES
---
WEIGHTINGS:- (float)
Accumulated weighting for pixels; Obviously clear to zero at beginning.
----
W_MD_ab_cd += MD_ab_cd;
W_FD_ab += FD_ab;
----
METRIC:-
Final scaling of measurements for whole frames before deciding if blended.
----
float mx = (vi.width * vi.height) * 255.0;
For scaling weighting into more managable & meaningful numbers.

W_MD_ab_cd = W_MD_ab_cd * 100.0 / mx;
W_UP_ab = W_UP_ab * 100.0 / mx;
W_FD_ab = W_FD_ab * 100.0 / mx;

If ((W_MD_32_45 < W_MD_21_34) && (W_MD_32_45 < W_MD_43_56))
{
// then, is possible 1st of pair of blends
if (W_MD_32_45 < W_FD_34)
{
// then IS 1st blend index
}
}
else If ((W_MD_21_34 < W_MD_10_23) && (W_MD_21_34 < W_MD_32_45))
{
// then is possible 2nd of pair of blends
if (W_MD_21_34 < W_FD_23)
{
// then IS 2nd blend index
}
}
else
{
// Is NOT a blend
}


Also this in there too (identities.txt).
EDIT: After staring at below gobble-de-gook for a while, added the stuff in blue [will be modded as below in next release].

Below written to assist understanding of how the screwed up clip was created, and is from point of view of the pre-blended clip.

The 'IN' (ie W,X,Y,Z) are frames before the bad blends were created, and 'OUT' is the resultant bad blending sequence that need be fixed,
C and D are the blended pair, and Y is the missing frame that is partially present in both C and D blended pair.
[Missing Y frame is the one that needs recovering].



IN W X Y Z # Pre blended frames.
| | \ /\ /|
OUT A B C D E # Bad frames that we have to deal with.


A = W
B = X
C = (X + Y) / 2 : X = 2C - Y : B = 2C - Y : Y = 2C - X : Y = 2C - B
D = (Y + Z) / 2 : Y = 2D - Z : Y = 2D - E : Z = 2D - Y : E = 2D - Y
E = Z

FD_CD = | ((X + Y) / 2) - ((Y + Z) / 2) | == | ((X + Y - Y - Z) / 2) | == | (X - Z) / 2 |
== | (B - E) / 2 |
== | B - E | / 2

FD_BC = | X - ((X + Y) / 2) | == | (2X - (X + Y)) /2 | == |(2X - X - Y)/2| == | (X - Y) / 2 |
== | (B - Y) / 2 |
== | B - Y | / 2


FD_DE = | ((Y + Z) / 2) - Z | == | (Y + Z - 2Z) / 2 | == | (Y - Z) / 2 |
== | (Y - E) / 2 |
== | Y - E | / 2
# Above FD_nm is expected Frame Difference between frames n and m.

[domator]

Hi StainlessS,
I think your plugin could help me fix a problem with Sliders DVD. I tried using it but I didn't get good results. Maybe you could take a look?
This is the problem:
https://forum.doom9.org/showthread.php?p=1869424#post1869424
And this is original file:
http://5.196.74.194/domek/VTS_02_1.demuxed.m2v
And this is my AviSynth script which didn't fix the problem:
LoadPlugin("Exblend_25.dll")
LoadPlugin("DGDecode.dll")
LoadPlugin("TIVTC.dll")
MPEG2Source("VTS_02_1.d2v")
TFM()
ExBlend()

[StainlessS]

ExBlend not intended for your kind of blending, sorry.

[Blackout_Out]

Hey! I'm getting an error that says "ExBlend ERROR, Mode 2 XBD, Cannot open XBD file," but how do I scan the clip using StaxRip?

Thanks!

[StainlessS]

Maybe best to ask in staxRip thread, I aint ever used it.

[Blackout_Out]

Maybe best to ask in staxRip thread, I aint ever used it.
Fair enough, thank you!

[StainlessS]

Blackout_Out,
Not sure but I seem to remember that XBD path is a bit broken on Vista+ (or maybe W7+),
default XBD directory is I think "C:\", but OS dont allow to be written there so does not exist, need give
either full path or eg ".\MyFile.xbd" for current directory relative, for both Mode=1 and 2.
Hope that helps.
[Above from memory, could be wrong]
EDIT: Also, you need do Mode=1 scan before Mode=2 pass.

[CruJones]

Thanks for the ExBlend plugin StainlessS. I recently taught myself QTGMC deinterlacing with AviSynth to work on a some DVD's I have. When I stumbled upon your plugin, I recognized that telecine pattern of double blend every 5 frames in an old 2 min clip from 1996 that I was hoping to fix. In mode 2 after the XBD scan it definitely improved the footage with 25 fps to get rid of the ghosting from the original but it's not perfect. There's a number of fades from one sequence to another in the editing and in those moments the frames seem a little off. I'd like to try and see if I can correct any of this further but without scripting experience I'm probably way in over my head with this. Any advice is much appreciated though. Cheers!

[StainlessS]

I guess that you're lucky that it worked at all, was really only intended for when NTSC telecine clip was downsized to half height via eg verticalReduceBy2(), producing 2 out of 5 blending.
Fades will/may cause ExBlend to lose sync, and take time to get back in-sync, ExBlend is a bit particular in the exact cause of blending, some have used SRestore() instead for same [likely that fades will prove problem there too].
Where loses sync [at eg fades] might want to look at the Overrides arg [string, filename, MODE=1 only].
Set Override at first blend pair following 'de-sync'.
Good Luck.

EDIT: From Settings.txt

string Override Default "", (not set), eg "Override.txt" (New to v1.01)
Filename of file to force manual override the blend detections.
In v1.03, is only used in MODE=1. Neither Mode 0 nor Mode 2
allows a manual override, you must force the override's in mode
1, v1.03 extends the override ability and greatly increases it's effectivness.
Also, logging will assist in setting up the overrides file.

Required command format:-

444=h1
555
666=*4

In the above case, frame 444 is set to HINTED, ie Not A Blend. The number
after the 'h' denotes the hinted count, in this case 1, eg 444=h2 would
set two consecutive hinted frames.

555 will be set to a SHORT BLEND PAIR ie (H12H), frame 555 (in this case) will
be set to blend index 1, 556 to index 2, and the frames before and after the
blend pair to HINTED. NOTE, in this case frame 555 is the 1st BLEND index and
NOT the first hinted frame.

Frame 666=*4 will set an override at frame 666 to 4 LONG BLEND PAIRS. A long
blend is shown (in logs) as (H12HHH) for FILM or (H12HHHH) for PAL. The frame
(in this case 666) is the position of the 1st blend index, it will (on only
the first LONG BLEND PAIR) be preceded by a hint, and the blend pair followed
by 3 or four hints depending upon whether FILM or PAL mode is in effect. As in
above case, '666=*4' would yield a result of (H12HHH12HHH12HHH12HHH) if in
film mode, or (H12HHHH12HHHH12HHHH12HHHH) in PAL.
With metrics shown, the 'M' flag will show where individual manual override 'H'
hints were set, or blend index 1 postions of either short or long blend pair
overrides (in the 666 example, it will flag the positions of all the 1's
within the parenthesis['(' and ')'] ).

ExBlend is intended to correct clips that do not stay in sequence, if you have
a clip that does stay in sequence (and you are sure of that), you can force
ExBlend to stay in sequence too using the manual override command file. If say
the first, 1st blend index is at frame 2, you could create a text file containing
the manual override command:-

2=*1000000

Thats it. (you would also need to set correct pal/film mode and such). Here
we gave a command to set 1 million LONG BLEND PAIR's, it will cease when
it comes to end of clip, and in the logs will tell you how many it actually set.
This excessive repetition count will not be considered an error by the override
command parser, however, trying to start a new command outside of clip range will
be considered an error. Also, NOTE, all manual override commands MUST be in ascending
order of frame number, you are NOT permitted to override previously set overrides,
some sort of order has to be maintained. As an aid (and byproduct of this strict
ordering) the parser can tell you (in logs) where the next permissable command
may be set. If trying to set up a manual override command file, use the logs, they
are your friend when trying to do this.

[CruJones]

Thanks for the response and info StainlessS. The clip that I was using the ExBlend on was not from a DVD source by the way but rather a TV capture from a Cinemax/HBO program which originally aired in 1996. (I was probably unclear in my first post and see why you were surprised it worked). I’m not sure of the source of the original capture but ExBlend has definitely improved a lot of the footage. Thanks again!

[Julek]

The Canaan blu-ray suffers from a chroma blend, I was trying to fix it via vapoursynth but was getting a worse result than Yousei-Raws.
So I went to see if AVS had something better and ended up discovering ExBlend which has the same result as my script. :D

My deblend function: https://github.com/dnjulek/jvsfunc/blob/main/jvsfunc.py#L148
In some scenes the result was really good: https://slow.pics/c/zz4RFF80
But in others the Yousei-Raws result was cleaner: https://slow.pics/c/Ff8LeIjy
I was wondering what Yousei used to fix it.
The blend probably happened because the video was upscaled from 720p to 1080p, destripe to 360p fixes the luma: https://pastebin.com/2eVi3ijq
Video samples: https://drive.google.com/drive/folders/1IXnaJiE0IFk82vwow9J3MS3ZVBfOx8VM

[Zarxrax]

Something doesn't seem to be working right when I try to use Exblend.
When I use mode=1, it actually seems to detect the blends, and doesn't display the blended frames in the output. But when I use mode=2, it just seems to leave them in and create new blends!

This is just a basic script loading a video through FFMS2 and then using Exblend(mode=1), doing a scan of the whole video, and then changing to mode=2.

Am I doing something wrong?
https://i.postimg.cc/tZXC9RNJ/exblend-mode2.jpg (https://postimg.cc/tZXC9RNJ)

[StainlessS]

Attachments can take forever, and show script probably more informative.
See Here for future Image post:- https://forum.doom9.org/showthread.php?p=1959414#post1959414

IIRC, VISTA+, broke the current directory relative file path stuff, and it might not work properley without
full path [or at least have the ".\" part for current dir relative eg ".\somefile.txt", a plain "somefile.txt" can fail].

So, dont rely on default paths for old Exblend, specify full path or proper current path relative preceding filename with ".\"

EDIT: I think W7 [or maybe W10] defaulted to saving any files without ''.\" in VirtualStore (whatever that is),
again. specifiy full path and ideally not in root of C: drive.

[StainlessS]

Zarxrax, Did you ever find a solution for this ?
Curious if it is possible for avisynth to do anything like this:
Take a frame or short clip from one video, and search for it's location in another video. While the 2 videos contain the same general footage, they may differ greatly in quality, resolution, cropping, and so on.

I have hundreds of clips that I am trying to locate within tens of hours of footage, so some semi-automated method would be much simpler than manually searching through all of the footage for every single clip.

If not avisynth, perhaps it might be possible using ffmpeg or some other tool?

And if not, are you still interested in a possible [maybe - sometimes] solution ?
Once prepped (make SEARCH clip, maybe 320x240 or bit bigger copy of big file, stretching to Max dynamic range PC_Levels, then make fingerprint file),
search on your single FIND Frame should take maybe max 1 second [find or fail].
Could even auto make FIND Clip same as SEARCH clip size etc, pick out say 10 frames from find clip, and show result SEARCH clip frame that matches first found
frame of FIND clip, with a small number of FIND frames (maybe 10) should nearly always find at least one of them.

Would take time to make small dimension SEARCH clip, and then maybe eg 16 mins [on my old XP machine core duo for 2 hours Jurassic Park DVD at DVD rez]
to do the fingerprint file [need keep both small SEARCH clip and fingerprint file]. After that, real fast to find/fail_find frames.
(When first written, was for EXACT same frame to search for, had it doing something like 3 Million frames/ second on the XP core duo, sadly, frames turned out not to be
exactly the same, and I spent 9 months trying to get it working with hit/miss results and quite a bit slower due to match tolerance).
https://forum.doom9.org/showthread.php?t=170332&highlight=RT_QwikScan

EDIT: The first posts in linked thread were of script functions, was later implemented as RT_Stats plugin.

[Zarxrax]

Attachments can take forever, and show script probably more informative.
See Here for future Image post:- https://forum.doom9.org/showthread.php?p=1959414#post1959414

IIRC, VISTA+, broke the current directory relative file path stuff, and it might not work properley without
full path [or at least have the "." part for current dir relative eg ".\somefile.txt", a plain "somefile.txt" can fail].

So, dont rely on default paths for old Exblend, specify full path or proper current path relative preceding filename with "."

EDIT: I think W7 [or maybe W10] defaulted to saving any files without ''." in VirtualStore (whatever that is),
again. specifiy full path and ideally not in root of C: drive.

I updated my post with the image link. It does not seem to be an issue with the file paths. The plugin correctly outputs the file in the same folder as the .avs without me having to specify anything. But I did try manually specifying the path just to be sure.

Regarding your qwickscan function, thanks, I'll hold onto that in case I need something like that again in the future.

[StainlessS]

Zarx,
OK, let me know if you want assistance with the QwikScan thingy. [is some basic functionality to search for frame somewhere in RT_stats thread (needs same frame size - try LumaTol=0.0 for identical frame)].

Can you post your scripts, also which version EXBlend (I aint used original for long time, I last used the Beta, think I fixed the path thing in Beta).

[Zarxrax]

Zarx,
OK, let me know if you want assistance with the QwikScan thingy. [is some basic functionality to search for frame somewhere in RT_stats thread (needs same frame size - try LumaTol=0.0 for identical frame)].

Can you post your scripts, also which version EXBlend (I aint used original for long time, I last used the Beta, think I fixed the path thing in Beta).

Script is simply just FFMS2("input.mov").ExBlend(mode=1)
I am using the 20181218 x64 version (I assume its the latest).

[StainlessS]

OK, it is the latest.

Well I dont know what to say, You script shows that you did not set the ExBFile to anything eg "D:\ExBlend.XBD", suggest that you do give specific path (not C:\) and filename at very least.
Also, the image shown suggests that it only found 2 BlendPairs at frame 419, methinks maybe you just jumped there after clip open, and so metrics mean nothing at all.
Also, unless you did COMPLETE scan from start of clip to end, then XBD file will NOT be created.
Metrics show PAL = -1, (default, ie unknown/auto detect mode), once you figure out what clip you have, then you need set PAL=0 (not PAL) or 1=PAL [Better detect when not guessing].
After FULL scan at correct PAL and Mode=1, then Ratio% should be ideally green and about 95%+, 98% a lot better, less than about 95% forget it, it aint the right detector plugin.
You have HIGH Ratio (too high actually due to not knowing whether should be PAL=0 or 1, and only 2 blends detected).

Suggest do as above, full scan, and check that on last frame that Metrics are at least 95+%, then close script and that ExBlend.XBD exists when you closed. [not written until closed].

Then can switch to Mode=2 (and can also then set Decimate=True otherwise there will be dupes to be decimated in result of mode 2 pass).

EDIT: I think thats about right, long long time since i used it.
EDIT: Seems that in Mode=2, Decimate default is also true, so you dont need set that. [Quickref has table of defaults and settings for each mode].
EDIT: Not sure, maybe you dont need supply ExBFile="Exblend", think should be ok without doing anything, but not sure if path is needed,
I dont have test clip, I shall try with any old clip and see what happens.

[StainlessS]

OK, just tried it with any old clip, did not of course have proper detections, but still created ExBlend.EXB file in current directory [I must have fixed directory stuff on last update],
and no longer needs path, indeed you must just give "ExBFile" or similar, the function of ExBFile has changed a little since older version,
it used to be a filename [and had path probs on Vista+], but has changed to be a Project Name, several files are created using the Project name, and the ".XBD"
part is auto appended to the correct one.

Also, pass 1 can just use

Avisource("...")
ExBlend() # Default is now ExBlend(Mode=1)

And can use VDub2() Video Analasis Pass for fast scan, will stop on last frame to see metrics [It used to jump back to frame 0 until I asked to be changed].
Close file and ExBlend.XBD file and also ExBlend_Decimated.TXT will be created, [ClipClop command file with frame list of frames that will be UnBlended on pass 2,
can use for selective denoise of only Unblended frames of the decimated result].


Pass 2

Avisource("...")
ExBlend(Mode=2) # Default Where Mode=2, is now ExBlend(Mode=2, Decimate=True)


Latest version was big change and many defaults changed with mode. [needed to be changed because of massive possibilites added by decimation and other stuff].
From the QuickRef [default for both mode=1 and mode=2]

Mode 1: (ExBlend v1.03+ Scan & Store Data, default, Need to do this before using MODE==2) ------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int 0 1 0 PAL -1 Not Allowed (Error)
LockThresh float 0.0 100.0 1.0 default rarely needs change.
IThresh float 0.0 100.0 3.0 default rarely needs change.
Show int 0 5 5 Metrics (0=OFF,5=FULL)
Exbfile string - - "ExBlend" Project: logs,'Exbfile'.XBD & Decimated (Mode 2) Frames Cmd file.
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Reverse Metrics Direction when show > 1
Override string - - "" Input Override Command file
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 3 1 Display mode, 0=BLENDED(original), 1=EXBLEND, 2= SRESTORE, 3=COMPONENT, (4=ADJACENT Invalid)
DClip clip - - UNSET Denoised Detection Clip, Same colorspace & dimensions as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool - - - IGNORED (MODE 2 ONLY)
Decompix int 1 2 1 IGNORED (MODE 2 decimated ONLY)
DecClpIx int <0 255 1 Clip index used in eg Exblend_Decimated.Txt, <0 == NONE.
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)


Mode 2: (Final Output mode, needs MODE 1 pass 1st.) -------------------------------------------------------------

Arg Type Min Max Default Comment

clip clip clip clip NONE YV12 & YUY2 PROGRESSIVE
Pal int - - - Ignored, Set from 'Exbfile'.XBD
LockThresh float - - - Ignored, Set from 'Exbfile'.XBD
IThresh float - - - Ignored, Set from 'Exbfile'.XBD
Show int 0 5 0 Metrics (NON-DECIMATED:0=OFF,5=FULL | DECIMATED:0=OFF,>0 ON) [DEFAULT different to modes 0 & 1]
Exbfile string - - "ExBlend" Project: (logging, Inputs Project Mode 1 XBD file)
Ver bool False True False ALL ARGS IGNORED if true
Revip bool False True False Not used when Decimate (Metrics are different).)
Override string - - - IGNORED, Set in Mode 1
Lv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (logfile)
Dv int 0 5 0 0=Off, 1=ERR, 2=WARN,3=INFO,4=VERBOSE,5=DEBUG (DebugView)
Disp int 0 4 1 If Decimate, 0_BLENDED, 1_EXBLEND, 3_COMPONENT & 4_ADJACENT valid (2_SRESTORE converted to EXBLEND)
DClip clip - - UNSET Usually as Mode 1 (or NONE), Same colorspace as Clip.
CompUB int 0 2 2 Component Unblend Mode for DISP display mode
Decimate bool false True True
Decompix int 1 2 1 Decimated Component Index. Valid if DECIMATE & disp=0_BLENDED/3_COMPONENT/3_ADJACENT else Ignored.
DecClpIx int <0 255 1 IGNORED (MODE 1 ONLY)
Debug bool false True False Switch On: ERRORS, WARNINGS and DEBUG logging (ie 1,2 and 5)



To use ExBlend(), you need to do a scan of the clip first using eg:

ExBlend(clip, Mode=1) # AssumeFPS(250.0)

or

ExBlend(clip, Mode=1,dclip=DCLIP) # AssumeFPS(250.0)

Where DCLIP is a denoised detection clip if required.

Scan through the file and it will create an ExBlend.XBD file for Mode 2 as in below example.

ExblendedClip=ExBlend(clip, Mode=2,dclip=DCLIP)


EDIT: Also, its v2.60 colorspace cabable [docs need update].
and Default on mode=1 is PAL=0, not pal, so thats ok.
So really, I presume that you just did not do complete scan of the clip in mode = 1 [but need also check the Ratio % on last frame].

[Zarxrax]

Hmmm, well I know it is NTSC content that was re-edited on an NTSC vhs deck, so I suppose that might have messed with the fields in a way that made it incompatible with this plugin. Also with it being anime, it was mostly animated on the 2's so most of the time there is only 1 visible blended frame per group of 5 frames. I tried the pal=0 setting with no improvement.

You are right that for my mode=1 screenshot, I did directly seek to that area. After some more investigation, if I have it on mode 1 and play from the beginning (or encode the output of the full scan and watch it back) I can see that it doesn't actually get fixed in mode 1. Only if I directly seek there, it will fix that section, I suppose by sheer chance. I just saw that it seemed like the plugin was actually working there and I guess I got my hopes up.

[StainlessS]

NTSC content that was re-edited on an NTSC vhs deck
Well that there would make it fail at [EDIT: unless extreme care taken] every edit and need re-acquire lock on the blend sequences.
most of the time there is only 1 visible blended frame per group of 5 frames
The plug is specifically for 3 good 2 blend sequences, I did not know what caused it when i wrote the plugin,
apparently ActionMan33 [EDIT: ActionMan133] wrote a script to fix similar stuff [and theres at least one more as well as SRestore].
It turns out that [I think] most clips fixable with Exblend are result of
VerticalReduceBy2 (clip), HorizontalReduceBy2 (clip) OR, ReduceBy2 (clip) on Interlaced source [EDIT: 3:2 pulldown source].
However, a few changes made 2 versions ago allowed it to be used to fix a JohnMeyer clip that was not
a result of that downsizing by 2, but had been an NTSC interlaced resize [EDIT: 3:2 pulldown source].

Also, I made mistake in saying was in AutoPal mode, I saw '?' flag and mistook it for the upside down '?' which flags AutoPal mode.

So, sorry, this plug is not for you, this time. :(

EDIT: Sorry, NOT NTSC interlaced resize, NTSC 3:2 pulldown telecine film resize.

EDIT: The PAL mode was to detect 4:2 [good/blend] sequences, I had no idea at the time what it was that caused the problems,
and was easy to add just incase they existed in the wild.

There were at one time a number of "Betty Page" clips available [maybe downsized for Video CD] which were easily available via Google,
[type in her name a a bunch of them were there] these all seemed to demo the problem. [those available last time I tried were much lesser
quality [flv maybe] from sites such as YouTube, but I think still un-exblend perfectly.

EDIT: A quick look and the BP videos are not at top of google list any more [mostly stuff about the Movie - The Notorious Bettie Page].

EDIT: I was actually doing a working version that fixed 3 (or 4) good, 3 blend, but dont know if I've even still got the source.
(the test clips were all artificially created)

EDIT: The original sorta reasonable quality betty page clips were actually mpeg (mpg1 so VCD I think).

[Zarxrax]

Well thanks. I previously wasn't able to obtain a good result with srestore, but I revisited it now and tried a few different settings, and I am getting fairly decent results from it now. Deblending is such a complex topic to wrap my head around.

[Selur]

@StainlessS: Any plans for an Vapoursynth port of Exblend?

[Selur]

I used:
ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LoadDll.dll")
LoadDLL("I:\Hybrid\64bit\Avisynth\avisynthPlugins\libfftw3f-3.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\TIVTC.dll")
LoadPlugin("C:\Users\Selur\Desktop\ExBlend_x64.dll")
Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
# loading source: C:\Users\Selur\Desktop\Courage the Cowardly Dog S02E17E18 SOURCE.mkv
# color sampling YV12@8, matrix: bt709, scantyp: telecine, luminance scale: limited
LWLibavVideoSource("C:\Users\Selur\Desktop\COURAG~1.MKV",cache=false,dr=true,format="YUV420P8", prefer_hw=0)
# current resolution: 720x480
# deinterlacing


TFM() # field match back to a film frames
TDecimate() #remove duplicates
AssumeFrameBased()
org=last
Exblend(mode=1, Exbfile="e:\Exbfile")
#Exblend(mode=0,revip=true, show=0)
StackHorizontal(last, org)


# filtering
# Converting from 8 to 10bit for encoder
ConvertBits(10)
# setting output fps to 23.976fps
AssumeFPS(24000,1001)
PreFetch(16)
# output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last
and called
avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL
Assuming that this would create "e:\Exbfile.XBD" which it does not and thus changing the script to
Exblend(mode=2, Exbfile="e:\Exbfile")
aborts with:
ExBlend: ERROR, Mode 2 XBD, Cannot open BXD file "e:\Exbfile.XBD".

-> How to use this properly?

[StainlessS]

Any plans for an Vapoursynth port of Exblend?
Nope, I dont do the vapourSynth thingy, yet. If somebody else wants to do conversion, I have no problems with that.


TFM() # field match back to a film frames
TDecimate() #remove duplicates
AssumeFrameBased()
org=last
Exblend(mode=1, Exbfile="e:\Exbfile")

Above in red tastes a bit off.
ExBlend is intended for clip that should have been de-telecined prior to resize down to VCD/SVCD size, where would create 3/2 blending [3good/2blend].

Stuff in red would seem to provide exblend with NTSC film rate progressive, not NTSC telecined(29.976), badly resized downwards.

aborts with:
ExBlend: ERROR, Mode 2 XBD, Cannot open BXD file "e:\Exbfile.XBD".

I can only imagine that ExBlend did not find any blended pairs, and so the XBD file is empty/non existing.
[EDIT: If indeed it was the type of badly resized clip that Exblend is intended for, then the stuff in red would likely bugger that up and make ExBlend fail to find anything]
[EDIT: You would best look at Exblend Metrics in Avisynth, and detect %]

[Selur]

Background:
A user of Hybrid wrote to me:
I've got an American NTSC VHS, which is an NTSC 29.98 conversion of a PAL shot on video movie. The original master is lost. I have it captured via S-Video.
and that no matter what he did in Hybrid, he couldn't get rid of the ghosting. Looking at the fields I noticed that some of the fields already had the ghosting.
Since I rembered I got rid of similar ghosting a few years back which ExBlend, I wanted to try with the latest version.

If I run ExBlend in mode=0 it properly finds some of the existing blended frames and 'fixes' them.
https://i.ibb.co/K54n9q8/exblend.png (https://ibb.co/rQ89Z6X)
But seeing that mode=0 is depricated and not the way one should use the filter I wanted to try it using 2passes.
Problem is:
Exblend(mode=1, Exbfile="e:\Exbfile")
doesn't create a 'e:\Exbfile.xbd' file or any other file and thus mode=2 aborts with the above message.
-> So I don't know if this is a bug in ExBlend or whether I don't know how to properly use it.

Cu Selur

Ps.: not sure whether ExBlend was meant for this, but it helps with at least some of the blends which is why I would like to use it. I can stick with mode=0, but thought that since it's depricated it would be wise to figure out how to use the 2pass processing. :)

[StainlessS]

See this thingy here:- https://forum.doom9.org/showthread.php?p=1860483&highlight=exblend#post1860483
where was used with a bit of a Didee kludge, and it worked ok.
Also has in post problem that seemed to come into existence with W7 (maybe Vista), for paths.

Also, just in case you are using older Ver$, here is the last 2018 ver[v1.04]:- https://www.mediafire.com/file/ejqfgiccrpmyj2o/ExBlend_25%252626_x86_x64_dll_v1-04_20181218.zip/file

Also, I seem to remember that no XBD file will be generated unless a complete scan is done {ie, it does not write a partially complete file}
[also, Might possibly abort write if detections were really not good, flagged in metrics in RED].
I think also writes message to DebugView when writing XBD, or tells if skipping write. [check it out].

[Selur]

I'm on Windows 11.

Also, I seem to remember that no XBD file will be generated unless a complete scan is done {ie, it does not write a partially complete file}
I called my clip with "avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL" which runs through the whole thing.

I now opened the script in avsmod, changed the call to
Exblend(mode=1, ExBfile="e:\Exbfile", debug=true, lv=5, dv=5)
downloaded and started Debugview and scrolled through the clip in avspmod.

Tried to save the content of Debugview, but then Debugview, simple froze.
-> took a screenshot and killed Debugview.
https://i.ibb.co/Ph3KRdx/Debugview.png (https://ibb.co/VSrRdsV)
Now E:\Exbfile_1.LOG was created, when I closed avspmod which contains:
ExBlend: INFO,
ExBlend: INFO, ExBlend() v1.04 - 18 Dec 2018 - (c) StainlessS.
ExBlend: INFO,
ExBlend: DEBUG, Limiting args to 'Mode 1'
ExBlend: DEBUG, Checking Colorspace
ExBlend: DEBUG, Checking valid 'mode'
ExBlend: DEBUG, Checking valid 'pal' range
ExBlend: DEBUG, Allocating fdat[]
ExBlend: DEBUG, Allocating MetricsDat[]
ExBlend: INFO, ------------------------------------
ExBlend: INFO, MODE = 1
ExBlend: INFO, PAL = 0
ExBlend: INFO, LOCKTHRESH = 1,00
ExBlend: INFO, ITHRESH = 3,00
ExBlend: INFO, SHOW = 5
ExBlend: INFO, EXBFILE = 'e:\Exbfile'
ExBlend: INFO, VER = False
ExBlend: INFO, REVIP = False
ExBlend: INFO, OVERRIDE = NOT SET
ExBlend: INFO, LV = 5 DEBUG
ExBlend: INFO, DV = 5 DEBUG
ExBlend: INFO, DISP = 1 EXBLEND [Default]
ExBlend: INFO, DCLIP = NO
ExBlend: INFO, COMPUB = 2 0=SSSS, 1=SDDS 2=DDDD [S=Source,D=DClip]
ExBlend: INFO, DECIMATE = False
ExBlend: INFO, DECOMPIX = 1 UNUSED
ExBlend: INFO, DecClpIx = 1
ExBlend: INFO, DEBUG = True
ExBlend: INFO, ------------------------------------
ExBlend: DEBUG, LOG Filename = e:\Exbfile_1.LOG
ExBlend: DEBUG, LOG file Verbosity = 5
ExBlend: DEBUG, LOG DebugView Verbosity = 5
ExBlend: DEBUG, XBD Filename = e:\Exbfile.XBD
ExBlend: DEBUG, FRAMES Filename = e:\Exbfile_Decimated.TXT
ExBlend: DEBUG, ------------------------------------
ExBlend: DEBUG, Constructor, no Errors encountered.
ExBlend: DEBUG, Constructor OUT >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
ExBlend: INFO, [569] Lost Sequence: Detected=2
ExBlend: DEBUG, Destructor IN <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ExBlend: DEBUG, Mode 1, checking if clip fully scanned.
ExBlend: DEBUG, Mode 1, Buffer Incomplete ( 0,3 Percent)
ExBlend: DEBUG, delete[] MetricsDat
ExBlend: DEBUG, delete[] fdat
ExBlend: DEBUG, Destructor OUT (CLOSING LOG) >>>>>>>>>>>>>>>>>>>>
>>>
No e:\Exbfile.XBD or e:\Exbfile_Decimated.TXT file was created.

Cu Selur

[StainlessS]

OK, I think I twigged whats happening,
I called my clip with "avs2yuv64.exe c:\Users\Selur\Desktop\test2.avs NUL" which runs through the whole thing.
think that avs2yuv64 is returning to frame 0 after scan, which I've a sneaky feeling will start afresh ie reset, as if nothing has been scanned.
I'm not on my compiler machine, and not even on windows at present, I'll see what I can do when I can.

EDIT: I've never actually used avs2yuv. [or cant remember it if I have done]

EDIT:
I still doubt that it would do any good if it is to follow this

TFM() # field match back to a film frames
TDecimate() #remove duplicates


but that's a different matter.

[StainlessS]

Selur,
https://forum.doom9.org/showthread.php?p=1846853#post1846853
above linked RT_ForceProcess() will scan entire file and NOT return to frame 0 on completion. [if that is any good]

[Selur]

Thanks will look whether I get RT_ForceProcess working.
a. Okay, the link in that thread doesn't work, but I got the RT_Stats_25&26_x86_x64_dll_v2.00Beta13_20201229.zip from your MediaFire site.
b. Looked at the source the TFM&Decimate isn't needed, source is wrongly flagged as interlaced while it's progressive (+ blended fields).
-> okay, I got no idea how to use that properly.
using:
ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
LoadPlugin("C:\Users\Selur\Desktop\ExBlend_x64.dll")
LoadPlugin("C:\Users\Selur\Desktop\RT_Stats_x64.dll")
LWLibavVideoSource("e:\clips\Courage the Cowardly Dog S02E17E18 SOURCE.mkv",cache=false,dr=true,format="YUV420P8", prefer_hw=0)
org=last
#Exblend(mode=1, ExBfile="e:\Exbfile", debug=true, lv=5, dv=5)
RT_ForceProcess()
#Exblend(mode=0,revip=true, show=0)
#StackHorizontal(last, org)
PreFetch(16)
# output: color sampling YV12@10, matrix: bt709, scantyp: progressive, luminance scale: limited
return last
also doesn't output a file (I could find), but it shows some stuff in the debugviewer.
https://i.ibb.co/DkjR1sv/dv.png (https://ibb.co/d6H5f1V)

Cu Selur
Ps.: Also uploaded the source to my GoogleDrive (https://drive.google.com/file/d/1vKS7UMcCW0sQmiQ42eSCYG3FTHqM1ZEp/view?usp=sharing)

[StainlessS]

Selur,
OK, Exblend only writes XBD file on exblend input clip [EDIT: ExBlend Filter] destructor, so we need to do a little hack for RT_ForceProcess thingy to work.


VideoFileName ="E:\SELUR\Courage the Cowardly Dog S02E17E18 SOURCE.dgi"
DGSource(VideoFileName)
ORG=Last

TMP=ORG
TMP.Exblend(mode=1, ExBfile=".\Exbfile", debug=true, lv=5, dv=5)
RT_ForceProcess() # Force PASS1
TMP=0 # Force Close ExBlend I/P clip [and so also destroy ExBlend Filter clip]. XBD file NOT written until ExBlend destructor is called.

ORG
Exblend(mode=2,ExBfile=".\Exbfile") # PASS2

Return last

EDIT: Above we had a bit of an 'Interoperability' style problem, as RT_ForceProcess returns only a 0 [TwriteAVI::ForceProcess() does return a clip and would probably be more suitable].
So, we had to kill input clip to Exblend instead of output clip. Making Last=0 [instead of "TMP=0"] after RT_ForceProcess() does not work, not sure why.

EDIT: Above PASS2 is decimated result, so any StackHorizontal whatsit will be out-of-sync frames.

EDIT: Metrics on last frame [single pass only] show 92.53% of maximum possible blend pair detections,
I would only use ExBlend() if above about 95% or 96%. [EDIT: Although it is a short clip and does fluctuate about 92% -> 96%, longer clip would be better accuracy %]

[StainlessS]

With stackhorizontal


VideoFileName ="E:\SELUR\Courage the Cowardly Dog S02E17E18 SOURCE.dgi"
DGSource(VideoFileName)
ORG=Last

TMP=ORG
TMP.Exblend(mode=1, ExBfile=".\Exbfile", debug=true, lv=5, dv=5)
#return last Return Pass 1

RT_ForceProcess()
TMP=0 # kill TMP clip and call ExBlend filter destructor

ORG

SHOW=1
Exblend(mode=2,ExBfile=".\Exbfile",Decimate=False,Disp=1,Show=SHOW) # Non decimated, Blend pair replaced with 2 x copies of ex-blended frame. Little colored block flags ex-Blended frame pair.
#Exblend(mode=2,ExBfile=".\Exbfile") # Deblended & Decimated

# Both below lines for Non Decimated ONLY.
StackHorizontal(ORG,Last)
#Subtract(ORG,Last)

Return last

[Julek]

Ps.: Also uploaded the source to my GoogleDrive (https://drive.google.com/file/d/1vKS7UMcCW0sQmiQ42eSCYG3FTHqM1ZEp/view?usp=sharing)

In this case there is no new information in the combed frame, it's just the prev blended with the next, the correct way here is to just drop it like this:


import vapoursynth as vs
core = vs.core

src = core.lsmas.LWLibavSource(r"C:\Users\conta\Videos\src\dog.mkv")
decimate = src.std.SelectEvery(cycle=5, offsets=[0, 1, 2, 4])
decimate.set_output()

But since the pattern is not fixed, you would have to adjust the offsets for each scene.

[Julek]

In this case there is no new information in the combed frame, it's just the prev blended with the next, the correct way here is to just drop it like this:


import vapoursynth as vs
core = vs.core

src = core.lsmas.LWLibavSource(r"C:\Users\conta\Videos\src\dog.mkv")
decimate = src.std.SelectEvery(cycle=5, offsets=[0, 1, 2, 4])
decimate.set_output()

But since the pattern is not fixed, you would have to adjust the offsets for each scene.
I just noticed that the background is true 30 fps, so decimate would not be ideal.
I will try to make a script to replace the damaged part using a mask.

[Selur]

But since the pattern is not fixed, you would have to adjust the offsets for each scene.
I know I could go scene by scene and manually drop fields, but that really is a pain to do for a whole file and not just a snipped.
I'm looking for some way to automate this. :)

[Selur]

@StainLess: Finally found the time to test it, and with this 2pass works fine: Thanks for the help!

[Drommer]

I have a video with a 29.97fps obtained using Telecine and already transcoded using progressive scan. The source material was not 24fps, but 25fps. That is using a sequence of 4 true frames and 2 mixed frames. The sequence of frames is constant throughout the video.

I tried using ExBlend plugin with PAL=1 flag. It generally does what I need, but automatic detection of mixed frames doesn't always work correctly. Sometimes the plugin falsely detects a 3/2 or 5/2 sequence probably because of poor video quality. Trying to change the values of some parameters and not getting the desired result, two questions arose:

1. Is it possible to hard specify the sequence 4/2? That out of every 6 frames of video the last two are always processed as mixed and no other frames are processed.
2. I noticed that deblended frames are visually different. They have more saturation and sharpness which makes them stand out a lot against the real frames. Is it possible any post-processing?

Thank you!

[StainlessS]

In Mode=1,
Set PAL=1.
Also, see "Override", https://forum.doom9.org/showthread.php?p=1860486#post1860486

ExBlend is intended to correct clips that do not stay in sequence, if you have
a clip that does stay in sequence (and you are sure of that), you can force
ExBlend to stay in sequence too using the manual override command file. If say
the first, 1st blend index is at frame 2, you could create a text file containing
the manual override command:-

2=*1000000

Thats it. (you would also need to set correct pal/film mode and such). Here
we gave a command to set 1 million LONG BLEND PAIR's, it will cease when
it comes to end of clip, and in the logs will tell you how many it actually set.
This excessive repetition count will not be considered an error by the override
command parser, however, trying to start a new command outside of clip range will
be considered an error. Also, NOTE, all manual override commands MUST be in ascending
order of frame number, you are NOT permitted to override previously set overrides,
some sort of order has to be maintained. As an aid (and byproduct of this strict
ordering) the parser can tell you (in logs) where the next permissable command
may be set. If trying to set up a manual override command file, use the logs, they
are your friend when trying to do this.

Where above 2 [2=*1000000] is frame number of 1st frame of a blended pair.

2)
In old Exblend thread, there is a demo to show how is possible to process blended frames manually outside of Exblend,
the demo uses a vertically blurred DCLip which you probably do not need.
Anyway, can you at least have a play with it to see if you can figure out how to do it, I'm in pain with broken/dislocated ankle,
and not in best mood nor able to think very straight [I'm on 6 different drugs, x number of times per day].
Thanks. [get back if problems and I'll try to assist as best I can].
https://forum.doom9.org/showthread.php?p=1558846#post1558846
Here from an example clip courtesy of JohnMeyer from his thread here:-
http://forum.doom9.org/showthread.php?t=162599


https://s20.postimg.cc/jxhxf8wot/Ex_Blend-_Dog.jpg (https://postimg.cc/image/51je7nla1/)

The clip in question is 3:2 pulldown that has been resized as if progressive.

Pic1 is the original, 2nd is a Blur(0,1) used as a Dectection clip,
3rd pic is ExBlend() output, and pic 4 is McDegrained ExBlend output.

ExBlend now outputs a command file for ClipClop() frame/range replacement plugin
so that you can use eg McDegrain on the clip and then pluck out the recovered frames
and put them back into the original clip again.
ExBlend also implements now native decimation, so that step is no longer necessary.
In addition, ExBlend, exposes some of it's inner working so that an advanced scriptor
could perhaps do some real clever deblending that is beyond ExBlend.

There follows a script that actually does the deblending itself, and the McDegrained
output of it is indistinguishable from the McDregrained ExBlend output.
There is a little redundancy in the script but I'll give it as supplied in the new version zip.


# ##########################################################################
# Unblend frames in script, use ExBlend() only for detection and decimation.
# Raw frames extracted from source/Dclip and unblended via script,
# then put back into decimated clip.
#
# Requires mt_MaskTools, MCDegrain, ClipClop() & Prune()
#
# ##########################################################################
Import("MCDegrain.avs")

AVISource("problem video.avi")
Crop(2,0,-2,-0) # VDUB DONT Like 636 width)

DCLIP=Blur(0.0,1.0) # Required as bad combing due to resize on Telecine clip.

#Return Exblend(mode=1,CompUB=2,Dclip=DCLIP) # 1st Pass to create ExBlend.XBD

ADJACENT=4
BLENDED=0
#
ORG=Exblend(mode=2,disp=0,CompUB=0,dclip=DCLIP) # Original combed Clip, Decimated
EXB=ClipClop(ORG,ExBlend(mode=2,CompUB=2,dclip=DCLIP).MCDegrain(3),CMD="ExBlend_Decimated.Txt") # ExBlend clip with denoising
# Above, Decimate=true, is default in Mode 2.
# Decimated clip with Frames Adjacent to Blend index 1 (from Dclip, [CompUB==2])
A1=Exblend(mode=2,disp=ADJACENT,Decimate=true,CompUB=2,dclip=DCLIP,Decompix=1)
# Decimated clip with Frames Blend index 1 (from Dclip, [CompUB==2])
IX1=Exblend(mode=2,disp=BLENDED,Decimate=true,CompUB=2,dclip=DCLIP,Decompix=1)
# Decimated clip with Frames Blend index 2 (from Dclip, [CompUB==2])
IX2=Exblend(mode=2,disp=BLENDED,Decimate=true,CompUB=2,dclip=DCLIP,Decompix=2)
# Decimated clip with Frames Adjacent to Blend index 2 (from Dclip, [CompUB==2])
A2=Exblend(mode=2,disp=ADJACENT,Decimate=true,CompUB=2,dclip=DCLIP,Decompix=2)
###########################################################################
# To recover a blend component frame, we use:
# (IX * 2) - A, ie ((blended_frame * 2) - Adjacent_frame)
#
# For Chroma,
# (((IX - 128) * 2) - (A - 128)) + 128
# IX*2 - (128 *2) - A + 128 + 128
# IX*2 - A - (128 *2) + 128 + 128
# IX*2 - A (ie same as for Luma).
###########################################################################
#Below 2 ClipClops NOT NECESSARY, X & Y same frame in non blended frames, however, including the
# ClipClop's, will be faster as non blended frames taken from A1 instead of mt_lutxy() output.
# As mt_lutxy affects all frames we recover the good non-blended frames from any decimated clip (A1), using ClipClop.
# 1st component unblended frames, (IX1 * 2) - A1
Component1=ClipClop(A1,mt_lutxy(IX1,A1,"x 2 * y -",Y=3, U=3, V=3),"ExBlend_Decimated.Txt")
# 2nd component unblended frames, (IX2 * 2) - A2
Component2=ClipClop(A1,mt_lutxy(IX2,A2,"x 2 * y -",Y=3, U=3, V=3),"ExBlend_Decimated.Txt")
###########################################################################
# Optional Denoise Component clips (Recovering from A1 again as McDegrain changes good frames too)
#Component1=ClipClop(A1,Component1.McDegrain(3),CMD="ExBlend_Decimated.Txt") # Can comment out
#Component2=ClipClop(A1,Component2.McDegrain(3),CMD="ExBlend_Decimated.Txt") # Can comment out

# Blend both component frames together to 'iron out' noise, [Should NOT alter good frames (same frame) ie (Z + Z) / 2 = Z].
Fixed = Merge(Component1,Component2) # (C1 + C2) / 2
###########################################################################
# Optional Denoise completely recovered clip. (Restore good frames again).
Fixed=ClipClop(A1,Fixed.McDegrain(3),CMD="ExBlend_Decimated.Txt") # Can Comment out

FixedFramesOnly = Prune(Fixed ,Fixed ,CMD="ExBlend_Decimated.Txt")
###########################################################################
#Return FixedFramesOnly
#Return Fixed

D=ClipDelta(EXB,Fixed)
TOP=Stackhorizontal(ORG,fixed)
BOT=Stackhorizontal(EXB,D)
return Stackvertical(TOP,BOT)
######################################
# ORIGINAL CLIP | SCRIPT UNBLEND #
# ---------------------------------- #
# EXBLEND DENOISE | Diff EXB-SCRIPT #
######################################

Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}



Here @ full size. (Note the FixBlendIVTC version on John's thread has vertical blur applied to all frames
of the output, the above code affects (blur) only the de-blended ones and MCDegrain would have the
effect of sharpening those up again [I think])

https://s20.postimg.cc/5sc4dfnnh/Dog_103b.jpg (https://postimg.cc/image/iwhoq4fp5/)

See new version, on MediaFire via Sig

EDIT:- See also 2 previous posts (exceeded D9 limit post limit).

EDIT: Below links might be useful.
"Problem Video.AVI" (link already included with ExBlend, or get from John's Thread.)
http://www.mediafire.com/?5vaci1oepo3kjb1

ClipClop Plugin.
http://forum.doom9.org/showthread.php?t=162266

Prune Plugin:
http://forum.doom9.org/showthread.php?t=162446

[johnmeyer]

Hah, I saw those stills in StainlessS' post and thought the OP was also working on 1971 Stanford campus protests. I then saw my name ...

I'd be happy to look at a clip of what the person is trying to fix, if one is available.

[StainlessS]

2. I noticed that deblended frames are visually different. They have more saturation and sharpness which makes them stand out a lot against the real frames. Is it possible any post-processing?
That post shows how to manually post process just the ex-blended result frames.

[flossy_cake]

Thanks a lot for this filter! I am surprised to find it is even possible to do this in theory.

I am using it on this VHS transfer (https://archive.org/download/video_archive_553-cd-207/Weeds%20%281987%29%20VHSRip.mp4) which looks like it was converted from 30i hard telecine to 30p using field blend deinterlacing, and ExBlend repairs it nicely and in realtime too (mode=0).

Strangely it seems to work better and produce less "embossed" frames with lockthresh and ithresh both set to 100.0. I don't really know if I should be doing that or if it would create problems with other clips. I will need to gain some more experience with using it on various clips.

For good frame pacing in mode 0 I seem to need to put in its own thread and preroll some frames, but that's probably specific to my system.

Anyway the panning shot at 5:48 and 13:55 the actor's face are showing better results for me with those thresh settings set to 100.0 if that is of any interest to you.

My script looks like this


# source

LWLibavVideoSource(source="Weeds (1987) VHSRip.mp4", stream_index=-1)

# fix blended frames

ExBlend(mode=0, show=0, lockthresh=100.0, ithresh=100.0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5).Prefetch(1)

# smoothing

QTGMC(InputType=2, TR2=3, EdiThreads=2).Prefetch(4)

# fix black level

blackLevel = -24.0
whiteLevel = 255.0/(255.0+blackLevel)

z_ConvertFormat(
\ pixel_type="RGBP8",
\ colorspace_op="601:601:170m:limited=>rgb:601:170m:limited",
\ dither_type="ordered").Prefetch(1)

RGBAdjust(
\ rb=blackLevel, bb=blackLevel, gb=blackLevel,
\ r=whiteLevel, g=whiteLevel, b=whiteLevel,
\ dither=true).Prefetch(1)

z_ConvertFormat(
\ pixel_type="YV12",
\ colorspace_op="rgb:601:170m:limited=>601:601:170m:limited",
\ dither_type="ordered").Prefetch(1)

[StainlessS]

Sorry, cant get to it just yet, am messing with hardware and stuff, also
just tried and Avisynth aint working proper.

Note, LSMashVideoSource is designed for mp4 container, does not need indexing and usually rock solid.

I did down the mp4, I note, "Score by Angelo Badalamenti", the same guy who did Twin Peaks (also Blue Velvet and Mulholland drive, apparently).

I'll havva look when I can.

[flossy_cake]

What a coincidence, I just saw Mulholland Drive last week and had read about that composer. The first thing that struck me about Weeds is how unusually good the soundtrack is. The VHS audio quality is strangely appealing with its wow and flutter. Strange that Nolte would make such an obscure little amateur film after the success of 48hrs.

[StainlessS]

Posted elsewhere onsite

@ Wonkey_Donkey
Penny Mordaunt, Yay oder Nay:- https://www.youtube.com/watch?v=t5DBdeTrgr4

(The Eyes to the left and the Nose to the right. < cryptic :) >)

EDIT: And Endless Loop <could watch it all day>:- https://www.youtube.com/watch?v=tSiplbYDHCU

[kedautinh12]

Hi, try your plugin, the result is good but I got wrong some frames, and I've no idea how to fix them, Do you have any suggestion values and parameters of Exblend to fix them???
Script:
QTGMC(Preset="Very Slow", Sharpness=0).SelectEven()
ORG=Last

TMP=ORG
TMP.Exblend(mode=1, ExBfile=".\Exbfile", debug=true, lv=5, dv=5)
RT_ForceProcess() # Force PASS1
TMP=0 # Force Close ExBlend I/P clip [and so also destroy ExBlend Filter clip]. XBD file NOT written until ExBlend destructor is called.

ORG
Exblend(mode=2,ExBfile=".\Exbfile") # PASS2

Return last
Video: https://www.youtube.com/watch?v=TiFWmEEwbmM

Frame 371
https://i.imgur.com/FyMltRP.png

Frame 375
https://i.imgur.com/cSZRMop.png

Frame 387
https://i.imgur.com/aZFq82I.png

Frame 395
https://i.imgur.com/4NKhQKE.png

Frame 399
https://i.imgur.com/e1hUZic.png

[flossy_cake]

Hi, try your plugin, the result is good but I got wrong some frames, and I've no idea how to fix them, Do you have any suggestion values and parameters of Exblend to fix them???
Video: https://www.youtube.com/watch?v=TiFWmEEwbmM


The field structure appears damaged due to resizing... this is what a field looks like according to SeparateFields():

https://i.ibb.co/7CbW8QD/debug-avs-snapshot-00-36-069.png

So not even QTGMC deinterlacing will be able to produce clean progressive frames for ExBlend to work with, which is probably making it difficult for ExBlend to do its subtraction thing to pull out a clean unblended frame.

The best I could manage was this:


# reduce field structure problem

nnedi3(field=-2).Prefetch(1)
Merge(SelectEven(), SelectOdd()).Prefetch(1)

# deblend to 24fps

ExBlend(mode=0, show=0, lockthresh=100.0, ithresh=100.0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5).Prefetch(1)

# final smoothing & sharpening

QTGMC(InputType=0, preset="slow", TR2=1, FPSDivisor=2, EdiThreads=2).Prefetch(4) # to keep more resolution: InputType=2, FPSDivisor=1



Result: https://drive.google.com/file/d/1mzMobt5uyulSjwKREh9E5Sp1YtBlsp-8/view?usp=sharing

There are still issues like on this sequence "embossed" frames and stutter:

https://i.ibb.co/BGT0jw7/debug-avs-snapshot-01-02-521.png

Perhaps you will get a better result using the prepass mode 1&2 -- I'll have a play around with that later.

edit: tried it just now like your code example with mode1 followed by mode2 and got the same result as mode0 on those problematic sequences.

Am I correct in hypothesising that the issue with embossed frames & stutter is occurring due to the source clip having cadence breaks in the GGGBBGGGBBGGGBB pattern causing deblend to subtract the wrong frame pair?

[kedautinh12]

Thanks for your help, I think I need to add Denoiser to smooth some scenes:
TemporalDegrain2(postFFT=4, fftThreads=2, devId=1)
https://i.imgur.com/SURCkN4.png

[StainlessS]

Sorry guys for not getting back to either of you. :(

About my broken/dislocated ankle,
it is recently much more swollen than it was a few weeks ago,
I suspect that they may be going to cut me open again for a 3rd time,
maybe the screws securing titanium plates have come loose or something.
Anyways, I've been doing [EDIT: almost] nothing since the swelling, except watching youtube videos,
I just feel somewhat pissed off and dejected, I aint much touched avisynth related stuff for several
months, and dont feel in any mood to do so any time soon. (hospital appointment on Friday coming,
where I may learn if I'm gonna be sliced open again).

Again sorry.

EDIT: Some youtube distractions,
GilstrapTV:- https://www.youtube.com/@gilstrap/videos
Texas Bushman:- https://www.youtube.com/@TexasBUSHMAN/videos
Anatoly:- https://www.youtube.com/@vladimirfitness/videos

EDIT:- Crazy Japanese Dinosaur Prank:- https://www.youtube.com/watch?v=0Vw4hXg2kIo
Jurassic Carpark:- https://www.youtube.com/watch?v=Ul26r2rbhE4
World's Scariest Pranks:- https://www.youtube.com/watch?v=vSidTZlSbzY

EDIT: Scary Magical Box Prank (Halloween special :) ):- https://www.youtube.com/watch?v=8yU_dQ_fRVs

[kedautinh12]

Don't worry, Take your care. Get well soon :D

[flossy_cake]

About my broken/dislocated ankle,
it is recently much more swollen than it was a few weeks ago,
I suspect that they may be going to cut me open again for a 3rd time,
maybe the screws securing titanium plates have come loose or something.
Anyways, I've been doing [EDIT: almost] nothing since the swelling, except watching youtube videos,
I just feel somewhat pissed off and dejected, I aint much touched avisynth related stuff for several
months, and dont feel in any mood to do so any time soon. (hospital appointment on Friday coming,
where I may learn if I'm gonna be sliced open again).


I find that sitting is really bad for circulation to the feet and prolongs my foot injury healing times by weeks.

So if you are spending a lot of hours watching youtube in a seated position, try keeping your feet elevated on a foot stool.

When sitting for extended periods, have a look at the veins on the top of your feet, if they are bulging or the skin colour is slightly purplish then there is inadequate egress flow. On the other hand if your feet are pale and white then its an ingress problem.

I saw a vascular specialist recently who said that blood pressure at the ankles is almost nonexistent for most people and it's actually the muscle contractions when you walk that pumps the blood to and from the feet. And the blood flow is what nourishes the cells with nutrients needed to heal. There are also foods like beetroot and rocket leaves which are vasodilators and can help too.

[StainlessS]

Cheers for that Flossy.

About every 3 or so hours I have a half hour on the bed (so raising foot, and reduced feeling of pressure, keeping elevated, a good idea).

Swelling seems to be going down quite a bit (since wearing "The-Boot" thingy for about 5 days indoors, and no outdoor walking).
I've also put an elastic tube bandage thingy back on, been off for about 10 days, perhaps helped reduce swelling.

I'll try get beetroot, next time I'm out of the house.

[tormento]

it is recently much more swollen than it was a few weeks ago
If you have a nearby pool, walk in the water, at least to your hip, for how much as you can, at least every other day.

If you dive (I mean, scuba diving), even better.

I have health issues too, water relieves many.

[StainlessS]

walk in the water
Well last time I saw the doc, I was told [if possible] to avoid getting the surgical wound wet.
(faster wound healing, & avoid possible infection [already had 1 possible infection scare, sorted with antibiotics]).

EDIT:
I think what was the greatest worry was if infection got 'inside' between titanium plates, where they would have to be removed,
and then the infection cured, and then plates returned again (ie two more surgical operations at least).

[Boulder]

Well last time I saw the doc, I was told [if possible] to avoid getting the surgical wound wet.
(faster wound healing, & avoid possible infection [already had 1 possible infection scare, sorted with antibiotics]).

EDIT:
I think what was the greatest worry was if infection got 'inside' between titanium plates, where they would have to be removed,
and then the infection cured, and then plates returned again (ie two more surgical operations at least).

Do you have any of this stuff in the UK? Works wonders on all sorts of nasty wounds, also bigger ones. Pine or spruce resin is something my grandma would have used during the WWII days and it never gets old (she did).

https://www.repolar.com/abilar-10-resin-salve/

[StainlessS]

Cant say that I've seen that stuff, however,
when I was 15 Y.O. at school (about 53 years ago), a lad had something called 'liquid skin' (or similar),
was either 'paint-on' or 'spray-on', but it may have prevented good contact with air/oxygen
which would have hindered healing a bit. (although I think it reduced scalds/burns pain at cost of healing time).

The surgical scar is about 7 inches in length, slightly outside center of back of leg, from heel upwards.
+ two holes where they had a 12 inch screwdriver pushed through my foot (crosswise beneath both knobbly ankle bones),
and for some reason another hole on arch (top) of foot, probably an aid in positioning titanium plates, or something [EDIT: maybe access hole for tightening screws].

The two holes where they hammered in two six inch StainlessS steel nails into shin, are fully healed,
but with weird dents where they were, and with slightly unsettling crosses of scar tissue where the surgeon
made incision in skin prior to hammering in the nails.
The screwdriver and nails were anchor points and part of an 'external fixator' contraption [structure made with a sort of mechano set],
that was in-situ for several days before the 2nd operation.

EDIT: Maybe 'liquid skin' was actually 'plastic skin'.

EDIT: Something [but not exactly] like this [previousy posted]
https://i.postimg.cc/yYt92vG4/44-P010-i250-00.jpg (https://postimages.org/)
Rod marked as 'cancaneal pin' was an actual 12->14 inch screwdriver, really :eek:
I did not have the 'metatarsal pin' in mine as pictured above, and mine was a lot wider,
with mechano set stuff instead of what looks like metal rods above.

[StainlessS]

Anyways, I did not have to have another operation, foot still swelling but maybe gets better one day.

Thought I'de point to a few open source youtube viewers on github, maybe there are better ones.

FreeTube: W10 YouTube client: https://github.com/FreeTubeApp/FreeTube

LibreTube: Android YouTube client: https://github.com/libre-tube/LibreTube
NewPipe: Android YouTube client: https://github.com/TeamNewPipe/NewPipe/releases

[kedautinh12]

YouTube revanced is good too
https://t.me/ReVanced_MMT
Github: https://github.com/inotia00/revanced-patches
https://github.com/inotia00/rvx-builder
https://github.com/inotia00/revanced-integrations
https://github.com/inotia00/revanced-manager

[StainlessS]

I got the Manager APK thing (installs on my 8" Samsung Galaxy Tab S7 Lite, but not 10" Samsung Galaxy Tab A (2019)[instruction set: 64-bit ARMv8a(32-bit Mode)])
But, any way to download the other thing [Revanced_MMT] without Telegram install ? [dont know what it is and dont want to]

[kedautinh12]

Telegram is chatbox like Discord

[johnmeyer]

StainlessS, if you're looking for something interesting to watch on YouTube, using your newly-downloaded viewer, given that you are a guy who actually remembers the 60s, you might get a kick out of this:

25 or 6 to 4 – Chicago (Leonid & Friends сover) (https://www.youtube.com/watch?v=9_torOTK5qc)

It is a group of Russians and Ukrainians who have amazing musical skills and have a love of late 60s and early 70s rock bands who featured brass instruments. They have really caught on and have been touring Europe and the USA the past year. I was going to see them when they came to the San Francisco Bay Area next week, but every last seat is sold out.

The guitar player, drummer, and lead singer in this clip are just amazing. In some of their other videos they feature a woman who is not only an amazing singer, but also drop-dead gorgeous. Great combination!

Best of luck on your recovery. I know a guy who has had both knees, both hips, and both shoulders replaced (one hip has been re-done three times!). He has had some pretty long recovery times, but he has eventually always made it through, and is able to play golf several times a week.

You'll make it!

[StainlessS]

Woah!,
did not know that the homunculus Johnny Galecki (https://en.wikipedia.org/wiki/Johnny_Galecki),
is Ruski, thought he was Belgian.
I was never a big fan of Chicago, but they are pretty darn good, big sound.

But, here one helluva good sound [3rd Street Promenade, Santa Monica, probably the only reason that I'de like to visit the big country],
(three consecutive tunes from the same performance)
https://www.youtube.com/watch?v=Q4HT6AU09CI
https://www.youtube.com/watch?v=-OEr9UYqn2Q
https://www.youtube.com/watch?v=ji6pGc4XRok
And with Ray Chen, one of the world top violinists:- https://www.youtube.com/watch?v=AsIz1xCRQz0

This Ukrainians channel on U tube:- https://www.youtube.com/@KarolinaProtsenkoViolin/videos
Fabulous.

Just got a pair of these (mens slippers, could wear them outside I think) for my dodgy foot (yesterday), quite comfortable.
https://www.primark.com/en-gb/p/faux-fur-slippers-tan-991077657612

I'de quite like to have a full body transplant, then and if that works out OK, a head replacement also.

EDIT: Amazing kid (Seol YoEun, was Wolfgang Amadeus ever this good ?), https://www.youtube.com/watch?v=uuItONGdPjQ
Humoresque (Suzuki Violin 3):- https://www.youtube.com/watch?v=Xf29NXOFcOw
Concert:- https://www.youtube.com/watch?v=QGq5LSfJGPU
EDIT: A reaction video, (cant find the the original):- https://www.youtube.com/watch?v=4LGiZf_M9Pc
Genius 5 Y.O. https://www.youtube.com/watch?v=tn_oLQiSbjc

A bit more Paganini,
https://www.youtube.com/watch?v=POe9SOEKotk
https://www.youtube.com/watch?v=2S24-y0Ij3Y

Roman Kim, Niccolò Paganini mark 2:- https://www.youtube.com/watch?v=KsxY3WL1cP8
This Violinist is NOT HUMAN?? (Reaction):- https://www.youtube.com/watch?v=S4oZZ-4MdlY

Unbelievable, Dimash (SOS):- https://www.youtube.com/watch?v=AUARjexCTlQ
Vocal Coach Reacts (1st time) to DIMASH:- https://www.youtube.com/watch?v=Xiq7KYw8VgM

Heart - Stairway to Heaven Led Zeppelin - Kennedy Center Honors HD(Tribute to Led Zeppelin):- https://www.youtube.com/watch?v=2cZ_EFAmj08
Drummer in above clip is John Bonham's son, his father was Robert Plants best friend,
Hearing Led Zeppelin Stairway to Heaven For the First Time(reaction, she does not say much, too dumb struck):- https://www.youtube.com/watch?v=SNs6BvR3d8E

[johnmeyer]

I've given that Kennedy Center "Stairway" link to dozens of people. The surviving Zeppelin band members' reactions, starting with the introduction of Bonham's son, and then continuing as the stage keeps expanding with more performers, is as amazing as the performance itself. If you've ever read the background on what they wanted to do when they wrote it, they wanted it to get bigger and bigger. The people who produced the tribute realized that dream.

[flossy_cake]

Just thought I would share this setting which seems to help reduce stutters/artefacts for GGGBBGGGBBGGGBB clips

ExBlend(mode=0, lockthresh=0.0, ithresh=100.0, show=0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5).Prefetch(1)
ChangeFPS(60000,1001)

I don't know why but those lockthresh & ithresh settings seem to make it respond more quickly to cadence breaks which NTSC telecine seems to be plagued with.

Outputting the repaired 24 as 60 seems to helps smooth it over a little bit more by making it a 3:2 cadence so that if you do get a small stutter it kind of hides it within the 3:2 judder a bit.

[flossy_cake]

Case study: Mad About You DVD Region 1 by Mill Creek Entertainment (https://www.amazon.com.au/Mad-About-You-Paul-Reiser/dp/B01CIS85YE)

Mill Creek strikes again!

If you read Amazon DVD reviews you may be aware that Mill Creek is a DVD publisher which likes making crappy releases in degraded picture quality. In this particular release their DVD authoring team seems to have taken the hard telecined 480i30 frames and just zoomed them in slightly (possibly to hide edge junk) and in doing so have corrupted the field alignment. So 2 in 5 frames are still combed but it's slightly zoomed in combing which makes it impossible to IVTC or deinterlace. Even bob() or separatefields() won't work since random pixels above/below may or may not contain different temporal data depending on whether or not that line happened to contain scaled pixels from the other field due to the zoom. Basically they have given it the touch of death - here (https://drive.google.com/file/d/1AXernIFPQvULtT-rtYqdep2yNqJ1bYKF/view?usp=sharing) is a remuxed sample.

ExBlend to the rescue...


# source
LWLibavVideoSource("c:\original.mkv", repeat=true)

# blur fields together to convert combing into frameblending, i.e what ExBlend specialises in reverse engineering
vinverse(sstr=0.25)

# ExBlend back to 24p
ExBlend(mode=0, lockthresh=0.0, show=0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5).Prefetch(1)

# QTGMC progressive repair mode 1
QTGMC(preset="slow", inputtype=1, rep0=13, FPSDivisor=1).Prefetch(6)

# aspect
Spline36Resize(720, 540)

# optional - convert to 3:2 to hide some ExBlend stutter on cadence breaks
# ChangeFPS(60000,1001)


Result (https://drive.google.com/file/d/11zc9yAYAAHt_nmx2t0-F8e0h6eRYgf-6/view?usp=sharing)

A substantial improvement. Not great, but not entirely unwatchable. There are some cadence breaks where you can see it momentarily stutter (I left it as 24 instead of 60). I might not throw these discs in the trash where they belong, after all, thanks to ExBlend.

:thanks:

[StainlessS]

ExBlend was my 1st plug for Avsisynth, and the reason I joined D9 was to provide it for users.
Glad that you're finding some additional use for it.

[flossy_cake]

ExBlend was my 1st plug for Avsisynth, and the reason I joined D9 was to provide it for users.
Glad that you're finding some additional use for it.

Well honestly I really am grateful for it. I don't think I would bother watching much content with 2 in 5 blended frames, it just looks so damn ugly with the blending blur and the stutter.

I've been working on my first Avisynth script for the last 6 months, just doing the final testing on it. It's basically an "adaptive IVTC" script that adaptively switches between TIVTC and BWDIF depending on the field cadence. Has a whole latching and learning system kind of like ExBlend would have as well. I went a bit nuts and put in around 100 threshold parameters for latching and learning, using multiple thresholds elevating to higher lock levels, dynamic cthresh & MI depending on scene motion. I'm trying to make it as user friendly as possible with presets like qtgmc, eg. 50% film, 65% film, 80% film etc. and it will automatically set those 100 threshold params to optimise. Made some animation presets too to handle all the duplicate frames in animation cadences like 8:7, 6:4, 5:5, 4:4 etc.

[kedautinh12]

Thank Stainroid for create a very good plugin and waiting for a new script from flossy_cake :D

[flossy_cake]

Thank Stainroid for create a very good plugin

Stainroid?!

[kedautinh12]

He's like chatGPT for avisynth :D

[flossy_cake]

He's like chatGPT for avisynth :D

Oh you mean StainDROID as in android. StainROID on the other hand sounds like roid stain which is something else :eek:

Case study: Mad About You DVD Region 1 by Mill Creek Entertainment (https://www.amazon.com.au/Mad-About-You-Paul-Reiser/dp/B01CIS85YE)
Mill Creek strikes again!
ExBlend to the rescue...


In case anyone benefits from it I've manually inspected all 164 episodes to compile a list of which ones need repairing and here is that list...

Season 1&2
No defects, can be TIVTC'd as usual with optional decombing

Season 3&4
Repair with ExBlend:
ExBlend(mode=0, lockthresh=0.0, ithresh=3.0, show=0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5, denoise=true).Prefetch(1)
QTGMC(InputType=1, preset="slow", Rep0=13, Sharpness=0.0).Prefetch(4)
Spline36Resize(720, 540)
ChangeFPS(60000,1001)

Season 5 E01-E12
Only needs TDecimate from 30 to 24 with optional decombing

Season 5 E13-E24
Repair with vinverse + ExBlend:
vinverse(sstr=0.25).Prefetch(1)
ExBlend(mode=0, lockthresh=0.0, ithresh=3.0, show=0).Prefetch(1).Preroll(8)
TDecimate(mode=1, CycleR=1, Cycle=5, denoise=true).Prefetch(1)
QTGMC(InputType=1, preset="slow", Rep0=13, Sharpness=0.0).Prefetch(4)
Spline36Resize(720, 540)
ChangeFPS(60000,1001)

Season 6 E01-E12
Same as season 3&4

Season 6 E13-E26
Same as season 1&2, except E18 same as S05E13-E24

Season 7 E01-E05, E07, E15
Same as season 1&2

Season 7 E06, E08-E14, E16-E22
Same as season 3&4

edit: reduced QTGMC sharpness from default 1.0 to 0.0 as this show has
lots of baked-in sharpening which seems to be interacting with QTGMC's
sharpener to make it look more sharp than usual.



What a disgusting mess made by Mill Creek Entertainment, and what an incredible feat of reverse engineering to salvage it. Is Mill Creek enshittification (https://en.wikipedia.org/wiki/Enshittification)?

[flossy_cake]

int pal -1, 0, 1 , Default 0, lets Exblend() know what the original frame rate was.

1, Forced PAL 25 FPS original frame rate. If the original source was 24 FPS and PAL is set
to 1, then this can DRAMATICALLY reduce the effectiveness of ExBlend() due to it hinting
out blends and overriding what should have been a blend detection. This will again, throw
it out of sync.


I was just wondering about the pal=1 setting - does this recover 24p from a 25p video, or does it recover 25p from a 30p video?

:thanks:

[StainlessS]

This one.
I was just wondering about the pal=1 setting - does this recover 24p from a 25p video, or does it recover 25p from a 30p video?
NTSC 720x480, DVD stuff where 23.976 film (progressive) has repeat fields inserted to bring it up to 29.97 FPS,
sometimes this 'stuff' was width-cropped 704x480 and downsized to 352x240 causing sequences of 3 good frames / 2 Blended frames, @ 29.97.
At the time of writing, I did not 100% know the cause of the 3 good / 2 blended sequences that ExBlend fixes,
so I just added the PAL option to attempt fix sequences of 4 good / 2 blend sequences.
I aint ever found any 4 good 2 bad sequences that Exblend fixes.

From docs

3:2 pulldown that has been resized as if progressive

So maybe PAL=1 mode would be 4:2 pulldown (dont know if such a thing exists), and resized as if progressive.


int pal -1, 0, 1 , Default 0, lets Exblend() know what the original frame rate was.

This sets the filters hints, after a blended pair is found, the detector will preset
3 (23.976/24 FPS) frames either side of the pair to HINTED meaning No Blend. If 'pal' (25 FPS)
is set to 1, it will preset 4 frames either side to HINTED. This helps it fall
into sync with sequence but may have slight delayed detection if the sequence goes
out of sync. If a frame has already been preset (visited) then the hint will not
be set, this allows hints to work when scanning both forward and backwards.
Not sure if PAL is actually useful or not, never seen any clip that exhibited this
problem with a good/blended signature of GGGGBBGGGGBBGGGG etc, where there are 4
good frames and 2 blends. As it did not add much difficulty to the plugin, PAL was
implemented just incase it was ever found to be useful.

-1 AutoPal mode. (MODE 0 ONLY)
If you are not sure what the original source was you can set PAL = -1 (MODE 0 Only).
If "Unblended:Blended Pairs" Ratio is between (4.0-0.01) and 5.0 at frame 1500 (1500=60*25
= 60 seconds @ 25 FPS), it will auto switch to PAL hints, the first 60 seconds
could contain the odd extra false blend pair detection.
If AUTO PAL is activated, the Metrics P (PAL) flag is hi-lited. It's
not a good idea to rely on this auto pal switch, you should set pal=1 when you
see it in orange. If autopal is activated and you later see the ratio rise above
5.0, then it would seem that the autopal detection was in error, this can happen
if the 1st 60 seconds contains a lot of static scenes with little movement, in this
case you should change to PAL=0 to disable the autopal detection and force it
to 23/24FPS original frame rate. Again, if the 1st 60 seconds contains a lot of static
scenes and it was originally PAL 25 FPS then the autopal may fail to detect and not
auto switch to PAL mode, in this case, ratio will later fall to above 4.0 and below
about 5.0. In this case you should change to PAL=1 to force PAL mode.

0, Forced 24 (or 23.976) FPS original frame rate (Default).
If the original source was 25 FPS and PAL is set
to 0, then this can reduce the effectiveness of ExBlend() due to it erroneously
detecting false blends where it should have been hinted, thus throwing it out of sync.

1, Forced PAL 25 FPS original frame rate. If the original source was 24 FPS and PAL is set
to 1, then this can DRAMATICALLY reduce the effectiveness of ExBlend() due to it hinting
out blends and overriding what should have been a blend detection. This will again, throw
it out of sync.

The R (RATIO) metrics flag will be displayed in orange at any time the ratio is not within expected
bounds, including during the AUTO PAL detection period (if it was a 25 FPS clip originally)
but will disappear once AUTO PAL has switched.

Original source rate is what you will try to recover. [what it was before the blends were erroneously produced in the bad source to be fixed].

[flossy_cake]

I have an NTSC region 1 DVD ("The Ray Bradbury Theater") where some episodes were shot in New Zealand at 25fps film and they appear to have converted it to 30i using a 3:2:3:2:2 cadence with no blended fields. Some other episodes appear to be using 23.0fps with a 3:3:2:3:2 cadence. These episodes look fine when played back at 60p as the judder is not really visibly different from 3:2 judder imo.

[StainlessS]

Is there a question in there ?

It dont sound as if ExBlend can do anything with it.

[flossy_cake]

Is there a question in there ?

Sorry no just a comment. If 25p goes well enough into 30i with 3:2:3:2:2 then there probably isn't any need for the authors to field blend it, and no need to deblend it.

Another one is PAL DVD of Curb Your Enthusiasm which is fieldblended from 30i 2:2 --> 25i 1:1 . This one can't be IVTC'd as each field is a unique image with a different blend amount from the original 30p frames.

[tormento]

Where can I find the latest version of exblend?

[StainlessS]

Where can I find the latest version of exblend?
@ MediaFire in my sig [under any of my posts when logged in]
Or direct link here:- https://www.mediafire.com/folder/hb26mthbjz7z6/StainlessS

[tormento]

Or direct link here
:thanks:

[tormento]

Seems a bit difficult to properly tweak.

Any help about this (https://forum.doom9.org/showthread.php?p=2007251#post2007251) would be welcome.

[StainlessS]

Does not look suitable source to me.
Originally intended for

ExBlend(), is a plugin originally written for telecined clips that were downsized from 720x480 to VCD 352x240.
The damage caused by this, looks similar to damage caused by blend Deinterlacing of telecined clips, which results
in a double blend, every five frames, GGGBBGGGBBGGGBB etc where 'G' is good and 'B' is blend.
Where damage is due to Deinterlacing of telecined clips, it is likely that a denoised detection clip is required
as although the damage looks quite similar, it is not exactly the same.

Use with defaults but with PAL = 0,

int pal -1, 0, 1 , Default 0, lets Exblend() know what the original frame rate was.

This sets the filters hints, after a blended pair is found, the detector will preset
3 (23.976/24 FPS) frames either side of the pair to HINTED meaning No Blend. If 'pal' (25 FPS)
is set to 1, it will preset 4 frames either side to HINTED. This helps it fall
into sync with sequence but may have slight delayed detection if the sequence goes
out of sync. If a frame has already been preset (visited) then the hint will not
be set, this allows hints to work when scanning both forward and backwards.
Not sure if PAL is actually useful or not, never seen any clip that exhibited this
problem with a good/blended signature of GGGGBBGGGGBBGGGG etc, where there are 4
good frames and 2 blends. As it did not add much difficulty to the plugin, PAL was
implemented just incase it was ever found to be useful.

-1 AutoPal mode. (MODE 0 ONLY)
If you are not sure what the original source was you can set PAL = -1 (MODE 0 Only).
If "Unblended:Blended Pairs" Ratio is between (4.0-0.01) and 5.0 at frame 1500 (1500=60*25
= 60 seconds @ 25 FPS), it will auto switch to PAL hints, the first 60 seconds
could contain the odd extra false blend pair detection.
If AUTO PAL is activated, the Metrics P (PAL) flag is hi-lited. It's
not a good idea to rely on this auto pal switch, you should set pal=1 when you
see it in orange. If autopal is activated and you later see the ratio rise above
5.0, then it would seem that the autopal detection was in error, this can happen
if the 1st 60 seconds contains a lot of static scenes with little movement, in this
case you should change to PAL=0 to disable the autopal detection and force it
to 23/24FPS original frame rate. Again, if the 1st 60 seconds contains a lot of static
scenes and it was originally PAL 25 FPS then the autopal may fail to detect and not
auto switch to PAL mode, in this case, ratio will later fall to above 4.0 and below
about 5.0. In this case you should change to PAL=1 to force PAL mode.

0, Forced 24 (or 23.976) FPS original frame rate (Default).
If the original source was 25 FPS and PAL is set
to 0, then this can reduce the effectiveness of ExBlend() due to it erroneously
detecting false blends where it should have been hinted, thus throwing it out of sync.

1, Forced PAL 25 FPS original frame rate. If the original source was 24 FPS and PAL is set
to 1, then this can DRAMATICALLY reduce the effectiveness of ExBlend() due to it hinting
out blends and overriding what should have been a blend detection. This will again, throw
it out of sync.

The R (RATIO) metrics flag will be displayed in orange at any time the ratio is not within expected
bounds, including during the AUTO PAL detection period (if it was a 25 FPS clip originally)
but will disappear once AUTO PAL has switched.

After full scan RATIO flag should NOT be orange, not suitable source.
Also, [EDIT: Below SHOW=4 or SHOW=5 metrics]

4 Show Blend count and ratio

BlendPairs =w (max=x)
Ratio =y (z%)

Gives a count of detected blend pairs and the ratio of
Unblended to BlendPairs. When you have scanned from start to end of the
clip (without jumping about or going backwards) 'Ratio' should give a
result of 3.xxxx for 24 FPS original and 4.xxxx for 25 FPS original,
meaning 3 good frames and 2 blended for 24 FPS or 4 good and 2 blended
for 25 FPS.
Should only scan forward (Otherwise, BlendPairs & Ratio will
be wrong). Following ratio, the percentage of detected blends is displayed,
based on the current pal mode, this may give a more humanistic idea as to
how well ExBlend() is doing.
Following BlendPairs (max=x) is displayed where x is the maximum number of
blend pairs that could exist in the clip using the current PAL mode, this
will change if AUTO PAL switch is activated. The max value displayed assumes
also that the clip stays in sequence throughout, if it does not, the true
max value could well vary from that shown and the true percentage of blends
found could well be higher or lower than shown, but not by much (small
fraction of a percent, probably).
Also note, ExBlend() may not detect the first or last blends in a clip due
to it needing 3 frames both before and after the current frame to do it's
detections.


The Ratio % (above "Ratio =y (z%)") should be at least about 95% otherwise
is unlikely to be suitable source for fix.

Was never envisaged to be useful for anything other than badly converted NTSC DVD downsized for VCD/SVCD.

EDIT: At a minimum, there should be constant stream of 3 good frames, and 2 bad/blurred frames.

[flossy_cake]

The holy grail would be to have an argument where the user can specify arbitrary patterns like ExBlend(pattern="GGGBBBGGBB"). It's painful to think about how this might be implemented.

[flossy_cake]

@StainlessS

Hello again StainlessS a year later!

I am still happily enjoying ExBlend - that whole box set of Mad About You was saved by your Avisynth filter, otherwise that show would have been lost to the sands of time.

I was wondering if you might be able to comment on this...

I have 25p video where every frame appears to be approximately a 30% blend with the previous frame. Here is a short clip (https://drive.google.com/uc?export=download&id=1yVliVchnWchSZWC_i0ZDL-vyqV2QsbBN) (the tip of the spears they are holding exemplify it)

In theory is it possible to deblend this?

Cheers

[StainlessS]

Sorry, is for an very specific type of double blending only.

[flossy_cake]

Sorry, is for an very specific type of double blending only.

No worries, I am more interested if you think it's possible in theory? Like, is that kind of frameblending pattern reversible in your opinion, or is information already lost in a way that would make it not even logically possible?

The way I'm imagining it in my mind, if every current frame is a 30% blend with the previous frame, then does that mean all I would need to do is "subtract" some kind of 30% dimmed version of the previous frame from the current frame? Something like that?

edit: after mucking around with Overlay(mode="subtract") it seems the answer is no - I can subtract a Levels() dimmed version of the previous frame, but that previous frame itself has a blend with its own previous frame and that is causing dark halos around moving objects (the dark bits are where it's subtracted the bits of the blend from 2 frames ago). If I go into double rate deinterlacing mode the blend alternates between current and next frame so that might allow some more information into the system to reconstruct the original unblended frame, but just thinking about how to subtract them in both directions to "get out" the original frame is doing me a concern and I think I'll just give up :)

[wonkey_monkey]

This seems to work fairly well:

prev = last.trim(0,-1) + last
expr(last, prev, "x y 0.3 * - 0.7 /")

https://i.imgur.com/V0czS2E.png

But the black background of the clip may be hiding things. Can you provide a more natural clip?

In theory, you could add back in an even smaller amount of two-frame separated information to compensate for newly-introduced haloes, and so on to infinity, but in practice this approach doesn't usually turn out that great (one problem with your subtract method is clipping at each iteration; you could try converting to float, but I think my expr method is preferable).

[flossy_cake]

This seems to work fairly well:

prev = last.trim(0,-1) + last
expr(last, prev, "x y 0.3 * - 0.7 /")


But the black background of the clip may be hiding things. Can you provide a more natural clip?

In theory, you could add back in an even smaller amount of two-frame separated information to compensate for newly-introduced haloes, and so on to infinity, but in practice this approach doesn't usually turn out that great (one problem with your subtract method is clipping at each iteration; you could try converting to float, but I think my expr method is preferable).


Amazing, I didn't even know about this expr function until now.

And I'm learning about "reverse Polish notation"... AI explained it to me with push/pop logic and I seem to understand it actually means (x-0.3y)/0.7 where x=last, y=prev. So "for each YUV channel, multiple prev clip's value by 0.3, subtract it from the current clip's channel, then brighten it back to where it was by dividing by the reciprocal (0.7)"

It seems to work well!

I tested on some scenes, a value of 0.3 works best, on other scenes 0.2 works best. Maybe cause the blend amount drifts slightly in a 29.97 to 50.0 blend. For now I pick a median value of 0.25

But there is a problem: scenechanges. The source fieldblends over scenechanges too there can be a luminance flash on scenechanges. I need to detect scenechanges and just output the source clip on those. No problem, easy to do in a scriptclip, but scenechange detection isn't totally foolproof

Here is a remux clip with some motion on other coloured backgrounds if you can be bothered looking at it and maybe offering a suggestion how to "add back in an even smaller amount of two-frame separated information to compensate for newly-introduced haloes": https://drive.google.com/uc?export=download&id=1ViyLuJcg9XVddgbKeHE6z8uyK9k6FMlU

It's the PAL DVD of Ernest Goes to Africa that I just bought off ebay, MPEG2 576i25. I'm guessing the movie was probably shot at 30fps or something like that and fieldblended to 50i. There are a couple of rogue scenes in the intro which look like they were shot at 24 but the rest of the movie is all 30 so that blend pattern is pretty much a lock so luckily we don't need some sophisticated locking algorithm like StainlessS has implemented in ExBlend to deal with cadence breaks. There probably are cadence breaks in Ernest too but it just ends up not mattering for whatever reason, the blend direction is always the same

NOTE: it's TFF and needs half rate deinterlacing to produce the blend pattern we've been discussing. I also tried just vinversing instead and that results in frames with an even smaller blend with prev+next, like maybe 10% in both directions. Fieldmatching first doesn't seem to help from what I can tell but I could be wrong.

:thanks:

[flossy_cake]

Somethings not adding up.

A frameblended 30i to 25i conversion shouldn't have this repeatable pattern of "every deinterlaced frame is a x% blend with previous frame". Mathematically it should have a VARIABLE blend pattern, and 1 in 5 frames should NOT be a blend. This is what I get in synthetic tests with ConvertFPS() (which uses frameblending).

Even if there are field cadence breaks, which I can simulate on a 30p sample with SeparateFields().DeleteFrame(0).Weave(). But maybe I'm not simulating the cadence break quite right, because I'm fairly confident this movie was shot at 30 so there must be some magic bodge that touches it in just the right way that makes every deinterlaced frame have the same blend amount.

[wonkey_monkey]

30fps would be incredibly unusual. There's no judder so I think it's just 24p sped up to 25p but badly transferred.

I didn't have much luck cleaning up residual ghosting, but I did notice that if you full rate deinterlace then it alternates between frames having a ghost of the previous frame and frames having a ghost of the future frame. So you can apply the same expr technique to both of these, and then merge. That way you end up with some future and past negative ghosts, but on average they're less visible.

Even scene changes don't come out too badly.

ffmpegsource2("remux.mkv")
bob(0.0, 1.0)

a = selecteven
prev_a = a.trim(0,-1) + a
a2 = expr(a, prev_a, "x y 0.2 * - 0.8 /")

b = selectodd
next_b = b.trim(1,0)
b2 = expr(b, next_b, "x y 0.2 * - 0.8 /")

return expr(a2, b2, "x y + 0.5 *")


(but use a better bobber)

Or if you can stomach a bit of residual interlacing at scene changes:

ffmpegsource2("remux.mkv")
separatefields

a = selecteven
prev_a = a.trim(0,-1) + a
a2 = expr(a, prev_a, "x y 0.25 * - 0.75 /")

b = selectodd
next_b = b.trim(1,0)
b2 = expr(b, next_b, "x y 0.25 * - 0.75 /")

interleave(a2,b2)
weave

[flossy_cake]

30fps would be incredibly unusual. There's no judder so I think it's just 24p sped up to 25p but badly transferred.

I didn't have much luck cleaning up residual ghosting, but I did notice that if you full rate deinterlace then it alternates between frames having a ghost of the previous frame and frames having a ghost of the future frame. So you can apply the same expr technique to both of these, and then merge. That way you end up with some future and past negative ghosts, but on average they're less visible.

Even scene changes don't come out too badly.

ffmpegsource2("remux.mkv")
bob(0.0, 1.0)

a = selecteven
prev_a = a.trim(0,-1) + a
a2 = expr(a, prev_a, "x y 0.2 * - 0.8 /")

b = selectodd
next_b = b.trim(1,0)
b2 = expr(b, next_b, "x y 0.2 * - 0.8 /")

return expr(a2, b2, "x y + 0.5 *")


(but use a better bobber)

Or if you can stomach a bit of residual interlacing at scene changes:

ffmpegsource2("remux.mkv")
separatefields

a = selecteven
prev_a = a.trim(0,-1) + a
a2 = expr(a, prev_a, "x y 0.25 * - 0.75 /")

b = selectodd
next_b = b.trim(1,0)
b2 = expr(b, next_b, "x y 0.25 * - 0.75 /")

interleave(a2,b2)
weave


Great, thank you very much. I feel like you just took this DVD from unwatchable to watchable :) :thanks:

And it works for the next movie in the series too so that one is saved from the trash heap as well.

I think it was just the last 2 movies that were shot on video, AI says it was likely shot at 29.97 but not definitively.

That blend pattern could be the result of a different blending algorithm or multiple conversions. The streaming service version is even worse, its 30p but has what appears to be 1:1:1:2 judder combined with a highly variable blend pattern. The fact there even is a 1:1:1:2 judder underneath all the blending seems to imply you are correct the movie probably was shot at 24p, I will need to do some more synthetic tests with ConvertFPS() . Maybe it's 24 blended to 50, or 24 blended to 25 and then framedoubled to 50.

[wonkey_monkey]

I think it was just the last 2 movies that were shot on video, AI says it was likely shot at 29.97 but not definitively.


Never believe AI. That is definltely 24fps film (sped up to 25fps). They were released direct to video.

[flossy_cake]

Never believe AI. That is definltely 24fps film (sped up to 25fps). They were released direct to video.

I think you are right that its 24fps originally but I think it's frameblended to 25i rather than sped up , cause i compared the music to the 30fps streaming service version and it sounds the same pitch

[flossy_cake]

I guess it doesn't matter if they are the same pitch cause we can pitch correct stuff anyway with TimeStretch, but the runtimes are almost identical only about 3 seconds difference

[flossy_cake]

Actually that's not adding up either, because even if it was 24 blended to 25, i don't think you can just deblend it to a 25p 1:1 cadence. Because deblending would just remove the blends that are attempting to smooth it out and you'd end up with stutter again.

So yeah, no idea what's going on with this movie.

I guess it's possible the 30fps streaming service version is actually based on a PAL 25fps speedup version that was frameblended to 30, and maybe that's why the runtimes are the same and also explains the uneven blend pattern (otherwise i would have expected the GGGBB pattern which ExBlend could have fixed)

Only one way to know for sure - buy the region 1 DVD. Not sure I wanna go that route as I've invested so much in this PAL version and I'm really happy with the result now. Kind of just wanna take the victory and call it a day lol.

Ran off an encode with x264 crf13 and quite happy with the result. Ended up lowering the deblend ratio to 0.175 as there are some scenes with actors heads moving on a bright blue sky background and the LCD-overdrive-esque trails were a bit too noticeable.

[flossy_cake]

Ordered the region 1 DVD — must get to the bottom of this :devil:

[flossy_cake]

Ordered the region 1 DVD — must get to the bottom of this :devil:

Aaaaand it's (https://www.amazon.com/Ultimate-Ernest-Collection-Jim-Varney/dp/B016P0GYP8?ie=UTF8) complete trash :eek:

Actual resolution: 240p and aliased to heck, plus it still has blended fields following no lockable pattern. And it's cropped to what looks like about 14:9 and then stretched horizontally to fill 16:9. Just complete trash, straight into the bin.

Luckily it contains some other unreleased movies and sketches and those are all in good NTSC quality, IVTC-able to true 480p with no field blending or vertical resolution loss.

So the PAL deblended versions of those 2 movies (Ernest Goes to Africa, Ernest in the Army) are actually the best versions available would you believe it! Now I feel like I got my time's worth from investing Avisynth repair time into it. The scene changes were still bad on the PAL versions after deblending, but managed to reduce the visual disturbance by using vinversed frames for the scenechanges in the first movie, and TFM.vinverse for scenechanges in the second movie. Then the amount of residual frameblending over scenechanges is very small and far less noticeable. The rest of the frames can all just be a subtraction of 17.5% of the previous frame for the first movie, and 17.5% of the next frame for the second movie. There were a couple of rogue scenes in the second movie where the blend direction flipped but roughly 99% of the movie is blended in the same direction thankfully so it didn't need manual editing of every scene.

I'll post some remux clips and such later if anyone's interested?