AutoOverlay: auto-alignment and color matching

[introspected]

Github: https://github.com/introspected/AutoOverlay
Releases: https://github.com/introspected/AutoOverlay/releases

AutoOverlay is an AviSynth plugin that provides tools for auto-alignment and color matching, enabling the seamless combination of different video clips in various ways. It automatically calculates optimal alignment parameters such as position, scale, and rotation using frame-by-frame analysis, while adjusting colors to ensure consistency across sources.

https://preview.ibb.co/eNbPO9/script000188.png (https://ibb.co/mAYSbU)

[Aktan]

Sounds interesting! Good job =).

[lansing]

Is this like the auto-align feature in photoshop?

[introspected]

Is this like the auto-align feature in photoshop?
In general, yes. But plugin supports only basic transofrmations such as resizing and rotation to align two clips.

[spoRv]

Thanks again for your great plugin!!!

[StainlessS]

LoadPlugin(".\AvsFilterNet.dll")
LoadNetPlugin(".\AutoOverlay_netautoload.dll")
return blankclip


Avisynth Open Failure
Unable to load plugin, make sure it is a valid .Net assembly

What is the minimum required OS for this plug ? (Above on XP32).

Thanx.

EDIT: Installed
.Net Framework 4 Client Profile
.Net Framework 4 Extended

[StainlessS]

Failing an answer from Introspected (the dev), has anybody had it working on XP ?
(I'm not sure if my newish setup is screwed up,
my old PowerDVD5 [or maybe 6] takes about 60 seconds [or more] to start up, but that should not need .NET at all,
something aint right somewhere.)

[introspected]

Full .NET 4.0 is required not client profile (forgot to change it). Try reinstall extended version, not just update.
XP is supported in theory.
Maybe there are some issues with target .NET settings in the project.
I have no time to test it now.

[StainlessS]

Thanx for your answer.
I did not pre-setup .NET 4 prior to running WususOffline update, presumed that it would do the full install.
(It certainly took a helluva a lot of time updating .Net 4 hotfixes/updates, dont know what it could have been
doing if it did not install the full thing).
Anyways, I'll re-do an install, and re-wusus again, thanks.

EDIT: No idea what the client profile etc does, I know nothing of .Net.

EDIT: Dont know if you saw this post in Usage, I'm doing a script to Spatially Align clips.
https://forum.doom9.org/showthread.php?p=1836806#post1836806

[introspected]

Try this build: https://www.dropbox.com/s/3ktixdpyj6ff9na/AutoOverlay_0.1.9.zip?dl=0
If OK then I will release it on github.

[StainlessS]

Well I've downloaded and loads plugins OK, not really had time to test it out though, will do soon.
(Thought I'de post as I saw you on-line).

Thanx muchly :)

EDIT: Above, without reinstall of .Net.

EDIT: Seems to work just fine, although not much idea what I'm doing just yet.
EDIT: I'll still work on Spatial Align script function as I want to be able to hack it for differing purposes.

[StainlessS]

Introspected,

that seems to be one hellova good plugin that you've constructed, nice job :)

Small typo in the examples

statFile=”c:\test\Overlay.stat”


Uses slanted double quotes rather than ordinary ones.

Thanx again, its bloody great !!!

[introspected]

StainlessS,
thanks for testing. Released on github.
Quotes fixed.

What do you mean about spatial align algorithm? The function from your link is not clear for me.

[StainlessS]

I meant that I'm experimenting on using the Zebra Bands (together with RT_Stats RT_LumaCorrelation) for matching of clips [EDIT: frames],
Zebra(Row=False) for Horizontal alignement, and True for Vertical.

The Zebra function was implemented to detect this JohnMeyer clip problem, "Bad 1950s Kinescope - Hopeless? ":- https://forum.doom9.org/showthread.php?t=167591&page=2

Where it detected a bright band travelling vertically at each frame.
EDIT: RT_YInRangeLocate() was used to locate the bright bar within the YIR Zebra Band.

The Equid function is just a convienent viewer for comparing frames and bands (might be of some use to somebody else).

Its quicker to resize/measure a thin zebra band than the entire frame.

This is one of the JohnMeyer examples where the Zebra band is evident
https://s12.postimg.cc/5l3g3zmj1/YIRL_zps21bcdbdc.jpg (https://postimg.cc/image/3tah93361/)

Or with CMAD (Centered Moving Averages Diameter=13)
https://s24.postimg.cc/wuwda1vkl/CMAD_zpsae30b66c.jpg (https://postimg.cc/image/qtyocz8y9/)

I got the idea of the Zebroid thing (daft experimental) from
[B]Automatic color enhancement and scene change detection of digital video, Jyrki Korpi-Anttila.

Logical Page 36(internal page number), physical page 39(pdf page).
Chen and Panchanathan [95] proposed a region based X-ray image method to detect
tilts, pans and zooming. Each frame is first partitioned into 9 regions, 3 regions in both
horizontal and vertical directions. So called simplified X-ray images are then formed by
averaging horizontally and vertically the intensity of the regions. It is noted that
different kinds of camera operations result in unique trace lines in the X-ray images.

These lines are detected and analyzed by Hough transform. Pans, tilts, and zooms are
detected on the basis of this analysis.

[StainlessS]

Demo of bar location on 2nd image from above post (original video clip is 404), can right clip Save Image As for source image.


Imagesource("CMAD_zpsae30b66c.jpg",end=0)
ConvertToYv24()
BANDS=6 PIX=48 # 6 Bands, each of 48 pixels
W=Width H=Height
n=0 # current_frame not available outside of runtime.
Bingo = RT_YInRangeLocate(n=n,x=W-(PIX/2),y=0,w=1,h=H-(BANDS*PIX),lo=80,hi=255,baffle=24)
(Bingo) ? Overlay(Last.BlankClip(Width=4,height=YIRL_H,color=$FF0000),x=W-(PIX/2)-2,y=YIRL_Y) : NOP
Return ConvertToRGB24


https://s20.postimg.cc/8k3owmev1/RT_YIn_Range_Locate.jpg (https://postimages.cc/)

EDIT: YIRL_H and YIRL_Y are variables set by RT_YInRangeLocate.

EDIT: I'm only interested in spatial alignment horizontal/vertical, not zoom/rotatation (which sounds way too complicated for my little brain).

EDIT:
Here a genuine pair of Row=False [Horizontal] Zebra bar sets for two misaligned frames.
https://s20.postimg.cc/wzl3p2d31/Mis_Aligned.jpg (https://postimages.cc/)
And after alignment (not perfect, yet, below only for primary rapid approximation [4 zebra bands for each frame, probably 1 {YIR} would be better])
https://s20.postimg.cc/7tk5ia98t/Align_Fast.jpg (https://postimages.cc/)
Above, only left alignment seems ok, right align a bit off [needs a few pixels cropped off RHS of bottom frame, and stretched a few pixels].

[StainlessS]

Introspected,

Some bad news, some good news.

Bad News:- After downloading from Github, plugin dont work no more :( (same message about invalid .net assembly).

After about an hour of head scratching where I was testing the DropBox temp plugin, against the new github one,
I was perplexed that the dropbox one worked ok, but Github not, despite CRC's and MD5 etc all being same (both on zip and on dll's).
finally I twigged that the failing dll's had a BLOCK message in File Properties.

This file came from another computer and might be blocked to help protect this computer
Some google links on that:- https://www.google.co.uk/search?q=this+file+came+from+another+computer+and+might+be+blocked&ie=utf-8&oe=utf-8&client=firefox-b&gfe_rd=cr&dcr=0&ei=M7K1WpjVD8rHXtLGk5AN

So I clicked UnBlock and ...

Good News: Everything hunky Dory Again. :)

I downloaded older v0.1.8, and same thing, unblocked the dll's and even that one works OK.

Dont know what it is on my XP machine that is doing the blocking, suspect FireFox.
The First Dropbox v0.1.9 I downloaded on a Windows 10, 10" Tablet/Laptop, and I think using FireFox (but a recent non XP compatible one)
and that one was never blocked and worked just fine.

I Looked at the files in NirSoft Alternate Streams Viewer and they did have alternates, although I'm not sure what part of the alternate streams were responsible for the blockage.

Hope Groucho sees this, its another possible dll problem for AvsMeter to detect.

Anyway, there never was a problem with v0.18, sorry for the bother ol' chap :)

EDIT: Seems that problem is from windows itself (probably installed when I did a WusuOffline Update, hotfixes etc).

Under XP with Installed additional Administrator tools from Server 2003, "ADMINPAK.MSI" (for GpEdit.msc, [Group Policy Editor]).
[EDIT: Actually already installed on basic XP, above "ADMINPAK.MSI" tools not necessary]

gpedit.msc
User Configuration > Administrative Templates > Windows Components > Attachment Manager
Enable: Do not preserve zone information in file attachments

EDIT: Above GpEdit.msc does NOT have an icon in Administrator tools (bit of a secret one), need use Start/Run.
Works fine for me on re-download of v0.1.8.0.

Source here:- https://social.technet.microsoft.com/Forums/office/en-US/147837e4-c0cd-4a81-91f6-a6b9a03ae00d/disable-this-file-comes-from-another-computer?forum=itprovistasecurity

Maybe the Windows 10 downloads were OK as it was on W10 Home Edition (without that group policy active).

[introspected]

Anyway I changed .NET 4.0 to Client Profile for AutoOverlay.dll. This is default .NET package for home users. So the new build is not useless.
As for Windows XP compatibility. The target platform (Windows SDK version) was changed for AvsFilterNet.dll only.

[StainlessS]

For plugin problems where dll's have been Blocked when downloading from the net (or any other files eg pdf etc).
(Problem dll's show in file Properties as BLOCKED, as mentioned two posts earlier).

Streams.exe by SysInternals (Micro$oft) for removing alternatve data streams:-
https://docs.microsoft.com/en-us/sysinternals/downloads/streams
Docs say Vista+, but seems to work just fine on XP.

[introspected]

The plugin is still developing.
Now auto-align feature is much faster.
Color adjust feature uses better color matching algorithm and can be used as stand-alone.
Current version: 0.2.2 (https://github.com/introspected/AutoOverlay/releases/tag/0.2.2)

[Frank62]

Oops, I never read in this part of the forum (found just via google). Can you help, please? Can't at all get your wonderful plugin to work:
https://forum.doom9.org/showthread.php?p=1848904#post1848904

[introspected]

Oops, I never read in this part of the forum (found just via google). Can you help, please? Can't at all get your wonderful plugin to work:
https://forum.doom9.org/showthread.php?p=1848904#post1848904
Did you try to unlock AutoOverlay.dll from file properties dialog as written above?

[StainlessS]

Further to previous posts on blocked dlls (or blocked any type files),
If you need to strip the alternate stream data from any files, and do not have net access to download Streams.exe from SysInternals (Microsoft),
then can just copy files to a FAT32 partition, (as FAT32 dont have alternate streams), and then just copy back and overwrite original files on your
NTFS partition.

EDIT: Link to Streams.Exe @ SysInternals in post #18. (XP+, despite web page saying Vista+).

EDIT: Better streams Alternate Streams Viewer, can delete Zone Identifier Alternate data stream.
https://www.nirsoft.net/utils/alternate_data_streams.html

[Frank62]

Thanks both of you!
Never ever had such kind of problems, but FAT32 sounds simple and doable without spending too much time to Microsoft.

What about your experiments with spatial aligning, @StainlessS?

And: What exactly is returned by RT_YInRangeLocate without debug=true? If I get the purpose right, shouldn't it return two values, one for x diff, and one for y diff?

[StainlessS]

Am still trying to get back to spatial align stuff, got a bucket load of stuff on back burner.
Dont have access to rt-stats docs right now, but think it sets local variables using Prefix arg, default "YIRL_" or something similar,
so sets eg YIRL_X, YIRL_Y, etc.
Actual return value is maybe bool, found, not found.

[poisondeathray]

Are there any other demos/examples besides "Lena" in the 1st post ? Can someone please post some if you have them ?

It looks like it could be useful

I haven't played with this plugin yet, but I'm wondering if another usage scenario might be filling in border edges from stabilization - has anyone had any success with that ?

Or is there some way you could massage the results of the alignment to sharpen chroma problems (such as chroma misalignment, blurred chroma)? Perhaps take U, and V and align according to "Y" but take "colors" or "grey" from the original U,V planes respectively ?

[Frank62]

@StainlessS
Could you post a very short example?

[introspected]

Are there any other demos/examples besides "Lena" in the 1st post ? Can someone please post some if you have them ?
Usually it is used to overlay BD onto low bitrate HDTV or web stream and/or histogram matching. You can find such video at appropriate resources.

I haven't played with this plugin yet, but I'm wondering if another usage scenario might be filling in border edges from stabilization - has anyone had any success with that ?
Stabilization is not the main purpose of the plugin. But it is interesting use-case. I think it needs some enhancements and settings for that to average align params during the scene if necessary.

[StainlessS]

introspected,
maybe you could knock-up some example stuff simply by using crop on some demo avi, user can substitue
their own avi, and get similar results (maybe with some resize, so we are all playing in the same ballpark).

[introspected]

introspected,
maybe you could knock-up some example stuff simply by using crop on some demo avi, user can substitue
their own avi, and get similar results (maybe with some resize, so we are all playing in the same ballpark).

It is exactly that what sample scripts from readme doing. To test script with Lenna from first post take it and source images from repo: https://github.com/introspected/AutoOverlay/tree/master/sample

[StainlessS]

I did actually have this producing some well good stuff some time ago, was quite amazed by results, but on cursory inspection of docs,
is really quite difficult to understand what the plug does or how to steer it.
I still think is one helluva good plug, but hard to understand.

EDIT: The docs on GitHub (or wherever it is) are way more readable than the txt file in zip, suggest look that up.

[introspected]

The docs on GitHub (or wherever it is) are way more readable than the txt file in zip, suggest look that up.
The docs are developed too. Actual readme is included to zip on each release. It is the same as at github project page. Maybe you need offline reader for markdown file type.

[StainlessS]

@StainlessS
Could you post a very short example?

From RT_Stats (next version, but slightly simpler version in RT_Stats AVS Locate directory)


#RT_YInRAngeLocate() Tester

# NOTE, Angle corners show detection.
# req RT_Stats & mt_tools_2

OBJECT_W = 20 # Object to find, size in pixels
OBJECT_H = 40 # Object to find, size in pixels

AREA_X = 0 # Area of frame to search (0,0,0,0=full frame {As Crop})
AREA_Y = 0
AREA_W = 0
AREA_H = 0

LUMA_MIN = 235 # Luma minimum of object to find
LUMA_MAX = 255

# Fake input clip, for testing
a = BlankClip(length=481,width=480,height=480)
b = BlankClip (width=OBJECT_W, height=OBJECT_H, color=$FFFFFF)
a.Animate (0, a.FrameCount () - 1, "Overlay", b, 0, 0, b, 480, 480)

ConvertToYV12() # Any colorspace ok so long as LUMA_MIN/MAX OK (RGB -> Luma-Y)

MarkerColor = $FFFF00 # Detection marker color
MARKER_TYPE = False # False = Corners only marker, True=Full Rect Marker
Msk = Hit_Marker(OBJECT_W+8,OBJECT_H+8,MARKER_TYPE)
Mrk = Msk.BlankClip(Color=MarkerColor)

DEBUG=True
NOTFOUND="NOT FOUND" # Last few frames should show not found, depends upon OBJECT_W/H (as partially disappears off frame).
sep = ", "
ket = "] "
COORDINATES=RT_GetFullPathName("Y_Coordinates.txt")
RT_FileDelete(COORDINATES) # Delete coordinates file

ScriptClip ("""
Bingo=RT_YInRangeLocate(debug=DEBUG,Baffle_W=OBJECT_W,Baffle_H=OBJECT_H,x=AREA_X,y=AREA_Y,w=AREA_W,h=AREA_H,Lo=LUMA_MIN,hi=LUMA_MAX)
(Bingo) ? WriteFile (COORDINATES,"current_frame","KET", "YIRL_X", "sep", "YIRL_Y", "sep", "YIRL_W", "sep", "YIRL_H")
\ : WriteFile (COORDINATES,"current_frame","KET", "NOTFOUND")
(Bingo) ? Overlay(Mrk,x=YIRL_X-4,y=YIRL_Y-4,Mask=Msk,Mode="Blend",output="YV12")
\ : NOP
Return Last
""")

Return Last

Function Hit_Marker(int W, Int H, Bool "Hit", Int "BW", Int "BH",Int "PW",Int "PH",Bool "YV12",Bool "Mod2") {
Hit=Default(hit,False) BW=Max(Default(BW,W/8),1) BH=Max(Default(BH,H/8),1)
BW2=(W>4)?BW*2:BW BH2=(H>4)?BH*2:BH
PW=Max(Default(PW,BW/8),1) PH=Max(Default(PH,BH/8),1) is26 = VersionNumber>=2.6
YV12=(!is26) ? True : Default(YV12,False) Mod2=Default(Mod2,True) Mod = (YV12||Mod2) ? 2 : 1
CanvasW=(W+Mod-1)/Mod*Mod CanvasH=(H+Mod-1)/Mod*Mod
Rpn= RT_String("x %d < x %d >= | y %d < | y %d >= | x %d < x %d >= | & y %d < y %d >= | & ",BW,W-BW,BH,H-BH,BW2,W-BW2,BH2,H-BH2)
Rpn= (Hit) ? Rpn + RT_String("x %d < x %d >= | y %d < | y %d >= | | ",PW,W-PW,PH,H-PH) : Rpn
Rpn= Rpn + RT_String("x %d < y %d < & & 255 0 ?",W,H)
Blankclip(width=CanvasW,height=CanvasH,Length=1,pixel_type=YV12?"YV12":"Y8").Killaudio
return mt_lutspa(relative = false,yExpr=Rpn, chroma = "-128")
}


And what it looks like (the knobbly thing outside of block is detected position).
https://s33.postimg.cc/ebwpz5gwf/YIn_Range_Locate_Test.jpg (https://postimages.org/)photoupload (https://postimages.org/)

EDIT: There is also a demo of RGB version in same directory.

[Frank62]

Thanks a lot!! Will try this tomorrow, looks very interesting.

[introspected]

0.2.5 released: https://github.com/introspected/AutoOverlay/releases/tag/0.2.5
- Enhanced filter arguments validation
- HDR support: OverlayEngine, OverlayRender
- BicubicResize by default instead of BilinearResize
- All filters described in readme

[color]

I have tried to get this to work, I use latest Avisynth+ and have the latest .Net installed. I get error:

Unable to load plugin, make sure it is a valid .Net assembly.


EDIT, Im using:

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AvsFilterNet.dll")
LoadNetPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AutoOverlay_netautoload.dll")

and tried before:

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AvsFilterNet.dll")
LoadNetPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AutoOverlayNative.dll")

same error

[introspected]

I have tried to get this to work, I use latest Avisynth+ and have the latest .Net installed. I get error:

Unable to load plugin, make sure it is a valid .Net assembly.

Try to unlock the file.

[color]

Try to unlock the file.

How do I unlock the file? Also I can not see the image that you posted.

[introspected]

https://image.ibb.co/fmybcf/1425-File-Properties-Unblock.png (https://imgbb.com/)

[color]

Thank you so much. Now It works perfect! :D I love this plugin, It will save so much time for me. :D

[StainlessS]

I'm back to trying to figure this plugin out again.
Just thought that this pdf might be handy for offline viewing:- http://www.mediafire.com/file/rnke2ndn9l77mrb/Introspected-AutoOverlay-AviSynth_plugin.pdf/file
Its just a PDF printout of Github page (prettier than the md text file):- https://github.com/introspected/AutoOverlay

using firefox addon, Save PDF (save current page as PDF), pdf is about 1.5MB, you could just print your own out if you got Save PDF.
[click on the Portrait image + landscape image -> script output line to show images before printout]

This just showing Editor thing

# Current dir ".\x86\" contains all 3 x86 dll's
LoadPlugin(".\x86\AvsFilterNet.dll")
LoadNetPlugin(".\x86\AutoOverlay_netautoload.dll") # Presumably also loads ".\x86\AutoOverlayNative.dll"
#return version

portrait=ImageSource(".\Lenna.portrait.jpg").ConvertToYV24() # Images saved from GitHub, "Save Image As".
landscape=ImageSource(".\Lenna.landscape.jpg").ConvertToYV24()
#return Portrait
#return Landscape

EDITOR=True

configs=OverlayConfig() # just using defaults
Engine=OverlayEngine(portrait, landscape, mode="default",configs=configs,Editor=EDITOR)
Engine.OverlayRender(portrait, landscape, colorAdjust=2, mode=2, gradient=20, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize")

return last


nuther test

# Current dir ".\x86\" contains all 3 x86 dll's
LoadPlugin(".\x86\AvsFilterNet.dll")
LoadNetPlugin(".\x86\AutoOverlay_netautoload.dll") # Presumably also loads ".\x86\AutoOverlayNative.dll"

A=Colorbars(Pixel_Type="YV24").Addborders(8,8,8,8,$FF0000).KillAudio
B=A
A=A.crop(0,20,0,0) # crop some off top
B=B.crop(0,0,-20,-0) # crop some off right
#Return A
#Return B

EDITOR=False

configs=OverlayConfig() # just using defaults
Engine=OverlayEngine(A, B, mode="default",configs=configs,Editor=EDITOR)
#Engine.OverlayRender(A, B, colorAdjust=2, mode=2, gradient=20, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize")
Engine.OverlayRender(A, B, mode=3, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize") # Also try with mode=4

return last

Result of nuther test (colorbars with red border, 1st image cropped top red border + a bit more, 2nd image cropped right red border + a bit more), results below.
[top rhs corner missing, but then missing from both input clips too]
https://i.postimg.cc/JtYM0Knx/Test2.jpg (https://postimg.cc/JtYM0Knx)

EDIT: with Engine.OverlayRender(mode=4) # see top RHS corner
https://i.postimg.cc/zyKk3tJR/Test2b.jpg (https://postimg.cc/zyKk3tJR)

[videophile]

Hi, I am using Avisynth 2.60 (non MT), I really would like to use this plugin to create an HDR video from two captures which are not perfectly aligned, but I get this message : "Avisynth open failure: Plugin plugins/AutoOverlayNative.dll is not an Avisynth 2.6 or 2.5 plugin".
What I am doing wrong?

[introspected]

v0.3 released: https://github.com/introspected/AutoOverlay/releases/tag/0.3.0

1. OverlayEngine: presize and resize instead of upsize and downsize.
2. OverlayEngine: new engine mode PROCESSED.
3. OverlayEngine: pan&scan (unstable sources) support.
4. SIMD Library using and increased performance.
5. ColorAdjust: new interpolation algorithms via Math.NET Numerics.
6. AviSynth API v8
7. OverlayEditor: new features.
8. OverlayRender: new features.
9. ComplexityOverlay: new filter.

There are many different fixes and enhancements.

[StainlessS]

Many thanx Intro, glad to see that you have not forgotten us, https://www.cosgan.de/images/smilie/froehlich/a0354.gif

[real.finder]

v0.3 released: https://github.com/introspected/AutoOverlay/releases/tag/0.3.0

1. OverlayEngine: presize and resize instead of upsize and downsize.
2. OverlayEngine: new engine mode PROCESSED.
3. OverlayEngine: pan&scan (unstable sources) support.
4. SIMD Library using and increased performance.
5. ColorAdjust: new interpolation algorithms via Math.NET Numerics.
6. AviSynth API v8
7. OverlayEditor: new features.
8. OverlayRender: new features.
9. ComplexityOverlay: new filter.

There are many different fixes and enhancements.

is this fix https://github.com/Asd-g/AutoOverlay/commit/abe5d2b015ddaa10829d858615e429add89b51c4 included?

[introspected]

is this fix https://github.com/Asd-g/AutoOverlay/commit/abe5d2b015ddaa10829d858615e429add89b51c4 included?

Yes, thanks.

[introspected]

v0.3.1 is out: https://github.com/introspected/AutoOverlay/releases/tag/0.3.1
1. Fix first frame x264 encoding.
2. ColorAdjust: HDR extrapolation fix, dynamicNoise parameter.
3. OverlayRender: extrapolation parameter.

[StainlessS]

Oooh, lovely :)

[AdamCarter]

I have a Very decent tv rip of an anime with english audio
And the same anime in a higher quality professionally authored japanese dvd.
They don’t Line up exactly so remuxing with english audio without Linear editing is not possible.

Would this program be able to use the lower quality source as a reference and overlay only the frames that the crappy sources has from the higher quality source?

Perhaps maybe deconstructing both videos into frames and then matching frames up. and provide a log of missing frames that couldn’t be matched to the low quality source, so the source frames could be used?

Thanks for The info.

[Frank62]

A "Holy Grail" also for my everyday-problems since more than 10 years, but no automatic solution nowhere. Also this filter can't help much with this.

The only automation possible I found was:
-log reliable scene changes of both sources
-write a semi-automatized assistent that lets you correct and align both logs interactively and then exports an Avisynth-script that handles the soundtrack with "timestretches" and "dissolves" to finally fit.

Unfortunately my tool is that buggy, quick, and dirty, that there is no way to publish. At a THAT low stage of development that nobody apart from me will be able to use it, and absolutely no time to improve, as so many things... And still a lot of work to do by hand each time.

If somebody out there has a perfect solution for this particular problem (full automization of soundtrack alignment of two slightly different sources)...? I would even pay a nice amount of money for this.

[AdamCarter]

Frank62

I wonder if this could work for us. Since the author of this script hasn’t responded.
Let me know what you think?

https://github.com/edwardanderson/ci2v