AutoOverlay: auto-alignment and color matching

[introspected]

Github: https://github.com/introspected/AutoOverlay
Releases: https://github.com/introspected/AutoOverlay/releases

AutoOverlay is an AviSynth plugin that provides tools for auto-alignment and color matching, enabling the seamless combination of different video clips in various ways. It automatically calculates optimal alignment parameters such as position, scale, and rotation using frame-by-frame analysis, while adjusting colors to ensure consistency across sources.

https://preview.ibb.co/eNbPO9/script000188.png (https://ibb.co/mAYSbU)

[Aktan]

Sounds interesting! Good job =).

[lansing]

Is this like the auto-align feature in photoshop?

[introspected]

Is this like the auto-align feature in photoshop?
In general, yes. But plugin supports only basic transofrmations such as resizing and rotation to align two clips.

[spoRv]

Thanks again for your great plugin!!!

[StainlessS]

LoadPlugin(".\AvsFilterNet.dll")
LoadNetPlugin(".\AutoOverlay_netautoload.dll")
return blankclip


Avisynth Open Failure
Unable to load plugin, make sure it is a valid .Net assembly

What is the minimum required OS for this plug ? (Above on XP32).

Thanx.

EDIT: Installed
.Net Framework 4 Client Profile
.Net Framework 4 Extended

[StainlessS]

Failing an answer from Introspected (the dev), has anybody had it working on XP ?
(I'm not sure if my newish setup is screwed up,
my old PowerDVD5 [or maybe 6] takes about 60 seconds [or more] to start up, but that should not need .NET at all,
something aint right somewhere.)

[introspected]

Full .NET 4.0 is required not client profile (forgot to change it). Try reinstall extended version, not just update.
XP is supported in theory.
Maybe there are some issues with target .NET settings in the project.
I have no time to test it now.

[StainlessS]

Thanx for your answer.
I did not pre-setup .NET 4 prior to running WususOffline update, presumed that it would do the full install.
(It certainly took a helluva a lot of time updating .Net 4 hotfixes/updates, dont know what it could have been
doing if it did not install the full thing).
Anyways, I'll re-do an install, and re-wusus again, thanks.

EDIT: No idea what the client profile etc does, I know nothing of .Net.

EDIT: Dont know if you saw this post in Usage, I'm doing a script to Spatially Align clips.
https://forum.doom9.org/showthread.php?p=1836806#post1836806

[introspected]

Try this build: https://www.dropbox.com/s/3ktixdpyj6ff9na/AutoOverlay_0.1.9.zip?dl=0
If OK then I will release it on github.

[StainlessS]

Well I've downloaded and loads plugins OK, not really had time to test it out though, will do soon.
(Thought I'de post as I saw you on-line).

Thanx muchly :)

EDIT: Above, without reinstall of .Net.

EDIT: Seems to work just fine, although not much idea what I'm doing just yet.
EDIT: I'll still work on Spatial Align script function as I want to be able to hack it for differing purposes.

[StainlessS]

Introspected,

that seems to be one hellova good plugin that you've constructed, nice job :)

Small typo in the examples

statFile=”c:\test\Overlay.stat”


Uses slanted double quotes rather than ordinary ones.

Thanx again, its bloody great !!!

[introspected]

StainlessS,
thanks for testing. Released on github.
Quotes fixed.

What do you mean about spatial align algorithm? The function from your link is not clear for me.

[StainlessS]

I meant that I'm experimenting on using the Zebra Bands (together with RT_Stats RT_LumaCorrelation) for matching of clips [EDIT: frames],
Zebra(Row=False) for Horizontal alignement, and True for Vertical.

The Zebra function was implemented to detect this JohnMeyer clip problem, "Bad 1950s Kinescope - Hopeless? ":- https://forum.doom9.org/showthread.php?t=167591&page=2

Where it detected a bright band travelling vertically at each frame.
EDIT: RT_YInRangeLocate() was used to locate the bright bar within the YIR Zebra Band.

The Equid function is just a convienent viewer for comparing frames and bands (might be of some use to somebody else).

Its quicker to resize/measure a thin zebra band than the entire frame.

This is one of the JohnMeyer examples where the Zebra band is evident
https://s12.postimg.cc/5l3g3zmj1/YIRL_zps21bcdbdc.jpg (https://postimg.cc/image/3tah93361/)

Or with CMAD (Centered Moving Averages Diameter=13)
https://s24.postimg.cc/wuwda1vkl/CMAD_zpsae30b66c.jpg (https://postimg.cc/image/qtyocz8y9/)

I got the idea of the Zebroid thing (daft experimental) from
[B]Automatic color enhancement and scene change detection of digital video, Jyrki Korpi-Anttila.

Logical Page 36(internal page number), physical page 39(pdf page).
Chen and Panchanathan [95] proposed a region based X-ray image method to detect
tilts, pans and zooming. Each frame is first partitioned into 9 regions, 3 regions in both
horizontal and vertical directions. So called simplified X-ray images are then formed by
averaging horizontally and vertically the intensity of the regions. It is noted that
different kinds of camera operations result in unique trace lines in the X-ray images.

These lines are detected and analyzed by Hough transform. Pans, tilts, and zooms are
detected on the basis of this analysis.

[StainlessS]

Demo of bar location on 2nd image from above post (original video clip is 404), can right clip Save Image As for source image.


Imagesource("CMAD_zpsae30b66c.jpg",end=0)
ConvertToYv24()
BANDS=6 PIX=48 # 6 Bands, each of 48 pixels
W=Width H=Height
n=0 # current_frame not available outside of runtime.
Bingo = RT_YInRangeLocate(n=n,x=W-(PIX/2),y=0,w=1,h=H-(BANDS*PIX),lo=80,hi=255,baffle=24)
(Bingo) ? Overlay(Last.BlankClip(Width=4,height=YIRL_H,color=$FF0000),x=W-(PIX/2)-2,y=YIRL_Y) : NOP
Return ConvertToRGB24


https://s20.postimg.cc/8k3owmev1/RT_YIn_Range_Locate.jpg (https://postimages.cc/)

EDIT: YIRL_H and YIRL_Y are variables set by RT_YInRangeLocate.

EDIT: I'm only interested in spatial alignment horizontal/vertical, not zoom/rotatation (which sounds way too complicated for my little brain).

EDIT:
Here a genuine pair of Row=False [Horizontal] Zebra bar sets for two misaligned frames.
https://s20.postimg.cc/wzl3p2d31/Mis_Aligned.jpg (https://postimages.cc/)
And after alignment (not perfect, yet, below only for primary rapid approximation [4 zebra bands for each frame, probably 1 {YIR} would be better])
https://s20.postimg.cc/7tk5ia98t/Align_Fast.jpg (https://postimages.cc/)
Above, only left alignment seems ok, right align a bit off [needs a few pixels cropped off RHS of bottom frame, and stretched a few pixels].

[StainlessS]

Introspected,

Some bad news, some good news.

Bad News:- After downloading from Github, plugin dont work no more :( (same message about invalid .net assembly).

After about an hour of head scratching where I was testing the DropBox temp plugin, against the new github one,
I was perplexed that the dropbox one worked ok, but Github not, despite CRC's and MD5 etc all being same (both on zip and on dll's).
finally I twigged that the failing dll's had a BLOCK message in File Properties.

This file came from another computer and might be blocked to help protect this computer
Some google links on that:- https://www.google.co.uk/search?q=this+file+came+from+another+computer+and+might+be+blocked&ie=utf-8&oe=utf-8&client=firefox-b&gfe_rd=cr&dcr=0&ei=M7K1WpjVD8rHXtLGk5AN

So I clicked UnBlock and ...

Good News: Everything hunky Dory Again. :)

I downloaded older v0.1.8, and same thing, unblocked the dll's and even that one works OK.

Dont know what it is on my XP machine that is doing the blocking, suspect FireFox.
The First Dropbox v0.1.9 I downloaded on a Windows 10, 10" Tablet/Laptop, and I think using FireFox (but a recent non XP compatible one)
and that one was never blocked and worked just fine.

I Looked at the files in NirSoft Alternate Streams Viewer and they did have alternates, although I'm not sure what part of the alternate streams were responsible for the blockage.

Hope Groucho sees this, its another possible dll problem for AvsMeter to detect.

Anyway, there never was a problem with v0.18, sorry for the bother ol' chap :)

EDIT: Seems that problem is from windows itself (probably installed when I did a WusuOffline Update, hotfixes etc).

Under XP with Installed additional Administrator tools from Server 2003, "ADMINPAK.MSI" (for GpEdit.msc, [Group Policy Editor]).
[EDIT: Actually already installed on basic XP, above "ADMINPAK.MSI" tools not necessary]

gpedit.msc
User Configuration > Administrative Templates > Windows Components > Attachment Manager
Enable: Do not preserve zone information in file attachments

EDIT: Above GpEdit.msc does NOT have an icon in Administrator tools (bit of a secret one), need use Start/Run.
Works fine for me on re-download of v0.1.8.0.

Source here:- https://social.technet.microsoft.com/Forums/office/en-US/147837e4-c0cd-4a81-91f6-a6b9a03ae00d/disable-this-file-comes-from-another-computer?forum=itprovistasecurity

Maybe the Windows 10 downloads were OK as it was on W10 Home Edition (without that group policy active).

[introspected]

Anyway I changed .NET 4.0 to Client Profile for AutoOverlay.dll. This is default .NET package for home users. So the new build is not useless.
As for Windows XP compatibility. The target platform (Windows SDK version) was changed for AvsFilterNet.dll only.

[StainlessS]

For plugin problems where dll's have been Blocked when downloading from the net (or any other files eg pdf etc).
(Problem dll's show in file Properties as BLOCKED, as mentioned two posts earlier).

Streams.exe by SysInternals (Micro$oft) for removing alternatve data streams:-
https://docs.microsoft.com/en-us/sysinternals/downloads/streams
Docs say Vista+, but seems to work just fine on XP.

[introspected]

The plugin is still developing.
Now auto-align feature is much faster.
Color adjust feature uses better color matching algorithm and can be used as stand-alone.
Current version: 0.2.2 (https://github.com/introspected/AutoOverlay/releases/tag/0.2.2)

[Frank62]

Oops, I never read in this part of the forum (found just via google). Can you help, please? Can't at all get your wonderful plugin to work:
https://forum.doom9.org/showthread.php?p=1848904#post1848904

[introspected]

Oops, I never read in this part of the forum (found just via google). Can you help, please? Can't at all get your wonderful plugin to work:
https://forum.doom9.org/showthread.php?p=1848904#post1848904
Did you try to unlock AutoOverlay.dll from file properties dialog as written above?

[StainlessS]

Further to previous posts on blocked dlls (or blocked any type files),
If you need to strip the alternate stream data from any files, and do not have net access to download Streams.exe from SysInternals (Microsoft),
then can just copy files to a FAT32 partition, (as FAT32 dont have alternate streams), and then just copy back and overwrite original files on your
NTFS partition.

EDIT: Link to Streams.Exe @ SysInternals in post #18. (XP+, despite web page saying Vista+).

EDIT: Better streams Alternate Streams Viewer, can delete Zone Identifier Alternate data stream.
https://www.nirsoft.net/utils/alternate_data_streams.html

[Frank62]

Thanks both of you!
Never ever had such kind of problems, but FAT32 sounds simple and doable without spending too much time to Microsoft.

What about your experiments with spatial aligning, @StainlessS?

And: What exactly is returned by RT_YInRangeLocate without debug=true? If I get the purpose right, shouldn't it return two values, one for x diff, and one for y diff?

[StainlessS]

Am still trying to get back to spatial align stuff, got a bucket load of stuff on back burner.
Dont have access to rt-stats docs right now, but think it sets local variables using Prefix arg, default "YIRL_" or something similar,
so sets eg YIRL_X, YIRL_Y, etc.
Actual return value is maybe bool, found, not found.

[poisondeathray]

Are there any other demos/examples besides "Lena" in the 1st post ? Can someone please post some if you have them ?

It looks like it could be useful

I haven't played with this plugin yet, but I'm wondering if another usage scenario might be filling in border edges from stabilization - has anyone had any success with that ?

Or is there some way you could massage the results of the alignment to sharpen chroma problems (such as chroma misalignment, blurred chroma)? Perhaps take U, and V and align according to "Y" but take "colors" or "grey" from the original U,V planes respectively ?

[Frank62]

@StainlessS
Could you post a very short example?

[introspected]

Are there any other demos/examples besides "Lena" in the 1st post ? Can someone please post some if you have them ?
Usually it is used to overlay BD onto low bitrate HDTV or web stream and/or histogram matching. You can find such video at appropriate resources.

I haven't played with this plugin yet, but I'm wondering if another usage scenario might be filling in border edges from stabilization - has anyone had any success with that ?
Stabilization is not the main purpose of the plugin. But it is interesting use-case. I think it needs some enhancements and settings for that to average align params during the scene if necessary.

[StainlessS]

introspected,
maybe you could knock-up some example stuff simply by using crop on some demo avi, user can substitue
their own avi, and get similar results (maybe with some resize, so we are all playing in the same ballpark).

[introspected]

introspected,
maybe you could knock-up some example stuff simply by using crop on some demo avi, user can substitue
their own avi, and get similar results (maybe with some resize, so we are all playing in the same ballpark).

It is exactly that what sample scripts from readme doing. To test script with Lenna from first post take it and source images from repo: https://github.com/introspected/AutoOverlay/tree/master/sample

[StainlessS]

I did actually have this producing some well good stuff some time ago, was quite amazed by results, but on cursory inspection of docs,
is really quite difficult to understand what the plug does or how to steer it.
I still think is one helluva good plug, but hard to understand.

EDIT: The docs on GitHub (or wherever it is) are way more readable than the txt file in zip, suggest look that up.

[introspected]

The docs on GitHub (or wherever it is) are way more readable than the txt file in zip, suggest look that up.
The docs are developed too. Actual readme is included to zip on each release. It is the same as at github project page. Maybe you need offline reader for markdown file type.

[StainlessS]

@StainlessS
Could you post a very short example?

From RT_Stats (next version, but slightly simpler version in RT_Stats AVS Locate directory)


#RT_YInRAngeLocate() Tester

# NOTE, Angle corners show detection.
# req RT_Stats & mt_tools_2

OBJECT_W = 20 # Object to find, size in pixels
OBJECT_H = 40 # Object to find, size in pixels

AREA_X = 0 # Area of frame to search (0,0,0,0=full frame {As Crop})
AREA_Y = 0
AREA_W = 0
AREA_H = 0

LUMA_MIN = 235 # Luma minimum of object to find
LUMA_MAX = 255

# Fake input clip, for testing
a = BlankClip(length=481,width=480,height=480)
b = BlankClip (width=OBJECT_W, height=OBJECT_H, color=$FFFFFF)
a.Animate (0, a.FrameCount () - 1, "Overlay", b, 0, 0, b, 480, 480)

ConvertToYV12() # Any colorspace ok so long as LUMA_MIN/MAX OK (RGB -> Luma-Y)

MarkerColor = $FFFF00 # Detection marker color
MARKER_TYPE = False # False = Corners only marker, True=Full Rect Marker
Msk = Hit_Marker(OBJECT_W+8,OBJECT_H+8,MARKER_TYPE)
Mrk = Msk.BlankClip(Color=MarkerColor)

DEBUG=True
NOTFOUND="NOT FOUND" # Last few frames should show not found, depends upon OBJECT_W/H (as partially disappears off frame).
sep = ", "
ket = "] "
COORDINATES=RT_GetFullPathName("Y_Coordinates.txt")
RT_FileDelete(COORDINATES) # Delete coordinates file

ScriptClip ("""
Bingo=RT_YInRangeLocate(debug=DEBUG,Baffle_W=OBJECT_W,Baffle_H=OBJECT_H,x=AREA_X,y=AREA_Y,w=AREA_W,h=AREA_H,Lo=LUMA_MIN,hi=LUMA_MAX)
(Bingo) ? WriteFile (COORDINATES,"current_frame","KET", "YIRL_X", "sep", "YIRL_Y", "sep", "YIRL_W", "sep", "YIRL_H")
\ : WriteFile (COORDINATES,"current_frame","KET", "NOTFOUND")
(Bingo) ? Overlay(Mrk,x=YIRL_X-4,y=YIRL_Y-4,Mask=Msk,Mode="Blend",output="YV12")
\ : NOP
Return Last
""")

Return Last

Function Hit_Marker(int W, Int H, Bool "Hit", Int "BW", Int "BH",Int "PW",Int "PH",Bool "YV12",Bool "Mod2") {
Hit=Default(hit,False) BW=Max(Default(BW,W/8),1) BH=Max(Default(BH,H/8),1)
BW2=(W>4)?BW*2:BW BH2=(H>4)?BH*2:BH
PW=Max(Default(PW,BW/8),1) PH=Max(Default(PH,BH/8),1) is26 = VersionNumber>=2.6
YV12=(!is26) ? True : Default(YV12,False) Mod2=Default(Mod2,True) Mod = (YV12||Mod2) ? 2 : 1
CanvasW=(W+Mod-1)/Mod*Mod CanvasH=(H+Mod-1)/Mod*Mod
Rpn= RT_String("x %d < x %d >= | y %d < | y %d >= | x %d < x %d >= | & y %d < y %d >= | & ",BW,W-BW,BH,H-BH,BW2,W-BW2,BH2,H-BH2)
Rpn= (Hit) ? Rpn + RT_String("x %d < x %d >= | y %d < | y %d >= | | ",PW,W-PW,PH,H-PH) : Rpn
Rpn= Rpn + RT_String("x %d < y %d < & & 255 0 ?",W,H)
Blankclip(width=CanvasW,height=CanvasH,Length=1,pixel_type=YV12?"YV12":"Y8").Killaudio
return mt_lutspa(relative = false,yExpr=Rpn, chroma = "-128")
}


And what it looks like (the knobbly thing outside of block is detected position).
https://s33.postimg.cc/ebwpz5gwf/YIn_Range_Locate_Test.jpg (https://postimages.org/)photoupload (https://postimages.org/)

EDIT: There is also a demo of RGB version in same directory.

[Frank62]

Thanks a lot!! Will try this tomorrow, looks very interesting.

[introspected]

0.2.5 released: https://github.com/introspected/AutoOverlay/releases/tag/0.2.5
- Enhanced filter arguments validation
- HDR support: OverlayEngine, OverlayRender
- BicubicResize by default instead of BilinearResize
- All filters described in readme

[color]

I have tried to get this to work, I use latest Avisynth+ and have the latest .Net installed. I get error:

Unable to load plugin, make sure it is a valid .Net assembly.


EDIT, Im using:

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AvsFilterNet.dll")
LoadNetPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AutoOverlay_netautoload.dll")

and tried before:

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AvsFilterNet.dll")
LoadNetPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AutoOverlayNative.dll")

same error

[introspected]

I have tried to get this to work, I use latest Avisynth+ and have the latest .Net installed. I get error:

Unable to load plugin, make sure it is a valid .Net assembly.

Try to unlock the file.

[color]

Try to unlock the file.

How do I unlock the file? Also I can not see the image that you posted.

[introspected]

https://image.ibb.co/fmybcf/1425-File-Properties-Unblock.png (https://imgbb.com/)

[color]

Thank you so much. Now It works perfect! :D I love this plugin, It will save so much time for me. :D

[StainlessS]

I'm back to trying to figure this plugin out again.
Just thought that this pdf might be handy for offline viewing:- http://www.mediafire.com/file/rnke2ndn9l77mrb/Introspected-AutoOverlay-AviSynth_plugin.pdf/file
Its just a PDF printout of Github page (prettier than the md text file):- https://github.com/introspected/AutoOverlay

using firefox addon, Save PDF (save current page as PDF), pdf is about 1.5MB, you could just print your own out if you got Save PDF.
[click on the Portrait image + landscape image -> script output line to show images before printout]

This just showing Editor thing

# Current dir ".\x86\" contains all 3 x86 dll's
LoadPlugin(".\x86\AvsFilterNet.dll")
LoadNetPlugin(".\x86\AutoOverlay_netautoload.dll") # Presumably also loads ".\x86\AutoOverlayNative.dll"
#return version

portrait=ImageSource(".\Lenna.portrait.jpg").ConvertToYV24() # Images saved from GitHub, "Save Image As".
landscape=ImageSource(".\Lenna.landscape.jpg").ConvertToYV24()
#return Portrait
#return Landscape

EDITOR=True

configs=OverlayConfig() # just using defaults
Engine=OverlayEngine(portrait, landscape, mode="default",configs=configs,Editor=EDITOR)
Engine.OverlayRender(portrait, landscape, colorAdjust=2, mode=2, gradient=20, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize")

return last


nuther test

# Current dir ".\x86\" contains all 3 x86 dll's
LoadPlugin(".\x86\AvsFilterNet.dll")
LoadNetPlugin(".\x86\AutoOverlay_netautoload.dll") # Presumably also loads ".\x86\AutoOverlayNative.dll"

A=Colorbars(Pixel_Type="YV24").Addborders(8,8,8,8,$FF0000).KillAudio
B=A
A=A.crop(0,20,0,0) # crop some off top
B=B.crop(0,0,-20,-0) # crop some off right
#Return A
#Return B

EDITOR=False

configs=OverlayConfig() # just using defaults
Engine=OverlayEngine(A, B, mode="default",configs=configs,Editor=EDITOR)
#Engine.OverlayRender(A, B, colorAdjust=2, mode=2, gradient=20, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize")
Engine.OverlayRender(A, B, mode=3, width=768, height=768, upsize="Spline64Resize", downsize="Spline64Resize") # Also try with mode=4

return last

Result of nuther test (colorbars with red border, 1st image cropped top red border + a bit more, 2nd image cropped right red border + a bit more), results below.
[top rhs corner missing, but then missing from both input clips too]
https://i.postimg.cc/JtYM0Knx/Test2.jpg (https://postimg.cc/JtYM0Knx)

EDIT: with Engine.OverlayRender(mode=4) # see top RHS corner
https://i.postimg.cc/zyKk3tJR/Test2b.jpg (https://postimg.cc/zyKk3tJR)

[videophile]

Hi, I am using Avisynth 2.60 (non MT), I really would like to use this plugin to create an HDR video from two captures which are not perfectly aligned, but I get this message : "Avisynth open failure: Plugin plugins/AutoOverlayNative.dll is not an Avisynth 2.6 or 2.5 plugin".
What I am doing wrong?

[introspected]

v0.3 released: https://github.com/introspected/AutoOverlay/releases/tag/0.3.0

1. OverlayEngine: presize and resize instead of upsize and downsize.
2. OverlayEngine: new engine mode PROCESSED.
3. OverlayEngine: pan&scan (unstable sources) support.
4. SIMD Library using and increased performance.
5. ColorAdjust: new interpolation algorithms via Math.NET Numerics.
6. AviSynth API v8
7. OverlayEditor: new features.
8. OverlayRender: new features.
9. ComplexityOverlay: new filter.

There are many different fixes and enhancements.

[StainlessS]

Many thanx Intro, glad to see that you have not forgotten us, https://www.cosgan.de/images/smilie/froehlich/a0354.gif

[real.finder]

v0.3 released: https://github.com/introspected/AutoOverlay/releases/tag/0.3.0

1. OverlayEngine: presize and resize instead of upsize and downsize.
2. OverlayEngine: new engine mode PROCESSED.
3. OverlayEngine: pan&scan (unstable sources) support.
4. SIMD Library using and increased performance.
5. ColorAdjust: new interpolation algorithms via Math.NET Numerics.
6. AviSynth API v8
7. OverlayEditor: new features.
8. OverlayRender: new features.
9. ComplexityOverlay: new filter.

There are many different fixes and enhancements.

is this fix https://github.com/Asd-g/AutoOverlay/commit/abe5d2b015ddaa10829d858615e429add89b51c4 included?

[introspected]

is this fix https://github.com/Asd-g/AutoOverlay/commit/abe5d2b015ddaa10829d858615e429add89b51c4 included?

Yes, thanks.

[introspected]

v0.3.1 is out: https://github.com/introspected/AutoOverlay/releases/tag/0.3.1
1. Fix first frame x264 encoding.
2. ColorAdjust: HDR extrapolation fix, dynamicNoise parameter.
3. OverlayRender: extrapolation parameter.

[StainlessS]

Oooh, lovely :)

[AdamCarter]

I have a Very decent tv rip of an anime with english audio
And the same anime in a higher quality professionally authored japanese dvd.
They don’t Line up exactly so remuxing with english audio without Linear editing is not possible.

Would this program be able to use the lower quality source as a reference and overlay only the frames that the crappy sources has from the higher quality source?

Perhaps maybe deconstructing both videos into frames and then matching frames up. and provide a log of missing frames that couldn’t be matched to the low quality source, so the source frames could be used?

Thanks for The info.

[Frank62]

A "Holy Grail" also for my everyday-problems since more than 10 years, but no automatic solution nowhere. Also this filter can't help much with this.

The only automation possible I found was:
-log reliable scene changes of both sources
-write a semi-automatized assistent that lets you correct and align both logs interactively and then exports an Avisynth-script that handles the soundtrack with "timestretches" and "dissolves" to finally fit.

Unfortunately my tool is that buggy, quick, and dirty, that there is no way to publish. At a THAT low stage of development that nobody apart from me will be able to use it, and absolutely no time to improve, as so many things... And still a lot of work to do by hand each time.

If somebody out there has a perfect solution for this particular problem (full automization of soundtrack alignment of two slightly different sources)...? I would even pay a nice amount of money for this.

[AdamCarter]

Frank62

I wonder if this could work for us. Since the author of this script hasn’t responded.
Let me know what you think?

https://github.com/edwardanderson/ci2v

[Frank62]

I am not sure. SSIM is mostly used for checking quality differences of one and the same picture. But when I find the time (can last a while...) I will test it, thanks for the idea.

[Goggen240]

FFMatch does 100% exactly what you want!
http://fsinapsi.altervista.org/code/ffmatch/index.html

It's kinda tricky to use, since the user manual is only available in Italian.
EDIT: And I was going to write it hadn't been updated since 2011, but it was updated last month! Happy days! But that means my guide may be out-of-date. Also, no screenshots, because I have to be at work in an hour...

To use it, you need both files encoded in XVID (some varieties of h.264 and x.264 *may* work), and both soundtracks as separate .MP3 files.
Top left, select movies 1 and 2 (doesn't matter which is which)
Top middle: Turn on supervisor mode (pencil-and-paper button)
Top middle: start (gear button)
This will run for a real long while, first it analyzes both movies, and then it starts matching them. It's cool to watch once it gets to that point.
Then, once it "finishes", you'll have matched maybe 80% of the frames to each other; to actually finish, you go in manually...
Bottom right you have video manuevering controls; for the sections that don't match, you line up two matching frames for both clips, and press the green arrow-down button.
Top right text is info of matching and unmatching ranges; green matches in frame count and content, red ranges do not match.
The goal is to make all the red ranges green, which means weeding out the missing frames:
A "red" range might say
[040684-040856] -/-> ??? {173-172}
Which means that movie 1 frames 40684->40856 is 173 frames, but the same region in movie 2 is 173 frames, and doesn't fit.
How to match manually:
Bottom right:
Left number is frame movie 1, right number is frame movie 2, and the frame-forward/back apply to both movies.
The drop-down menu is key; it lets you pick matching frames for a frame range: [Number is first frame in a matching range, Number] is last frame, and (Number) is first frame *after* it loses matching.
In this case, I select (40684), the first unmatched movie 1 frame in that region, and it automatically picks movie 2 frame 21755, the first unmatched frame in movie 2's region.
I then arrow right a little (both movies) until I find a frame that is easy to match; there's a big-ol' hair on movie 1 frame 40694, and it matches one frame "early" to movie 2 frame 21764.
(Between movie 1 frames 40684 and 40694, there's one frame missing compared to movie 2.)
I press green arrow down button (bottom right), linking 40694 to 21764, and top right text now reads:
[040684-040693] -/-> ??? {10-9}
[040694-040694] ---> [021764-021764] {000001} (diff=-18930)
[040695-040856] -/-> ??? {162-162}
First range is misaligned, second range is 1 matched frame, third range is aligned, but has not been matched yet, which you do by pressing the start gear button again. But first:
To get rid of the unmatched region entirely, I arrow left until I am at 40685-21755, and press the green arrow button. The range now reads:
[040685-040685] ---> [021755-021755] {000001} (diff=-18930)
[040686-040693] -/-> ??? {8-8}
[040694-040694] ---> [021764-021764] {000001} (diff=-18930)
[040695-040856] -/-> ??? {162-162}
The one extra frame is now "gone", but listed top right here:
1st movie frame(s) not found in the 2nd one:
[040684-040684] ---> [021754] {1}
If you press the gear button, it will "green" out the range, tidying it up. Keep doing this until the whole thing is green.

And now for the important part for you two:
Top middle there's a headphone button, "audio tracks".
If you load audio 1 and audio 2, MP3 tracks for movies 1 and 2, you can output either audio 2 synced to movie 1, or vice versa.
That gives you a single MP3 file where it uses audio 2 for movie 1, and audio 1 for parts missing from audio 2.

So, in short:
Convert both animes to XVID as .avi
Make both soundtracks into MP3
Load both .avis (top left)
Run automatic alignment (gear button)
Finish manual alignment (bottom right, green arrow button and such)
Select both .mp3s (headphone button)
And finally "save track as..." and you're done!


P.S.: I've done this for several full-length movies, such as 1966's "Is Paris Burning?" so I can get rid of dubbing: Instead of everyone being dubbed to English, or French, or German, now everyone speaks their original languages, after I combined the soundtracks.
I have a project going on with the Soviet "Blockade" movies (1973-1977), where the Russian dub is pan-and-scan 4:3 and the German dub is full 2,2:1 but lousy quality, so I'll need to learn AutoOverlay, and I'm hoping to use the FFMatch "top text" for that, since I'll align them anyway for the soundtrack.
So if FFMatch turns out useful, good thing I came here checking up on AutoOverlay! And sorry for going off topic!

[Frank62]

This looks really interesting! The question will be, how exact matches are, when the two sources are really different in resolution, position, etc. I will check this, thank you very much!

[poisondeathray]

If somebody out there has a perfect solution for this particular problem (full automization of soundtrack alignment of two slightly different sources)...? I would even pay a nice amount of money for this.

Auto sync clips by similar audio, slightly different cuts or cameras, already exists with software/plugins like plural eyes.

Adam wants sync clips by video similarity (he has different audio, different language).

Maybe temporal alignment iterations, and avisynth optimizer (zopti) using temporal iterations
https://forum.doom9.org/showthread.php?t=175723

[Frank62]

Auto sync clips by similar audio, slightly different cuts or cameras, already exists with software/plugins like plural eyes.


Yes, I know. For mixing shots from different angles, but with very similar audio.


Adam wants sync clips by video similarity (he has different audio, different language).

Yes, that's my purpose, too. And the tool, Goggen240 recommended is very promising. I will test this with different scenarios. If the automatic matching algorithm was doing well in different situations it would be a big progress in work and time saving for me. Maybe I can use the protocol to do the final sound mixing myself, I'll see.


Maybe temporal alignment iterations, and avisynth optimizer (zopti) using temporal iterations
https://forum.doom9.org/showthread.php?t=175723
Will not be sufficient, I fear. I tried a lot in the last few years. The way that is used by the above tool, to split in scenes before even trying to find 100% matches, seems mandatory. Very difficult to automate this process in pure avisynth.

And I still have no really good idea of how to compare two sources that may be different in
position (even changing)
width/height (even changing)
sharpness

AND to get results that are nearly 100% good matches. A really tough problem, I fear.

[Goggen240]

This looks really interesting! The question will be, how exact matches are, when the two sources are really different in resolution, position, etc. I will check this, thank you very much!


And I still have no really good idea of how to compare two sources that may be different in
position (even changing)
width/height (even changing)
sharpness

I've been doing comparative tests on a Soviet movie that is available in 4:3 Russian pan-and-scan (cropped sides) and 2.35:1 German (cropped top/bottom). Both have been cropped from something probably 2.20:1; the full height of the Russian version (1.33:1.00) and the width of the German version (closer to 2.20:0.94). Also, the position of the 4:3 version jumps around side-to-side inside the 2.35:1 version, as the pan part of pan-and-scan implies.

I checked old and new versions against this material:
The 2011 FFMatch version worked fine, but slowly, and the 2020 version would only get going once I disabled Preferences->Checks->Scaling, which was the only setting that changed between the two versions.
But yes, it will match a tall image moving around inside a wide image, with wildly different framing, colour, contrast, and edit.
First run with 2020 version it got about 95k out of 157k frames matched, where the 2011 version got 138k, so some more settings have to change, but it works.

In the past I've matched Blu-rays to VHS rips, so it is reeeal flexible if you're fine with some manual labour...

[Frank62]

But yes, it will match a tall image moving around inside a wide image, with wildly different framing, colour, contrast, and edit.

Wow! Fascinating! Do you have any idea how comparing is done?

And: From where did you get the older version? I would like to compare with the newer, too.

[Goggen240]

And: From where did you get the older version? I would like to compare with the newer, too.

I got it from that site, but it seems gone now. I couldn't seem to attach it to this post, but I can send it to you by some reasonable conveyance, it's 11MB.

Wow! Fascinating! Do you have any idea how comparing is done?

It compares frames using SIFT. As far as I understand it, that's an algorithm that finds patterns (say, purple corner on blue background) and assigns points to them, and then it looks for similar points in both files, and if enough points line up it assigns it as a match. It deals with rotation, scale and warp really well; it treats the point patterns as "constellations".
The same technique is used in photogrammetry, a 3D-modelling method where you take dozens (or tens of thousands) of pictures of an object from different angles, and it SIFT matches them to map out the position of cameras, and work out the parallax between them to find depth, and map the point depths to 3D space. I've made 3D models of a dead rat and a tractor, among others. But those are really the high points.

[StainlessS]

I couldn't seem to attach it to this post, but I can send it to you by some reasonable conveyance, it's 11MB.
You can post a file without account on SendSpace.com, file deleted 30 days after most recent download.
[If you have an account, it will email you a warning 7 days (I think) before file deleted]

Max file size for D9 attachment is about 200KB.

[Frank62]

It compares frames using SIFT.

Learned something. Really new to me, and very, very promising!

Does any filter for avisynth exist that uses this technique?

[Goggen240]

Here you go, the 2011 version of FFMatch:

https://www.sendspace.com/file/ghzfj7

Learned something. Really new to me, and very, very promising!

Does any filter for avisynth exist that uses this technique?

I'm not sure, but phones use it for panorama stitching, and possibly camera deshake methods. There are some camera deshake filters that use very similar methods; I found somebody's FFMPEG project that uses BRIEF, whatever that is. But it seems to be a similar same point-finding constellation-matching technique.
"Deshaker" for VirtualDub does not use SIFT though, that matches blocks for motion.

[Frank62]

Great, thank you.

Besides I found a .net library ( http://www.nowozin.net/sebastian/tu-berlin-2006/libsift/ , seems a bit older) and a C# project ( https://www.codeproject.com/Articles/95453/Automatic-Image-Stitching-with-Accord-NET ).
If I only had the time, but I will try everything as soon as I will find some, ffmatch first, of course. Very curious.

[AdamCarter]

Goggen!!!!

This looks amazing.

What do we need to do with the github files
I have no programming skillz

Thanks

[StainlessS]

What do we need to do with the github files
I have no programming skillz


I got
503 Service Temporarily Unavailable
Think I got that the day it was posted too.
[It might be a "region restricted" thing, ie not available in your/my country].

[AdamCarter]

Stainless i was able to download the files

Try using a vpn maybe to connect.

Would love to see this as an aviscript that could work without converting to xvid.

[StainlessS]

without converting to xvid.
OK, link is working now, was as it said "temp unavailable".

I remember ffmatch from some time back, it requires XVid to work its magic, it uses compression data
(DCT, Discrete Cosine Transform stuff I think) and wil not work with decompressed video (as in avisynth).
Its also the reason that it is so fast.

[AdamCarter]

Well at least it tells me which frames are good or mismatched, for avisynth i can use the trim function on the real video. It’s a step in the right direction.

Wonder if something similar could be programmed for avisynth from scratch.

It would go a long way for restoration.

[introspected]

v0.4.0 with warp transformations released: https://github.com/introspected/AutoOverlay/releases/tag/0.4.0

[wonkey_monkey]

with warp filter by wonkey_monkey

Yay! I'm useful!

[introspected]

Yay! I'm useful!
Thanks again for your support

[wonkey_monkey]

I notice it says "Only 8 bit clips support" on that line, but Warp should support all clips. Is it a limitation of AutoOverlay?

[introspected]

I get this message for the YUV420P16 color space:
warp/quad: High bit depths not yet supported
sample: ColorBars(pixel_type="YUV420P16").Warp(0,0,0,0, 2,2,2,2, 4,4,4,4)
Note that I use v0.1b

[StainlessS]

"Warp (includes quad)" here shows v0.1:- http://horman.net/avisynth/
Granted its a little non obvious what any of Wonkey's stuff is, no version info in zip name (could all be exact same version),
and no version resource on dll.
Its no real surprise that he sometimes posts the wrong version, sad but there you go.

[introspected]

"Warp (includes quad)" here shows v0.1:- http://horman.net/avisynth/
Granted its a little non obvious what any of Wonkey's stuff is, no version info in zip name (could all be exact same version),
and no version resource on dll.
Its no real surprise that he sometimes posts the wrong version, sad but there you go.
v0.1b is the latest without this (https://forum.doom9.org/showthread.php?p=1923288#post1923288) problem. So that's why it's included to the AutoOverlay package.

[StainlessS]

OK, so Wonkey does his beta version after the stable version, novel choice, to each his own. :)

[wonkey_monkey]

It's true, my ways are inscrutable and mysterious.

[StainlessS]

Anybody any success getting this to do anything at all [v0.4.0] ?
I've given up trying to get it loading plugs directly, [no idea what to do with MathNet.Numerics.dll and simd.dll]
and just dumped the lot in plugins [with ImageSeq.dll for jpg loading].
Only loading image, and display, crash on close Vdub2, Access Violation [x86 OR x64]. [ie no use of AutoOverlay, except during plugin autoload from plugins]

Also, for x64, using AvsMeter64 -avsinfo, get this [32 bit dll's ???]


C:\Users\root>avsmeter64 -avsinfo

AVSMeter 3.0.8.0 (x64), (c) Groucho2004, 2012-2021

VersionString: AviSynth+ 3.7.0 (r3382, 3.7, x86_64)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 21:36:39 (UTC)
PluginDir2_5 (HKLM, x64): C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plug
ins
PluginDir+ (HKLM, x64): C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plug
ins


[C++ 2.6 Plugins (64 Bit)] [Version, Time stamp]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\AvsFilterNet.dll [1.0.4
.0, 2020-06-28]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\ImageSeq.dll [n/a, 2021
-01-11]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\warp.dll [n/a, 2020-09-
13]

[Uncategorized DLLs (32 Bit)] [Version, Time stamp]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\AutoOverlay_netautoload.
dll [0.4.0.0, 2021-01-17]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\MathNet.Numerics.dll [4
.12.0.0, 2020-08-02]

[Uncategorized DLLs (64 Bit)] [Version, Time stamp]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\AutoOverlayNative.dll [
n/a, 2021-01-07]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\Simd.dll [0.3.0.0, 2020
-08-28]

C:\Users\root>

Then AvsMeter64 crashes.

Avsmeter -avsinfo
32 bit, just hangs [seemingly forever].

[StvG]

I have no issues. x64 version used.
For example:OM=Colorbars(pixel_type="yv12").ColorRangeMask(20,60)
WS=Rect(50,60,70,80)
OverlayEngine(OM.ConvertToY8(), WS.ConvertToY8(), configs=OverlayConfig(subpixel=2))
OverlayRender(OM.ConvertToYV24(), WS.ConvertToYV24(), debug=true)
https://images2.imgbox.com/95/72/EGv7pQNf_o.png

[StainlessS]

Thanks for responding StvG,
maybe my machine [W7 Pro] is misconfigured or something.
I'm not seeing any DotNet 4.8 installed, although I do see 3 updates for it installed.
I'm gonna forget this again for a while.

EDIT: Guess I'm gonna re-install W7 at some point soon, already got the ISO.

[introspected]

Thanks for responding StvG,
maybe my machine [W7 Pro] is misconfigured or something.
I'm not seeing any DotNet 4.8 installed, although I do see 3 updates for it installed.
I'm gonna forget this again for a while.

EDIT: Guess I'm gonna re-install W7 at some point soon, already got the ISO.
Copy x86/x64 DLL's to AviSynth plugins folder.
DLL's may be blocked after downloading. Open file properties to unblock them.
All dll's from x64 folder should be unpacked to the x64 plugins folder.
I don't know how AvsMeter64 works but I guess it marks AutoOverlay dll's as uncategorized because they are not avisynth plugins themself. Only AvsFilterNet is the native plugin.

[StainlessS]

Thanks for your answer Introspected,
Most people object to non avs plugins/scripts in the bittage specific Plugins directories, so eg
32 bit system or 64 bit system with x64 avs, standard OS dll's go in system32.
64 bit system and x86 avs, standard OS dll's go in SysWOW64.

It seems strange that there should be 32 bit AutoOverlay supplied dll's in x64 plugins directory, and is rightly flagged as probable problem by AvsMeter.

[Uncategorized DLLs (32 Bit)] [Version, Time stamp]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\AutoOverlay_netautoload.
dll [0.4.0.0, 2021-01-17]
C:\VideoTools\AvisynthRepository\AVSPLUS370_x64\plugins\MathNet.Numerics.dll [4
.12.0.0, 2020-08-02]


Can you please outline how to load your plugins that are using manual load and NOT using AVS AutoLoad plugins directory.
(LoadDLL (http://avisynth.nl/index.php/LoadDLL) can be used to load SYSTEM dll's not in system32 or SysWOW64)
Many of us dont like dumping everything in plugins directory, and would prefer to manual load where necessary.

[introspected]

.NET dll's are cross-platform so they are the same both for x86 and x64 :)
Explicit plugin loading is described in the readme file:
LoadPlugin("%plugin folder%\AvsFilterNet.dll")
LoadNetPlugin("%plugin folder %\AutoOverlay_netautoload.dll")

v0.4.1 released: https://github.com/introspected/AutoOverlay/releases/tag/0.4.1

[Tlen]

Hi,
i'm trying to use the plugin but no luck.
Virtual dub just crashes on opening.

I've tried loading the dll from plugin folder

LoadPlugin("D:\Programs\AviSynthPlugins\AutoOverlay\x64\AvsFilterNet.dll")
LoadNetPlugin("D:\Programs\AviSynthPlugins\AutoOverlay\x64\AutoOverlay_netautoload.dll")

or copying the x64 files in C:\Program Files (x86)\AviSynth+\plugins64+

Same situation.

I've not found in the readme the section on how to load manually all the needed .dll

Could you help?

Thanks

[torturesauce]

Hello, all! I am bad at AviSynth and would like your help. I can use AutoOverlay without any problems, I want to use ColorAdjust by itself in order to color match two video frames. On AutoOverlay's Github, these are the instructions:

ColorAdjust(clip sample, clip reference, clip sampleMask, clip referenceMask, float intensity,
int adjacentFramesCount, float adjacentFramesDiff,
bool limitedRange, string channels, float dither, float exclude, string interpolation,
bool extrapolation, bool dynamicNoise, bool simd, bool debug)

So, this is my script. I want to make mpv-shot0002.png look like mpv-shot0001.png and I used all the defaults mentioned here (https://github.com/introspected/AutoOverlay#coloradjust):

video = ImageSource("c:\mpv-shot0002.png").ConvertToYV12()
video = ImageSource("c:\mpv-shot0001.png").ConvertToYV12()

ColorAdjust(clip="c:\mpv-shot0002.png", sample="c:\mpv-shot0002.png", reference="c:\mpv-shot0001.png", samplemask=0, referencemask=0, intensity=1,
adjacentframescount=0, adjacentframesdiff=1,
limitedrange=true, channels="yuv", dither=0.95, exclude=0, interpolation="linear",
extrapolation=false, dynamicnoise=true, simd=true, debug=false)
)

This script doesn't work. I'm sure I've made tons of mistakes, but there are some mind-boggling things about this that prevent me from fixing it:

What is the correct way to leave both sampleMask and referenceMask empty? Is zero the correct value?
Apparently the latest version of ColorAdjust has a new, undocumented parameter called "CacheId". What should I add there?
Do I need to call MathNet.Numerics.dll, or is it automatically loaded?


I use AutoOverlay 0.4.2, AvsPmod x64, AviSynth+ x64 (both latest versions) on Win 10 x64.

[introspected]

v0.4.3 released: https://github.com/introspected/AutoOverlay/releases/tag/0.4.3

[StainlessS]

Thanks very Much :)

[introspected]

v0.5 released: https://github.com/introspected/AutoOverlay/releases/tag/0.5.0
Many user functions added. Most interesting one is for color level fixing at frame borders.

[StainlessS]

Thanks again :)

[introspected]

v0.5.1 released: https://github.com/introspected/AutoOverlay/releases/tag/0.5.1

[Dogway]

I'm having a hard time trying to make this work.
I want to align two manga pages in different languages, so I create a mask for only the four corners (so it doesn't pick the text bubbles).
a=ImageSource("S:\a.jpg",start=0,end=0).ConvertToY8("PC709")
b=ImageSource("S:\b.jpg",start=0,end=0).ConvertToY8("PC709")

# masking everything except four corners
msk = a.BoxMask(0,250,0,250).PadResize(a.width()/2,a.height()/2,biasw=b.width()/4,biash=b.height()/4)
msk = stackhorizontal(msk,msk.FlipHorizontal())
msk = stackvertical (msk,msk.FlipVertical())

cfg = OverlayConfig(branches=3,acceptableDiff=20,correction=1,angle1=-5,angle2=5,debug=false)
OverlayEngine(a,b, configs=cfg,backwardFrames=0,forwardFrames=0,sourceMask=msk,overlayMask=msk,editor=false)
OverlayRender(a,b)

It doesn't seem to do anyhing. Is there also a way to enable autoassisted distort or warping?

[Frank62]

For me it works, but I have another big problem with the result:
I have two clips, one in HD 1920*1080 in 16:9 from a BluRay and the other in SD 720*480 letterboxed 1.66:1. The goal was to extend the cropped 1.77:1 HD at top and bottom with the 1.66:1 source. This works quite well, when I first

-crop the SD clip so that there are no black borders left
-resize the SD clip so that it nearly fits the HD clip

With the correct parameters, especially minoverlayarea and minsourcearea, the result seems quite good at first sight.
But in several scenes the whole picture shakes a lot, which it doesn't at the original sources (in none of the two). It seems that resizing and fitting to each other does not work 100% but only 99%, so that there are small errors all the time that cause random misaligning and a shaking over the whole result.

This would be acceptable, if only the overlayed clip would shake, but the problem ist that BOTH clips shake! No matter what I declare as overlayed and source clip, the clips are touched BOTH by the filter.

So the question: Does anyone know if there are settings that let the source clip 100% alone, without any changing, by only adapting the overlay clip, in my example the SD-clip? So that only the extended area shakes a little bit, but the rest keeps stable?

It seems that @introspected didn't visit us a longer time, I hope he reads this.

:thanks:

[Frank62]

If anyone interested:
1. It seems that the shaking that is not in the source and thus generated by AutoOverlay is mostly horizontal, but not 100% sure.

2. The parameter "backwardframes" seems to be sensitive to it, so when you turn it to 4 (instead of 0) most(?) of the instability is gone. Seems to refer temporally somehow and so calm down the wobbling. Strange: higher values like 10 seem to be rather counterproductive.

[PoeBear]

It might be worth also checking on the sister thread at FanRes (https://forum.fanres.com/thread-1974.html). There's been more discussion there in the past, and there could be some useful ears that aren't over here

[Frank62]

Thanks for the hint, but for me not necessary any more. With the above parameter plus a bit of stabilisation afterwards my result looks very good meanwhile.

[BilboFett]

I've spent probably about 3-4 hours now testing, retesting, trying every possible setting.. I cannot get 2 identical scenes (same rez, similar color grading, same fps, etc.) to overlay correctly. Is it because one is PAL widescreen and the other is PAL fullscreen? I want to overlay the higher quality widescreen footage on top of the lower quality fullscreen, so it becomes like an "open matte". Would it help if I artificially forced one or the other's aspect ratio? I also cropped the widescreen one.. didn't make any difference. Res of both are 720x576 (widescreen one gets down to 544 if I crop). Here's my script, which is prob full of mistakes:
----
widescreen=directshowsource("V:\PAL SCENE 16x9-from premiere.mov")
openmatte=directshowsource("V:\PAL SCENE Full Frame-from premiere.mov")
OverlayConfig(aspectRatio1=1.78, subpixel=3)
OverlayEngine(openmatte, widescreen, editor=true)
#ComplexityOverlay(widescreen, openmatte)
OverlayRender(openmatte, widescreen, colorAdjust=1, mode=3, gradient=20)
#OverlayCompare(widescreen, openmatte)

[Frank62]

Did you try to resize first both to (nearly) quadratic pixels?
4:3 source to 768*576.
1.77 source to 1024*576, then crop.

[BilboFett]

Did you try to resize first both to (nearly) quadratic pixels?
4:3 source to 768*576.
1.77 source to 1024*576, then crop.

I will try that, thank you

[BilboFett]

Did you try to resize first both to (nearly) quadratic pixels?
4:3 source to 768*576.
1.77 source to 1024*576, then crop.

Thank you, this was a good tip!
That indeed lined them up the best they've ever been, but, for some reason, the two images warp and shift around (so they match?) and its just really artificial and fake looking, almost like a CG effect w/ the camera.
Wish I knew what was going on.
Its the identical scene, frame for frame, same rez, same fps, similar color grade, but one is full-frame and the other is letterbox..I don't know why they won't line up 100% and meld/merge into one.

[Frank62]

I didn't use AutoOverlay for quite a time, I will look at what I used, and what worked. But I also remember, that one source did the same as yours, and I couldn't fix it... Will look into this tomorrow.

[Frank62]

This is my code snippet (vu and vd were the two clips):

config=overlayconfig(subpixel=2,minoverlayarea=80,minsourcearea=80)
o=overlayengine(vu.converttoy8(),vd.converttoy8(),configs=config,backwardframes=0,forwardframes=0,mode="update",stabilize=false)
vd=overlayrender(o,vu,vd,debug=false,opacity=1,colorAdjust=1)

I don't remember much, but stabilize did never do anything.
Very important were the numbers minoverlayarea and minsourcearea. (You should look in the readme)

Good luck!

[BilboFett]

This is my code snippet (vu and vd were the two clips):

config=overlayconfig(subpixel=2,minoverlayarea=80,minsourcearea=80)
o=overlayengine(vu.converttoy8(),vd.converttoy8(),configs=config,backwardframes=0,forwardframes=0,mode="update",stabilize=false)
vd=overlayrender(o,vu,vd,debug=false,opacity=1,colorAdjust=1)

I don't remember much, but stabilize did never do anything.
Very important were the numbers minoverlayarea and minsourcearea. (You should look in the readme)

Good luck!

I tried it, it sorta works, there's still weird shifting and morphing between the 2 sources thats visible. Wish it wasn't doing that. Oh well

[poisondeathray]

Its the identical scene, frame for frame, same rez, same fps, similar color grade, but one is full-frame and the other is letterbox..I don't know why they won't line up 100% and meld/merge into one.

I tried it, it sorta works, there's still weird shifting and morphing between the 2 sources thats visible. Wish it wasn't doing that. Oh well

Maybe there are more differences / other distortions between the sources ? I find AutoOverlay does not tend to perform well with warping

Can you align them manually with just crop and simple translations ?

Sample ?

[introspected]

v0.6.1 released: https://github.com/introspected/AutoOverlay/releases/tag/0.6.1
- multiple clips rendering
- new options in engine and render
- auto-align refactoring

[BilboFett]

Did you try to resize first both to (nearly) quadratic pixels?
4:3 source to 768*576.
1.77 source to 1024*576, then crop.

What do I set (quadratic) for source NTSC:

1) Widescreen source is 720x360 (non-anamorphic letterbox DVD, in a rectangle, seems to look right @ 4:3, 16x9 stretches it)
2) overlay (open-matte) source is 704x480. It looks "right" when AR is set to 4:3 or 8:9

TIA!

[Tlen]

Hi to all,
I'm testing this promising plugin,
but after initial enthusiams with default parameters,
I got in deep with gradient=0 and it resulted in non-perfect align.

I'm not able to perfectly align the picture no matter what parameter.
In Photoshop the aligning result perfect just by a resize + reposition.

Anyone can help?

Here's 2 example pictures (but it applies througout the video).

EDIT:
Done with other methods.

I'll retry the plugin with the new version.

[introspected]

In the new version, HDR support is improved, auto-alignment and color matching algorithms (including SDR 709 <-> HDR PQ 2020) are enhanced, and performance is boosted: https://github.com/introspected/AutoOverlay/releases/tag/0.7.8