Dual Clip/Window video editor/viewer ?

[StainlessS]

Simple question, anybody aware of a dual window dual video clip, viewer / editor, where can scroll top window and bottom window separately or both together.
(for alignment purposes, Avisynth supporting, open source preferable).

Thanks for any replies.

EDIT:
I got a clip where there is supposed to be Theatrical version and Uncut version, and a missing/unreleased longest original version.
Disk I got has both existing versions, but there seems to be some extra in short theatrical that is missing from UnCut/Directors_Cut.
So, could contain some of the missing stuff (I think I've found one such scene), above viewer/editor could assist.
EDIT: SirPlant.avsi (inserts frames from better quality short clip into lesser quality long clip) dont deal well where both clips have unique sequences,
such a vewer could assist in figuring out what I gotta do.

[VoodooFX]

Why not use StackVertical()?

[StainlessS]

Cos it dont have any scrolly wheel thingies.

EDIT: Can AvsPMod do that ? [dual window with individual and joint slider whotsits].
Joint slider would adjust both relative to individual sliders.

[VoodooFX]

AvsPmod has. And you can detach the preview window.

EDIT:
Dunno about dual window there.
But if in the cuts the scenes are in a same sequence then you just go from start and align scene by scene, gradually inserting Trims into a shorter version.

[StainlessS]

Nah, Nah, Nah, I want something that is easy without any trimmy thingies.
Dual AVS input support only, can just be a twin pane viewer and nothing else.

EDIT: Gotta attach screen protector to SM-T295 (Galaxy TAB A 8" 2019), then off to pub.

EDIT: Something similar to this:- https://support.microsoft.com/en-us/office/video-view-files-in-split-and-multiple-windows-8c6377a1-ed45-498e-80dc-635a6ae34547
only for video not Word/Excel/etc files.

EDIT: Apparently VLC can view two but I also want easy individual and joint scrolly thingy control. [Google link]
https://www.google.co.uk/search?q=twin+pane+video+viewer&source=hp&ei=Q5i5Y4fRBMf5gQaY0a-YCQ&iflsig=AK50M_UAAAAAY7mmUxMeC9OAtZ-sXCvdp1HLxLBJxQAU&ved=0ahUKEwjH_tWc67X8AhXHfMAKHZjoC5MQ4dUDCAk&uact=5&oq=twin+pane+video+viewer&gs_lcp=Cgdnd3Mtd2l6EAMyBwgAEB4QogQyBQgAEKIEMgcIABAeEKIEMgUIABCiBDoECAAQQzoFCAAQgAQ6CwgAEIAEELEDEIMBOhEILhCABBCxAxCDARDHARDRAzoICC4QsQMQgwE6DgguEIAEELEDEMcBENEDOgUIABCRAjoHCC4Q1AIQQzoQCC4QgwEQxwEQsQMQ0QMQQzoKCAAQsQMQgwEQQzoICAAQsQMQgwE6EQguEIMBEK8BEMcBELEDEIAEOhAILhCxAxCDARDHARDRAxBDOgQILhBDOg4ILhCABBCxAxCDARDUAjoICAAQgAQQsQM6BQguEIAEOg4ILhDHARCxAxDRAxCABDoOCC4QgAQQsQMQxwEQrwE6BwgAEIAEEAo6CwguEIAEELEDEIMBOg4ILhCDARDUAhCxAxCABDoLCC4QgAQQxwEQrwE6CwguEK8BEMcBEIAEOg0ILhCvARDHARCABBAKOgUIABCGAzoGCAAQFhAeOggIABAWEB4QDzoHCCEQoAEQCjoFCCEQoAE6CAghEBYQHhAdOg0IIRAWEB4QDxDxBBAdUABYgDRg0jloAHAAeACAAd8BiAHeF5IBBjMuMjAuMZgBAKABAQ&sclient=gws-wiz

EDIT: Above google link might have some, I'll check in/after pub.

[wonkey_monkey]

Something like this?

https://i.imgur.com/KIcKxel.png

The trouble is it's very much a work in progress and completely undocumented. I had high hopes of writing it properly when I started, but it turned into more of a personal gizmo, with GUI modes for unreleased plugins. I've been meaning to make it available as it is at some point, just to see whether people want to take any ideas from it.

I also haven't found a "proper" way of doing playback - it does work okay, but if you scroll around or try to use any of the tools there is some tearing and you may see parts of the next frame before you're supposed to. That's probably the main reason I haven't released anything yet.

There's also no audio support yet.

[Reel.Deel]

That sounds/looks very interesting, wonkey_monkey. Hopefully you'll release it one day. I'm still holding my breath on your other teaser (https://forum.doom9.org/showthread.php?t=183457) :)

[wonkey_monkey]

Oh yeah... that... I think it's pretty much ready to go but I need to make a video tutorial to accompany it, as I think it's too complex to explain well in words, and I keep getting distracted by other projects (like Avisynth+ Studio above).

[StainlessS]

RD, on the 'other teaser', WM said it would be 'ready soon', so probably almost there already.

Wonkey, that looks fine. but maybe one above the other only because of prevalence of wide screen stuff. [probably dont matter which]
Would also need maybe only 1 slider, with switch window mode button, eg top window, bottom window, both windows scroll together relative current frame locked offsets,
and also show frameNo for each window.

I think VDub2 may use v4w [EDIT: or is it vfw] play control, the directshow stuff basically can use play and stop and not much else [last time I made any attempt at doing similar in VB 10 or 12 years ago].
You could do worse than check out VDub2 source, but there will be one helluva lot of it :)

Shekh-The-Magnificent, might be persuaded to point you in the right direction, or at least tell which source files to dissect,
his last post was 22 July 2022, however was last on-line, 27 Dec 2022.

EDIT: AutoIt has a tool (AU3Info) which show details about windows and scrollbars and the like, and reading slider positions and button text etc,
useful in dissecting binary or Source. [Visual Studio 20?? might have similar]
Its what I used to write SawBones and VD_EdKey apps that control Vdub2, and also MeGUI_AutoEncodeBatcher.

https://i.postimg.cc/d3zPRtfL/AU3Info.jpg (https://postimg.cc/V0BVm1m8)

[wonkey_monkey]

Wonkey, that looks fine. but maybe one above the other only because of prevalence of wide screen stuff. [probably dont matter which]

Pressing tab in that mode cycles the grid through all possible configurations (2 clips: 1x2, 2x1. 3 clips: 3x1, 2x2(1 empty), 1x3. 4 clips: 4x1, 2x2, 1x4. And so on.

Would also need maybe only 1 slider, with switch window mode button, eg top window, bottom window, both windows scroll together relative current frame locked offsets,

Not sure what you mean. The panes are permanently locked to the same offset.

and also show frameNo for each window.

Also permanently locked to the same frame (frame number and timestamp bottom right).

Anything beyond that may be invoiceable...

[StainlessS]

Not sure what you mean. The panes are permanently locked to the same offset.
You can alter either top / bot (or Left / Right) mode slider, to align one clip to the other clip,
then switch to joint/paired mode slider to advance both clips together until they maybe go out of sequence again.
(if one clip ends prematurely, just show end frame for that clip)
A 3 option MODE radio button [or set of 3 Mutually Exclusive buttons] would switch between LEFT/RIGHT/JOINT modes for the single slider.

Also permanently locked to the same frame (frame number and timestamp bottom right).
Nope, both maintain alignment as set prior to switching to joint/paired mode,
step one frame forward, both clips step one frame forward.

I have not put a lot of thought into exactly how it would work, but that seems quite straight forward,
its only for viewing and spotting when they go out of sequence so you can see which one has extra frames/sequence.
Screwing around with trims and splices would seem a lot less elegant.
(Also, would be great if could also load AVI as well as AVS)

EDIT: The first link I gave (post #5) is Video from M$ for doing similar sort of thing viewing two long Word Docs side-by-side (Office 365).
https://support.microsoft.com/en-us/office/video-view-files-in-split-and-multiple-windows-8c6377a1-ed45-498e-80dc-635a6ae34547

[wonkey_monkey]

Nope

Well, "yep", because I'm telling what the program does, not what you want it to do ;)

But a keyboard shortcut to change the frame offset for the second pane might not be too difficult. Just need to give some thought to a good way of indicating to the user that such an offset is in effect.

(Also, would be great if could also load AVI as well as AVS)

That it can do. AviSource is the default input filter for .avi. Other files will prompt for a source filter.

But to use multi-pane mode you'll have to do so via a single .avs clip.

[StainlessS]

But to use multi-pane mode you'll have to do so via a single .avs clip.
Nah, Nah, Nah, I want more !

[wonkey_monkey]

I could MAYBE allow you to drag-n-drop multiple AVI files or one .avs file + multiple AVI files. Maybe.

You might find "Overlay diff" mode useful:

https://i.imgur.com/THa1Sn2.png

[StainlessS]

Nah, Nah, Nah, they may be hundreds/thousands of frames offset, [entire scenes missed out]
I want more !

(no dont worry bout it, was just a thought, I might still find something via that google page of links [ and if I do, I'm not gonna tell ya :) ])

[wonkey_monkey]

Not a problem, the offset can be changed. I'm thinking < and > keys to change the second clip's offset. Maybe shift-drag on the timeline to do that too, with a red marker to show the second clip position. The frame number to the right of the timeline would say "101 (103)" or "101 (+2)" (switchable) to indicate the offset.

[VoodooFX]

I think it's pretty much ready to go but I need to make a video tutorial to accompany it, as I think it's too complex to explain well in words
That's nice, could you share it for the sneakpeek tests without that video?

[StainlessS]

I'm thinking < and > keys to change the second clip's offset.
Sounds good.

timeline would say "101 (103)" or "101 (+2)" (switchable) to indicate the offset.
Or even "103 (101+2)", without need for switch.

[wonkey_monkey]

While I work on this, just a note to say that it has native support (no driver config needed, maybe no driver needed at all) for the Contour ShuttleXpress (https://www.amazon.co.uk/gp/product/B0032Y0OH0) (though it's much more expensive than it was 7 years ago, there are some on Ebay). I'll add support to use the buttons to change which clips are to be jogged.

[wonkey_monkey]

Progress:

https://i.imgur.com/t2uYzcB.pnghttps://i.imgur.com/P9wAkKk.png

[poisondeathray]

Looks cool ! Do I spy some some roto/masking tools ?

[wonkey_monkey]

Pay no attention to the roto/masking tools behind the curtain!

https://www.youtube.com/watch?v=-7Yi0TbnYQQ (briefly visible at 11s)

[spoRv]

Bino does this:
https://bino3d.org/
even if it's basically a 3D player, it could play two (related or not) clips at the same time.

[PoeBear]

Why not use StackVertical()?

That's how I do it in AvsPmod. You just need to find the earliest same matching frame and use their frame differential to either Trim or BlankClip the start of the script. Then you can scrub through until you find a mismatched scene, and then repeat the initial step, either Trimming or BlankClips until the sources line up again. Repeat until done

#Master source of truth
MASTER=LWLibavVideoSource("MASTER.mkv").ShowFrameNumber(x=8,y=32).Subtitle("Master",align=7)

#Imperfect source for editing
SOURCE=LWLibavVideoSource("SOURCE.mkv").ShowFrameNumber(x=8,y=32).Subtitle("WIP",align=7)

#Blank frames and Trims, to match the sources
WIP=Blackness(16,1920,1080,"YV12",24000,1001,0)+\
SOURCE.Trim(0,87654)+\
Blackness(12,1920,1080,"YV12",24000,1001,0)+\
SOURCE.Trim(88765,89103)+\
SOURCE.Trim(98765,0)+\
Blackness(9999999,1920,1080,"YV12",24000,1001,0)

StackVertical(MASTER,WIP)

[StainlessS]

Also, with this double window thingy, maybe need similar to VDub scene change whotsit [arrow thingies on status bar], but for frame mismatch where goes out of sync.

[Dogway]

I do this constantly, using two synced tabs in AvsPmod. This is an example of an old script:
srcA
b=srcB

trim(0,86288)+trim(b,86288+1,86309)+trim(86288+1,0)
trim(0,86498)+trim(b,86498+1,86502)+trim(86498+1,0)
trim(0,86721)+trim(b,86721+1,86724)+trim(86721+1,0)
trim(0,86782)+trim(b,86782+1,86782)+trim(86782+1,0)
trim(0,86889)+trim(b,86889+1,86889+3)+trim(86889+1,0)
trim(0,86958)+trim(b,86958+1,86970)+trim(87002,0)
trim(0,87060)+trim(b,87060+1,87130)+trim(87060+6,0)
trim(0,87234)+trim(b,87234+1,87654)+trim(87355+1,0)
...

I search for scene cuts and see when they don't match, then adjust the trims. I usually do this for audio though when there's no recut Spa audio.
I switch between synced Tabs using ctrl+Tab.

[StainlessS]

The functionality that I want should also work with AVI/mov/mpeg etc files, ... not necessarily avs script.

We wants it, we wants it now!

[wonkey_monkey]

What I have so far:

1. Although it can open individual clips without an Avisynth script, it can currently only handle multiple clips if they are returned as an array from an Avisynth script. In any case, you have to wait for indexing so there's little advantage to being able to open multiple files directly, but I'll work on it. Clips have to be the same resolution (but colourspace can vary).

2. Left/right arrow keys jogs both clips. <, > keys jogs left/first clip, [, ] keys jogs right/second clip. In side-by-side mode, you can use scroll left/right (gesture or tilting mousewheel) on each pane or on the timestamps (bottom right) to jog clips separately, or you can scroll left/right on the timeline to move both clips in sync.

3. Clicking/dragging on timeline jogs clips in sync. Holding shift jogs second clip only, holding ctrl jogs first clip only.

4. Keyboard shortcut "J", or red collapse arrow in timestamps, toggles offset mode on and off (turning it off resets the offset to 0).

Mismatch detection sounds like a good idea - maybe shift+ctrl+arrow key as in VirtualDub. For sanity's sake, it will work on 8-bit greyscale versions of the clips. What's the best way to make the detection? A selectable threshold on average pixel difference? Or averaged squared difference?

[StainlessS]

What's the best way to make the detection? A selectable threshold on average pixel difference? Or averaged squared difference?

Given that two clips intended for sync are more than possibly of different brightness/contrast or whatever, maybe similar to RT_Stats RT_LumaCorrelation would be best,
with care where correlation relatively close to 0.0 [two grey/white/black frames have no correlation but should really match], where additional average per pixel difference to judge if really dissimilar.
You can view correlation method in RT_stats Source, I also provided JMac698 with my source which he used in re-implementing his Correlation plugin (cant remember the function name he used).

Probably best the other way around, test per pixel diff, and only test correlation if judged different (where correlation can OverRide pixel difference conclusion to Not Different) [faster, correlation is quite slow to calc].
(would require two thresholds, could perhaps be both hard coded)

EDIT: I seem to recall that JMac posted his source (for many plugins) on some other site under Creative Commons licence, and then lost the lot completely, so maybe RT best to get algo from.

EDIT: RT files RT_Luma.cpp, and PVF2_Planar.cpp (for Planar).

EDIT: Low level stuff from PVF2_Planar

double __cdecl PVF_LumaDifference_Planar(const PVideoFrame &src,const PVideoFrame &src2,
const int xx,const int yy,const int ww,const int hh,const int xx2,const int yy2,bool altscan) {
const int pitch = src->GetPitch(PLANAR_Y);
const int pitch2 = src2->GetPitch(PLANAR_Y);
const int ystep = (altscan) ? 2:1;
const int ystride = pitch*ystep;
const int ystride2 = pitch2*ystep;
const BYTE *srcp = src->GetReadPtr(PLANAR_Y) + (yy * pitch) + xx;
const BYTE *srcp2 = src2->GetReadPtr(PLANAR_Y) + (yy2 * pitch2) + xx2;
__int64 acc = 0;
unsigned int sum = 0;
const int yhit = (altscan) ? (hh +1)>>1 : hh;
const unsigned int Pixels = (ww * yhit);
if(ww == 1) { // Special case for single pixel width
for(int y = yhit; --y>=0 ;) {
sum += abs(srcp[0] - srcp2[0]);
srcp += ystride;
srcp2 += ystride2;
}
} else {
const int eodd = (ww & 0x07);
const int wm8 = ww - eodd;
for(int y = yhit; --y>=0 ;) {
switch(eodd) {
case 7: sum += abs(srcp[wm8+6] - srcp2[wm8+6]);
case 6: sum += abs(srcp[wm8+5] - srcp2[wm8+5]);
case 5: sum += abs(srcp[wm8+4] - srcp2[wm8+4]);
case 4: sum += abs(srcp[wm8+3] - srcp2[wm8+3]);
case 3: sum += abs(srcp[wm8+2] - srcp2[wm8+2]);
case 2: sum += abs(srcp[wm8+1] - srcp2[wm8+1]);
case 1: sum += abs(srcp[wm8+0] - srcp2[wm8+0]);
case 0: ;
}
for(int x=wm8; (x-=8)>=0 ; ) {
sum += (
abs(srcp[x+7] - srcp2[x+7]) +
abs(srcp[x+6] - srcp2[x+6]) +
abs(srcp[x+5] - srcp2[x+5]) +
abs(srcp[x+4] - srcp2[x+4]) +
abs(srcp[x+3] - srcp2[x+3]) +
abs(srcp[x+2] - srcp2[x+2]) +
abs(srcp[x+1] - srcp2[x+1]) +
abs(srcp[x+0] - srcp2[x+0])
);
}
if(sum & 0x80000000) {acc += sum; sum=0;} // avoid overflow
srcp += ystride;
srcp2 += ystride2;
}
}
acc += sum;
double result = ((double)acc / Pixels);
return result;
}


double __cdecl PVF_LumaCorrelation_Planar(const PVideoFrame &src,const PVideoFrame &src2,
const int xx,const int yy,const int ww,const int hh,const int xx2,const int yy2,bool altscan) {
const int pitch = src->GetPitch(PLANAR_Y);
const int pitch2 = src2->GetPitch(PLANAR_Y);
const int ystep = (altscan) ? 2:1;
const int ystride = pitch*ystep;
const int ystride2 = pitch2*ystep;
const BYTE *srcp = src->GetReadPtr(PLANAR_Y) + (yy * pitch) + xx;
const BYTE *srcp2 = src2->GetReadPtr(PLANAR_Y) + (yy2 * pitch2) + xx2;
const int yhit = (altscan) ? (hh +1)>>1 : hh;
const unsigned int Pixels = (ww * yhit);

unsigned long cntx[256],cnty[256];
int i;
for(i=256;--i>=0;) {
cntx[i]=0; // Clear counts
cnty[i]=0;
}

unsigned int Sxy_lo = 0;
__int64 Sxy=0;
if(ww==1) {
for(int y=yhit ; --y>=0;) {
const unsigned int dx=srcp[0];
const unsigned int dy=srcp2[0];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;
srcp += ystride;
srcp2 += ystride2;
}
} else {
const int eodd = (ww & 0x03);
const int wm4 = ww - eodd;
for(int y=yhit; --y>=0 ;) {
switch(eodd) {
case 3: {
const int dx=srcp [wm4+2];
const int dy=srcp2[wm4+2];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;}
case 2: {
const int dx=srcp [wm4+1];
const int dy=srcp2[wm4+1];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;}
case 1: {
const int dx=srcp [wm4+0];
const int dy=srcp2[wm4+0];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;}
case 0: ;
}
for(int x=wm4; (x-=4)>=0 ; ) {
{
const int dx=srcp [x+3];
const int dy=srcp2[x+3];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;
}
{
const int dx=srcp [x+2];
const int dy=srcp2[x+2];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;
}
{
const int dx=srcp [x+1];
const int dy=srcp2[x+1];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;
}
{
const int dx=srcp [x+0];
const int dy=srcp2[x+0];
++cntx[dx];
++cnty[dy];
Sxy_lo += dx * dy;
}
}
if(Sxy_lo & 0x80000000) {Sxy += Sxy_lo; Sxy_lo = 0;}
srcp += ystride;
srcp2 += ystride2;
}
}

Sxy += Sxy_lo;
__int64 Sx = 0;
__int64 Sy = 0;
__int64 Sx2 = 0;
__int64 Sy2 = 0;

for (i=256;--i>=0;) {
__int64 z;
z = i * cntx[i]; Sx += z; Sx2+= i * z;
z = i * cnty[i]; Sy += z; Sy2+= i * z;
}

double num = ((double)Pixels * (double)Sxy) - ((double)Sx*Sy);

double div1 = ((double)Pixels * Sx2) - ((double)Sx*Sx);
double div2 = ((double)Pixels * Sy2) - ((double)Sy*Sy);

if(div1 < 0.0) div1 = - div1; // fabs
if(div2 < 0.0) div2 = - div2; // fabs

double div;
if ((div1<0.0001)||(div2<0.0001)) {
div = 0.000001;
} else {
div = sqrt(div1) * sqrt(div2);
}
double ret=num / div;
ret = max(min(ret,1.0),-1.0);
return ret;
// http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Mathematical_properties
}


http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Mathematical_properties

EDIT: The Wikipedia page above has changed quite a bit, what we used (I think) came from about this line on the page
This formula suggests a convenient single-pass algorithm for calculating sample correlations, though depending on the numbers involved, it can sometimes be numerically unstable.
Search for above in blue.
this is the part which fixes the numerically unstable thingy (I think)

double div;
if ((div1<0.0001)||(div2<0.0001)) {
div = 0.000001;
} else {
div = sqrt(div1) * sqrt(div2);
}


EDIT: Pre conversion to 8 bit (Y8) for detect clips, is probably more than sufficient so you dont have to worry about float or other HBD.

[wonkey_monkey]

I'll probably make the scene detection just a basic sum of absolute differences - then it can be nice and fast in SSE2.

I've been giving it some thought and I think separate timelines for each pane will make things much easier, but it'll take some time to implement. StainlessS, I don't mind PMing you a link for a sneak peak if you want to try it as it is now, but you'll have to work it out for yourself!

[StainlessS]

PM, not necessary, I'm up to my eyeballs already, doing that other stuff (deshaker in pieces stuff) in Usage forum is just a bit of a distraction,
cos I'm a bit sick of the other stuff I'm doing.

But thanks muchly just the same.

[wonkey_monkey]

Oh that's good, because I'm a bit sick of programming this offset mode right now ;) But I will tidy it up for the eventual release.

[StainlessS]

No sweat Wonky, take as long as you like (which I know you would anyways, without my permission). :)