Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VirtualDub, VDubMod & AviDemux

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th October 2019, 14:49   #1  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Virtualdub2 "merge layers" filter?

How does the "merge layers" filter work?

I've been fiddling around with VirtualDub2, and one of the new features (from VD 1.9.11 at least) is the "merge layers" filter.
The description within VirtualDub says "Blends two layers together using a third layer", but how?

It doesn't seem to come with a readme, and I haven't found any documentation for this in Help, the VirtualDub2 Wiki, this forum, the VideoHelp forum, or just googling it. (It's also a somewhat hopeless search term to start with.)


As far as I know, you would select layers the following way:
Add the "merge layers" filter
Right click
Add Input
Right click input ("No connection")
Choose between "No connection", "Source", or "Previous output".

That's two inputs, and only one of those could be considered a "layer".
The only options within the filter is a choice between the blending modes Lerp or Select. And it doesn't say what they actually are. I haven't found anywhere it says what Lerp actually means.
"Select" seems like an obvious place to select layers, but neither "Select" nor "Lerp" seem to actually do anything.
Having three other blended filters doesn't seem to make any difference.

This question also applies to the "blend layers" filter, which has two inputs, but the only two possible inputs seem to be "Source" and "Previous output", which seems like only one "layer".

How does the "merge layers" filter work?
Have I missed some obvious place where this filter and how to use it is described?
Goggen240 is offline   Reply With Quote
Old 4th October 2019, 16:26   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Dont know if any of this may point you in right direction.

http://forum.doom9.org/showthread.ph...56#post1875156

https://sourceforge.net/p/vdfiltermod/wiki/Home/

good luck.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 4th October 2019, 19:33   #3  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Quote:
Originally Posted by StainlessS View Post
Dont know if any of this may point you in right direction.

http://forum.doom9.org/showthread.ph...56#post1875156

https://sourceforge.net/p/vdfiltermod/wiki/Home/

good luck.
No, I'm afraid not.
The post/thread is about bugs in filter blending for inputs from AviSynth and fflayer (and how to turn on "Swap input/output panes").
Merge layers, blend layers and fflayer do different things; fflayer loads and overlays an image or video.

"fflayer": Overlays an image or video on top of "Previous output" or "Source". I think you can choose, if so it's right-clicking and selecting Add Input. Using the Curve Editor you could also blend fflayer however you want.
I've been using an older VideoOverlay filter for that for years, and if it's images I just use Logo. But it's nice you can set the fflayer overlay to a specific opacity (50%), rather than eyeballing what the Curve Editor is set to.
"blend layers": Blends two filters together, but they don't have to be a separate image/video file like fflayer, should work with any filter. It says it blends two layers, but I don't know how to actually do that. I've only been able to blend "Source" and "Previous output", and I don't think "Source" counts as a layer.

"merge layers": Blends two layers using a third layer. I *assume* this could mean a mask, that's what I'm trying to use it for at least, but I have no idea how to load three inputs, I've only gotten to one (Previous output) and that doesn't do anything.

The project I'm tinkering with is to try and fix a video with messed up deinterlacing; the fields are alternating two by two rather than one by one. I've gotten some of the way there with VideoOverlay and dozens and dozens of intermediate files (field by field!), but it would be very nice to be able to use merge layers to blend a complicated filter stack through a mask for that.
If that's what merge layers is supposed to do...?
I'm hoping to extend the principle to files that have even worse messed up deinterlacing; scaling from 576i to 480i has a (roughly) 34 line pattern of alternating TFF-blend-BFF-blend-TFF, and that's real bad. I have a concert video that's been scaled from 1080i50 to 720p, 540p, 360p and 270p; it should be possible to reconstruct most of a 1080p50 file by blending that mess together.

Trying to do that is my white whale, heh.

Last edited by Goggen240; 4th October 2019 at 19:40.
Goggen240 is offline   Reply With Quote
Old 4th October 2019, 20:49   #4  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Example using blend layers:



You need to use "name output" to add options for input menu.
I'm not 99% sure "merge layers" does blend between input0 and input1 using input2 as blending factor in "lerp" mode, and that blending factor is threshold in "select" mode.

Btw this was actually implemented in original 1.10 and IMO this whole idea only reached prototype - very difficult to use in practice.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 5th October 2019, 01:40   #5  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Quote:
Originally Posted by shekh View Post
Example using blend layers:



You need to use "name output" to add options for input menu.
I'm not 99% sure "merge layers" does blend between input0 and input1 using input2 as blending factor in "lerp" mode, and that blending factor is threshold in "select" mode.

Btw this was actually implemented in original 1.10 and IMO this whole idea only reached prototype - very difficult to use in practice.

This works!
(Well, it's a flickery mess in a hopelessly complicated filter stack, but it's a horribly complicated blended flickery mess!)
And yes, the third layer does actually work as a mask.

See attachment


I've been using 1.9.11 for about 10 years, so a lot of the newer functions (Add Input) are new to me. They also do seem a bit poorly documented, unfortunately.
But thankfully this forum has helpful people like you!

What the heck I'm doing:
I have a 1280x720p25 file that was downscaled incorrectly from 1080i50:
Line 1: Field 2
Line 2: Field 2
Line 3: Field 1
Line 4: Field 1
And so on.
Obviously this can't be deinterlaced in this state.
So, step by step, the filter stack does this:
Step 1: Drop even lines, for step 3.
Step 2: Drop odd lines, for step 3.
Step 3: Combine both top lines to one field (odd lines) and both bottom lines to bottom field (even lines) at half resolution. (A simple downscale would blend all four lines together a little, which is worse.
Step 4: Deinterlace the half-resolution "reinterlaced" file, 2xYADIF (1280x360p50)
Step 5: Upscale to 1280x720p50
Step 6: Null Transform with an Output Name, for troubleshooting, not strictly necessary
Step 7: Load a separately made mask file for the merge layer filter, RGB24 uncompressed AVI, which is a 100GB 1280x720p50 video of alternating black lines, like so:
Odd frames:
White
White
Black
Black
Even frames:
Black
Black
White
White
This allows the appropriate original field lines to be blended on top of the downscaled deinterlaced video. That is: Odd frames; lines 3 and 4, even frames; line 1 and 2.
Step 8: I invert the mask because it's the wrong way around and I'm not sure why and it's late and I'm tired
Step 9: Null Transform with an Output Name, for troubleshooting, not strictly necessary
Step 10: I double the framerate of the original video from 720p25 to 720p50 so the framerate matches the new deinterlaced video, simply repeating each frame twice.
Step 11: Null Transform with an Output Name, for troubleshooting, not strictly necessary
Step 12: Finally, I merge the layers together:
First layer: The frame doubled video from step 10
Second layer: The upscaled (step 5) deinterlaced (step 1-4) video
Third layer: The mask that allows the original lines to be overlayed over the interpolated lines, where there are original lines to be had.

A lot of the filters (null transforms) and filter names/inputs are unnecessary so it can be streamlined a bunch.

I'm pretty sure flickeriness is happening in the downscale/upscale process; at half resolution it's looking at 3 vertical pixels for upscaling, which is actually 6 pixels because of the line doubling mess. Nearest neighbor doesn't look at enough pixels, though. Hmmm...
Or I messed up the mask. Or the deinterlace order. Or something else. I'll figure it out eventually!

I'm going to try to find the time to document some more of my deinterlacing experiments (and improve this one) so future generations of video restorers have a reference, now that I think I know how.
In case somebody else wants to rescue old messed up files, hopefully I'll be helpful too!
Attached Images
 

Last edited by Goggen240; 6th October 2019 at 00:38. Reason: more detail
Goggen240 is offline   Reply With Quote
Old 5th October 2019, 13:59   #6  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
The file I'm talking about was downscaled from 1080i to 720p causing a BFF-TFF pattern that repeats every 4 lines, but here's a better (double) example to illustrate:
Upper left: downscaled from 576i to 540p, causing blend-BFF-blend-TFF-blend pattern that repeats every 34 lines
Lower left: failed deinterlacing (wavy edges every 34 lines)
Upper right: upscaled from 576i to 720p, causing blend-BFF-blend-TFF-blend pattern that repeats every 10 lines
Lower right: failed deinterlacing (wavy edges every 10 lines)

Combining the two should recover most of the field information for 86% of the lines if my math is correct, the rest would havebe unrecoverable overlapping blends. I think.

Attached Images
 

Last edited by Goggen240; 6th October 2019 at 00:41. Reason: added picture
Goggen240 is offline   Reply With Quote
Old 5th October 2019, 18:18   #7  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Goggen240,
attachments are hidden for users. Please reupload them to https://postimages.org/
However I don't even try to understand the problem, it looks too complex. I honestly recommend to try solving it with avisynth (also probably there is already some filter or script for your needs)
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 6th October 2019, 00:50   #8  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Quote:
Originally Posted by shekh View Post
Goggen240,
attachments are hidden for users. Please reupload them to https://postimages.org/
However I don't even try to understand the problem, it looks too complex. I honestly recommend to try solving it with avisynth (also probably there is already some filter or script for your needs)
Oh it's definitely too complex, but it's a known problem that happens often, whenever anyone rescales interlaced material without deinterlacing first.
As far as I know it's just not a solved problem, so I don't think there's a script for it. I've certainly never found any.
Also I'm really not very good at AviSynth, I need to see it to solve the problem, and text-only video editing isn't a good fit for me...
So thank you for VirtualDub2!

And don't worry, I don't think I need much more help with it for the time being, now that I (you) figured out blending and masking. I've got enough to tinker with to last me a few months, heh.
Thanks again!

Last edited by Goggen240; 6th October 2019 at 00:52.
Goggen240 is offline   Reply With Quote
Old 6th October 2019, 04:16   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Goggen240, you do know that Vdub2 has an excellent avs script editor[tools menu], yeh ?, just
press F5 key for instant refresh of script to display, easy peasy.

EDIT: or some people use avsPmod, cant be much different to using vdub.

EDIT: Here an avs thread on your prob,
Undo interlaced resizing mess :- https://forum.doom9.org/showthread.p...ght=legged+dog

JohnMeyer was kinda lucky that ExBlend() worked in that particular case, it was not intended for such problems, but kinda accidentally worked ok.
There was a thread and script posted in Avisynth Usage forum, within last 12 months I think, particularly for this problem, dont know where it is though,
maybe someone else remembers. Post in Avs Usage, maybe.
Just noticed the 576i thing, so maybe ExBlend a total bum steer (is for NTSC), but I still think there was a script posted not so long ago
for pure interlaced downscale fix (but was maybe specifically for reversing Bilinear downsize only).
EDIT: There seems to be a script (DeBilinear) for reversing interlaced upsizing, but of course you want the reverse.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 6th October 2019 at 14:31.
StainlessS is offline   Reply With Quote
Old 6th October 2019, 22:30   #10  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Quote:
Originally Posted by StainlessS View Post
Goggen240, you do know that Vdub2 has an excellent avs script editor[tools menu], yeh ?, just
press F5 key for instant refresh of script to display, easy peasy.

EDIT: or some people use avsPmod, cant be much different to using vdub.
I didn't know that, or at least I never thought to look. Could be helpful.
Those times I've used AviSynth I've just used Notepad or even command line; I grew up with DOS so it's almost nostalgic.
The problem I have with AviSynth is that I'm not a coder, can't write code, and can barely read AviSynth scripts. So it can get pretty impenetrable pretty quick. I know it's great at what it does (and probably what I'm trying to do), but the analogy is that it's better to use a hammer you know how to use than a nail gun you don't.
And I've gotten pretty good at using VirtualDub for nail gun jobs...

Quote:
Originally Posted by StainlessS View Post
EDIT: Here an avs thread on your prob,
Undo interlaced resizing mess :- https://forum.doom9.org/showthread.p...ght=legged+dog

JohnMeyer was kinda lucky that ExBlend() worked in that particular case, it was not intended for such problems, but kinda accidentally worked ok.
There was a thread and script posted in Avisynth Usage forum, within last 12 months I think, particularly for this problem, dont know where it is though,
maybe someone else remembers. Post in Avs Usage, maybe.
Just noticed the 576i thing, so maybe ExBlend a total bum steer (is for NTSC),(...)
From the description that's improper telecine/IVTC, so you're right that it's not quite the same. Here there's too much field information, so the problem is to get rid of the right (well, wrong) stuff. I don't quite understand the scripts they come up with, but I'm seeing a lot of Decimate and Blend, and for 24FPS trapped in 59,94Hz that's probably appropriate.
I suppose the principle would also apply for other similar issues (5:6 pulldown), but for straight PAL or NTSC rescaled improperly it doesn't quite fit. I did find some other ones though.

I've been googling this once in a while for years so I've been through a lot of this kind of thread, heh.
One big problem now is that all the example pictures and video are long dead, unfortunately. So it's a bit difficult to find out exactly what they were trying to fix.
I didn't find the new one either, but at least I'm having fun trying to work out the problem for myself.
Learning is fun.

But anyway:

Sequel thread:
https://forum.doom9.org/showthread.p...61#post1531961
More detail on how they ended up with the messed up telecine.
One point that comes up is that the field issues happen irregularly due to timing issues, so there was an issue of how to identify "broken" frames from the rest.
That pretty much never applies to "normal" messed up interlaced videos, which would be perfectly regular. Usually. There are a lot of ways to mess up video...

Another old thread that's on to something.
Sounds like my second examples, where it switches TFF/BFF, blend, BFF/TFF, blend, and so on.
https://forum.doom9.org/showthread.p...96#post1492396
Their suggestion: QTGMC Deinterlace/FrameDouble-->Resize-->Reinterlace
Kinda what I'm going for, but this sorta blurs the heck out of the video until the problem is gone. Also I don't know why the resize is *before* the reinterlace? Wouldn't that be bad?
I've had more luck just blending the fields, but I've been fiddling with using a similar technique to theirs:
Deinterlace 2x Top-Field-First -> Deinterlace 2x Bottom-Field-First -> Blend 'em together
(Blurry, and you lose half the motion information, but simple)
As far as I can tell, a simple blend is less... Stable? It's trying to blend odd and even lines always, but when the field order swaps it goes off a little. The extra blur from blending 2x filters seems to cancel that out. I think.
I'm probably wrong and blending is just better.

Followup to that first thread, this time for a video with completely separated fields every 18/18 lines (seems to be nearest-neighbor rescaled before deinterlacing).
https://forum.doom9.org/showthread.php?t=170813
They seemed to arrive at a solution of motion-interpolating the missing 9 lines and masking it all together. And then blending the two fields, because that's how it was done.
I'm trying to do that (except 2x), but in one file for 2x2 and not for 18/18 lines (which is a horrible nightmare). Although some files with messed up interlacing actually have messed up deinterlacing, like the wavy edges example I made. Those could be kinda repaired with that sort of approach with the right kind of mask, so I've been thinking about that. (I've seen that thread before.)

Pretty new thread:
https://forum.doom9.org/showthread.php?t=174507
A more general approach, but this also ends up blending the fields, rather than separating them and recovering (some) motion information. My suggestion before, "just blend 'em", should work much the same.

A new and suprisingly hostile one:
http://www.digitalfaq.com/forum/vide...ded-video.html
They settle on giving up. Pretty much.
They do hypothesize a lot of the solutions I'm trying to put into practice.

Quote:
Originally Posted by StainlessS View Post
(...)but I still think there was a script posted not so long ago
for pure interlaced downscale fix (but was maybe specifically for reversing Bilinear downsize only).
EDIT: There seems to be a script (DeBilinear) for reversing interlaced upsizing, but of course you want the reverse.
EDIT: Found it, this seems to be a AVISynth script package that includes DeBilinear and a bunch of others (BiCubic, Lanczos, etc.). I don't know how to use it.
https://github.com/jpsdr/ResampleMT
Also found some kind of... Thing... I think... That analyzes frames/pictures to find the original resolution. It's in Python. I really don't know how to use it.
https://github.com/Infiziert90/getnative
Like I mentioned before, since I'm not a programmer, documentation written by programmers for programmers is pretty much impenetrable to me.
I've been messing around with ResampleMT and the example script included, and in a couple hours I've gotten it from access violation on line 1 to access violation on line 6. So that's progress.

Oh, this looks real interesting!
Pretty much all analog video is 704x480 or 716x576, so recovering data from an upscale is really cool.
In fact, the example I posted has both:
The script (apparently) used to compress and post the videos downscales to a bunch of different resolutions; 180p, 270p, 360p, and 540p. But the videos also get upscaled to 720p!
So I'll play around with that script once I have time and figure out how.

Yeah, recovering 576i from 720p is a lot more promising...
My original plan was to blend 720x540 and 960x720 together, where they would have to be upscaled to the same resolution. The resolutions actually line up at UHD 2880x2160, which is a really dumb size for a 1967 broadcast in 576i...

One still unsolved problem that I might fumble my way towards is 544p; in the early days of digitization, they had one job: Scale 720x576 up to 768x576, and 720x480 down to 640x480. To get square pixels.
...Or what they often ended up doing, which was scaling down to 720x544 and call it a day, since it ended up 4:3!
(Actually 1.32:1, but at least they got close...)
So lots of interlacing issues at that exact resolution. And a bunch of other resolutions too of course. (512x384 was real popular too.)

Lastly, the part where I'm being difficult is that I'm trying to recover motion information as well, like with YADIF/QTGMC 2x deinterlacing, and not just blend the fields.
Naturally I collect files with messed up interlacing issues just for the fun of trying to fix 'em. Who wouldn't?
(And if they aren't available in other forms. Usually that one.)

Back in them good old days the goal of deinterlacing was always to blend the fields, or discard the other field as "nicely" as possible. 25/30 frames per second was the goal.
But I'm all for deinterlacing 2x. You can leave the fields/lines you have untouched, interpolate what you don't have, and get 50/60 FPS instead of 25/30. For now at least it's as close to original you can get, and since it doesn't really touch the original lines, it should be fairly future proof as well.

Doesn't mean it looks better though, heh.

Last edited by Goggen240; 7th October 2019 at 12:26.
Goggen240 is offline   Reply With Quote
Old 7th October 2019, 21:44   #11  |  Link
Goggen240
Registered User
 
Join Date: Sep 2019
Posts: 14
Problem solved.

https://postimg.cc/cvLGBZD1

That was easy.
Huh.

Upper left side is the downloaded 1280x720 version, and bottom what deinterlaced looks like (2x). Lots of wavy edges.
Upper right side is the DeCubicResized 1024x572 version, and bottom what deinterlaced looks like (2x). Looks fine!

You'll need JPSDR's DeBicubicResizeMT plugin for AviSynth+
https://forum.doom9.org/showthread.php?t=174248

And here's the script that works for a video that was messed up by scaling up from 1024x572i50 to 1280x720p25 without deinterlacing first:

Quote:
#Input file
DirectShowSource("G:\720p.mp4")

#Reverse Upscale
DeBicubicResizeMT(1024, 572)
More info:
The file was cropped slightly to get rid of the half lines top and bottom, it should have been 576 lines. Also, there are a few crossfades that are out-of-sync top-to-bottom, but I assume that 1967 live broadcast video technology might have caused that.

Now, this is not a solution for videos that are downscaled, such as 576 lines to 540 lines like in the other example picture earlier.
So I'll keep tinkering along on those.
Goggen240 is offline   Reply With Quote
Old 8th October 2019, 11:35   #12  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
I had been reading your LONG-ish post but was falling asleep at the time and could not 'take it in', and so left any response till I was awake.
I see now that you have it sorted out, good job, looks great fix. [looks to be ResampleMT/Desample: 2.2.2 that has DeCubicResize()].
I fixed a similar clip a few years ago, but have no idea how I did it and did not keep copy of the script, I'm sure that I still have at least one
clip that could benefit from ResampleMT/Desample, so thanks to both you and Jpsdr for this.

If JohnMeyer sees this, take a look at his images, he did a fine job.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 9th October 2019, 02:30   #13  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
StainlessS PM'd me.

The thread he linked to above was a different problem than what is being discussed here. The one I'm sure he remembers is this one:

Repair Bad Deinterlacing

We worked on this together and, as you will see, came up with some pretty decent results.

I spent a few minutes replacing the defunct Photobucket images so the thread makes a little more sense.

I have no idea how the quality of what I did would compare to the approach here. One thing for sure: this approach is 100x simpler.
johnmeyer is offline   Reply With Quote
Old 9th October 2019, 10:16   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Thanks john, I had originally posted link to that thread too, I musta removed it at some point.
I also restored a couple of missing PostImage.org images yesterday.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Reply

Tags
blend layers, layers, merge layers, virtualdub, virtualdub2

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.