How to do Picture in Picture (PIP)?

[DDogg]

A question was asked in the NLE forum about PIP and I would like to help out and check if anybody has done this in Avisynth. I tried searching "Picture in Picture" and got 281 hits, none of which seemed to be about that subject.

http://forum.doom9.org/showthread.php?s=&postid=314127#post314127

I would also be very interested in how to take two sources, zoom down and position one as an PIP overlay on the other. Does anybody know how to do this via script or whether it is even possible?

WarpE did bring up the need for a Pan and Scan filter which I think a PIP function would be part of, but that topic went dead. Seems like a worthy candidate for some programmer to consider.

[Guest]

It's easy, but my script is at home. I'll post it from there tonight if no-one else gives it to you before then. :)

[matrix]

Wouldn't layer do that?

main clip
Layer(pip_clip.bicubicresize(x,y),"add",255,x_pos,y_pos)

[DDogg]

Yeah, easy for you :) I would really appreciate seeing it. Thanks, Donald.

[Guest]

Our friend matrix is quite right. Here is a simple script:

main=avisource("tango.avi").ConvertToYUY2()
pip=avisource("spider.avi").bicubicresize(120,80)
Layer(main,pip,"add",255,20,20)

You need to get the clips into the same color space and so use convert calls as needed. I needed one for tango. Resize the PIP as needed and use the last two Layer() parameters to position the PIP.

I think it qualifies as easy. :)

[DDogg]

Thanks to both of you. Matrix' example got me pointed in the right way, and I got something working ... but, well, let's just say it was a little ugly.

Donald, your's was more laid out in the way I think. (read:linear) and you are right, it is pretty easy to understand. Avisynth never ceases to amaze me with its flexibility and power.

Donald, aren't those fingers of yours just intching to do a Pan and Scan filter? :)

[Guest]

Originally posted by DDogg

Donald, aren't those fingers of yours just itching to do a Pan and Scan filter?Give me a succinct statement of requirements and typical applications and I will consider it. Is it something like my VirtualDub Pan filter and/or Zoom filter that you have in mind?

[DDogg]

I thought I would just get a "no" and maybe a smiley if I was lucky :) Sure, of course. I'll think on it and try to do a proper write-up for your consideration in the next day or so.

Also, anybody else that could use a pan and scan/zoom filter don't be bashful with input. It might be particularly synergistic now that Richard Berg is getting the image handling stuff put to bed.

Donald, some background here in the meantime (I see you have already commented): http://forum.doom9.org/showthread.php?s=&threadid=47974&highlight=ken+burns

[Guest]

He showed a script code for the effect. Why not post an AVI so we can *see* it? I mean, you guys post some long obscure script code and say "what do you think?" Post the effect as a finished clip! Duh. :p

I'm thinking you can do Pan and Scan with Layer and Animate. When I see your requirements I'll know whether that is the case. But don't forget the applications; there has to be a reason for doing things!

[WarpEnterprises]

Is something more meant than my Zoom filter?
(with it you can move, rotate and zoom and use the current frame number as a variable argument)

[mf]

Pan&Scan? Oo, that sounds like a filter that moves a 4:3 rectangle over the source material, depending on where the most motion is, and that adaptively!
It would kinda choke on calm conversations with a moving background though..

[DDogg]

WarpEnterprises, with all due respect, all I have gotten from Zoom is access violations in Avs 2.5x. Perhaps you remember I reported this to you twice. As I asked then, if you will give me a test line that you have verified working with 2.5x I will try again.

Alternately, if you could suggest a line that would take PicA.xxx (as you direct) and open, pan right 300 pixels and then zoom that section to full frame, I would appreciate it.

I do hope you know I am just having a little fun with you when I suggest your docs are more targeted at your peer group of video antennae heads than the average Joe user like I. :)

I assumed from the docs that some of the examples titled "try this" were standalone examples that could be cut and pasted into a script and ran without alteration. Those yielded the exception errors. Even this:

version().converttorgb32()
Zoom(angle="85")

or

Sounds complicated?
Not really, try this:

version().converttorgb32().zoom(dstx="n", angle="n/10")

Here you see that the variable "n" can be used to reference to the current frame number (and this is of course the most important thingie).

Or try:

version().converttorgb32()
zoom(factorx="n<41 ? pow((n+1)/41,0.5) : 1", factory="n<41 ? pow((n+1)/41,0.5) : 1", dstx="n<41 ? width/2 : width/2+5*(n-41)", angle="n<41 ? 3.1415*n/10 : 0")

or want to shake:
zoom(dsty="rand(10)+height/2", dstx="rand(10)+width/2")


And the new image gets a defined alpha layer (black=transparent, else opaque), so you can layer the result on top of another video.


Of course it can be used e.g. as plain rotate:

e.g. Zoom(angle=...) will do nothing but rotate your image.

[McQuaid]

One thing I found difficult to accomplish with animate was trying to have something start off extremely small and zoom out to encompass the whole screen and then zoom back to extremely small with a new image. I could do the image switching no problem it was just that animate lends itself to zooming in making something larger versus something becoming extremely small. To visualize an example of what I'm talking about, think about the batman symbol zooming in/out during scene changes in the old campy 60's series.

Also, something I thought would be possible at first with animate was trying to fix the squishing of credits at the end of shows. I couldn't accomplish this as animate doesn't allow something to have different resolution even though the end result was going to have the same resolution. Something like this would suffice to fix a simple squish. A more advanced plugin would have that feature like photoshop to specify a boxed area and flatten it to the screen even though the original portion was on an angle. The example I saw of this photoshop function was a picture of a building on an angle and 4 corners being specified and then it flattened that area(of a poster on the building). Probably out of scope for avisynth, but would be kinda handy sometimes. But for the most part I've given up on trying to salvage credits in this day an age.

[WarpEnterprises]

I really tested it and it worked.
Can we start once again? I put up a new 2.5version (pls download it from www.avisynth.org\~warpenterprises)

I tested it with AviSynth 2.51 build May 15 2003
with the following script:

LoadPlugin("c:\myprojects\zoom25\release\zoom.dll")
Colorbars(720,324) # "cinemascope" picture
source = AddBorders(0,126,0,126) #letterboxed

Pan = Zoom(source, srcx="360+n*16") # from frame 0 to frame10 it pans right by 160 pixel
ZoomIn = Zoom(source, srcx="360+160", factor="1+(n-10)/10") # zoom=1 at frame 10, it is fullscreen at frame 18
ZoomInFix = zoom(source, srcx="360+160", factor="1.8") #stay at fullscreen factor
return trim(pan,0,9) + trim(zoomin,10,18) + trim(zoomInFix,19,0)

As you see defining sections is quite uneasy, so I added a function

Spline(x, x1,y1, x2,y2, ..., true/false) (it is already in the AviSynth core of the current version)

It interpolates between given x1/y1 ... pairs and calculates the value at position x. "true" does this with smooth splines, "false" linearly (which is more suited to your example).
So you can do the same as above with

LoadPlugin("c:\myprojects\zoom25\release\zoom.dll")
Colorbars(720,324) # "cinemascope" picture
source = AddBorders(0,126,0,126) #letterboxed
Zoom(source, srcx="spline(n,0,360,10,360+160,11,360+160,false)", factor="spline(n,0,1,10,1,18,1.8,19,1.8,false)")

In other words, you only define some points of the values (typically on the borders of the sections).

I really cannot understand why it would crash.
At least Zoom() without any parameters should pass through the picture.

@McQuaid:
If I understand you right you want to take a corner of an image (e.g. the bottom left quadrant) and zoom it to full screen.

loadplugin("c:\myprojects\zoom25\release\zoom.dll")
version.converttorgb32.bilinearresize(440,80)
zoom(srcx="spline(n,0,220,10,110,11,110,false)", srcy="spline(n,0,40,10,60,11,60,false)", factor="spline(n,0,1,10,2,11,2,false)")

does this.

[McQuaid]

Thats the first method I tried. Taking a portion of the screen and zooming it to full screen(I used the middle of the screen). The problem with this is that small portion suffers from its low resolution when occupying the full screen.

Think about it this way, using my batman example again. Say in a 640x480 image we had the batman symbol occupying only say 64x48 pixels. Zooming that center portion would look awful once it reaches the 640x480 res. What I would want is to have the bat symbol drawn at 640 x 480 and then by able to zoom out (in essence shrink it) to 64x48 and then be able to do the reverse. I started trying to use ridculous resolutions like 4000x3000 to have it only occupy a portion of the screen with limited success.

[DDogg]

WarpE, it sounds like you have done much extra work to help solve this problem so I am very bummed (for you) to report that:

loadplugin("D:\zoom.dll")
version().converttorgb32()
#Zoom()

Works ok, but,

loadplugin("D:\zoom.dll")
version().converttorgb32()
Zoom()

Throws an exception error. (caught an access violation at 0x77e7650d,attempting to read from 0x00a7e524)

Perhaps somebody with XP and Avisynth 2.51 (May 15) could run the second example (or what you instruct) and verify if this is system specific or a general fault. My machine has been behaving a little odd lately so we need to check and get an independant verification of the problem.

Edit: this example also threw the error:

LoadPlugin("d:\zoom.dll")
Colorbars(720,324) # "cinemascope" picture
source = AddBorders(0,126,0,126) #letterboxed
Pan = Zoom(source, srcx="360+n*16") # from frame 0 to frame10 it pans right by 160 pixel
ZoomIn = Zoom(source, srcx="360+160", factor="1+(n-10)/10") # zoom=1 at frame 10, it is fullscreen at frame 18
ZoomInFix = zoom(source, srcx="360+160", factor="1.8") #stay at fullscreen factor
return trim(pan,0,9) + trim(zoomin,10,18) + trim(zoomInFix,19,0)

[Si]

@DDogg
Slight deviation but maybe helpful?
I'm getting exceptions using plugins in RGB colourspace.
Could you try this script and see what happens.

LoadPlugin("MSharpen.dll")
AVISource("whatever")
ConvertToRGB32()
MSharpen()
return last


Just in case you've got the same problem as me.

regards
Simon

[WarpEnterprises]

just tested on a different W2K machine - works ok.
(the other test was on W98).

Another guess: make sure you really have only one version of zoom (maybe there is a second, old one in the plugin dir or else?)

Please, can someone test

loadplugin("D:\zoom.dll")
version().converttorgb32()
Zoom()

on XP ?

OT: what exactly means "bummed"? I can't find it e.g. here (http://dict.leo.org/)


@DDogg: you are right, the doc is really bad. If it seems useful to some people I will go to more detail. It's a little cumbersome in principle to define the parameters, but a friend of mine made a whole 20min-movie out of still photographs using Zoom (and Premiere for other blend effects)

Could you eventually test the 2.08 version, too?

[manono]

Not to put words in DDogg's mouth, but bummed=sad and/or disappointed.

[DDogg]

@ siwalters - Good thought, but your example works OK on my machine (XP SP1 AMD)

@WarpEnterprises -
1> No other zoom.dll on machine. I also renamed the plugin dir in case something in it was causing a problem like we saw with warpsharp.dll.
2> Sorry for old hippy slang :) Yes, it is as manono said.
3> No, I do not think docs are bad. They are full of good information. They just written by somebody who is a good programmer and we all know good programmers can not write good docs (except Donald) :D

I would be happy to help add a few things to the docs that might help less experienced people when we find out what the problem is. I do hope this is not just specific to my machine because this is causing you extra work.

4> Yes, I will try 2.08 today and report back.

@Anybody with XP, as WarpE asked:

Please, can someone test

loadplugin("D:\zoom.dll")
version().converttorgb32()
Zoom()

[Valky]

This was fun topic. This script works great for me:

main=avisource("e:\nutty.avi").ConvertToYUY2()
pip=avisource("e:\nutty2.avi").bicubicresize(120,80).ConvertToYUY2()
Layer(main,pip,"add",255,5,5)


but how to keep audio in already encoded material? VdubMod gives me acm and pcm error when trying to open this script if the original movie nutty.avi has still it's own mp3-track and nutty2 has been saved without audio?



Btw, I also tried to make a clip what could show two scenes in one picture with this layer function but didn't succeed.
Example if I wanna show to my friend a clip with two different angle in the same sample-clip.
If my original movies are 640x352 and I cut half of the resolution in both clips so that my final resolution in this clip would be 320x352 (the clips are then vertically in stack). How to do that?

Best I got was this script:

main=avisource("e:\nutty.avi").bicubicresize(320,176).ConvertToYUY2()
pip=avisource("e:\nutty1.avi").bicubicresize(320,176).ConvertToYUY2()
Layer(main,pip,"add",255,0,88).bicubicresize(320,352)

but tis isnt good..the final resolution is just what I was trying, but both movies also lack half of the picture...

[Pontus]

I tested the following under XP, as per requested, (and then with parameters to Zoom too) and it worked fine.

LoadPlugin("E:\PROGRA~1\Video\GORDIA~1\zoom.dll")
version().converttorgb32()
Zoom()

[vinetu]

Please, can someone test

loadplugin("D:\zoom.dll")
version().converttorgb32()
Zoom()

on XP ?


Here work OK:
P4 1.8GHz@2.8GHz ,WinXP.
Both AviSynth Legacy 2.0.8(zoom_20_dll_20030404.zip)
and AviSynth 2.51(RC 3.1)-zoom_25_dll_20030402.zip ...

...Cheers!

[WarpEnterprises]

Thanks, then at least the error is not straightforward :)

@valky: what about StackVertical?

[DDogg]

Also, Pontus is AMD so that does not seem to be it. Drat!

Well the good news is is does indeed work using Avisynth 2.08. I really do not know where to go now unless you can furnish some type of debugging tool to gather information for you. As for me, everything is set up for 2.5x and I see 2.08 as obsolete with no future so that is not an option (for me).

@Valky, here is something Matrix posted in NLE that you may find interesting.
http://forum.doom9.org/showthread.php?s=&postid=314799#post314799

[DDogg]

WarpE, sorry I caused you extra work. I have double verified the problem I am having is specific to this one machine. I set up my kids XP AMD machine with AVS 2.51 and zoom.dll and it worked fine.

I will try to figure out what in the heck is causing this. In my defense, it is odd because this is the only filter effected. AVS 2.08 and zoom.dll works, it is only when AVS 2.51 and zoom.dll are used where this specific machine has the problem.

Does anybody have any ideas where I should start checking first? Any suggestions would be very much appreciated as this is a little like looking for a needle in a haystack.

[WarpEnterprises]

no problem, DDogg.

There is nothing special in my DLL.
There is little code, only two things are used not so often by other authors:

memset(...);
sin(...);


If you want to play around with me I will make versions with less and less code and then we see when the crash occurs no more.
It should be easy possible as at least the crash occurs ALWAYS.

Is the crash already on open in VD or only if one frame is to be displayed?

[DDogg]

Well, this may turn out to be simple. Like a lot of folks I use WMP 6.4 when testing any avs script. When I rendered my zoomtest.avs in graphedit, surprise, it played. So I tried it in MPC and it played [as well as in VDub]. SO, now my problem is down to why in the heck this specific filter crashes in WMP 6.4 [just on my machine] but works with other players. This is nuts.

Oh, to your questions, No frame is displayed. Black on red only in WMP. I am very angry at myself for not trying this in VDub before saying anything. I know better :(

[Valky]

Warp, could you give me a little demonstration script of this Stackvertical?

Let's say my original clips are 640x352 and I wanna half resolution of those and stack them together?

[matrix]

@Valky

main=avisource("e:\nutty.avi").bicubicresize(320,176).ConvertToYUY2()
pip=avisource("e:\nutty1.avi").bicubicresize(320,176).ConvertToYUY2()
Layer(main,pip,"add",255,0,88).bicubicresize(320,352)

You don't even need layer there (if I understand correctly)
Just stack them vertically. Instead of layer, use:

StackVertical(main,pip)

[Valky]

thank you very much! This was just what I was looking for when I wanted to compare two movies. Other one normal version and the second strongly uncut :)
And no need to Layer here.

[MrTVideo]

clip1 = AviSource("D:\Vid 1.avi").Flipvertical
clip2 = AviSource("D:\Vid 2.avi").Flipvertical.BilinearResize(240,180)

Overlay(clip1, clip2, x=0,y=100,Mode="blend")

ConditionalReader("opacity.txt", "ol_opacity_offset", false)
ConditionalReader("Xoffset.txt","ol_x_offset", false)
ConditionalReader("Yoffset.txt","ol_y_offset", false)

This is one I am using for one GUI I am creating. This is a file writen by the GUI. The flipvideo is because the avi capture in my case is by way of a DC10 card. I am gradually getting a handle on the AVISynth>>>Virtualdubmod combination and I will create a GUI for simular editing features. I am looking for critiques so if you would try the GUI I created at my web site I would appreciate any comment.

http://mrtvideo.com Sorry this one does not do pic in pic that will be the first I will sell. Is this mercenary of me, Of course but I am retired and want to buy a tv projector. I know that this script is simplistic but there are people out there who have trouble geting basic things working thus the GUI approach. There are approx. 300 downloads/day at present so someone is trying it.

Actually this script allows the layered video to follow a moving path you have to provide the necessary co ordinate files

Edited:-
WoW! Site now up again. I would appreciate comments as I said. Give me a caneing if you would. Better than a customer left dissatisfied or worse disillusioned with their AVIsynth/Virtualdubmod experience.