Any way to clean up this splice script?

[Bexley]

Greetings. I'm continuing the project I posted about here a couple of years ago. I'm still working with the same material because I stopped the project due to time constraints, but I've picked it back up now. My issue here is different, though.

My solution I eventually came up with is to just do the FPS conversion twice with 2 intermediate files, one for video and one for film, and then splice them back together manually. My current splice script is a real mess, and I'm wondering if there isn't an easier way to do this. It looks like this:

p=AVISource("film.avi")
i=AVISource("video.avi")
scene1=Trim(p,0,950)
scene2=Trim(i,951,1500)
scene3=Trim(p,1501,2000)
scene4=Trim(i,2001,0)
AlignedSplice(scene1,scene2,scene3,scene4)

Except there are something like 50 or more transitions per episode, and this script gets to be very cumbersome pretty fast. In addition, I'm spending a ridiculous amount of time trying to chase down stupid, careless mistakes in typing in the frame numbers. It's making this project take a lot longer than it really should.

Is there an easier way splice these back together with less human error?

[manono]

Assuming one is used more often than the other (Film.avi in this example), then just replace the parts needing replacing like so:

p=AVISource("film.avi")
i=AVISource("video.avi")

ReplaceFramesSimple(p,i,Mappings="[3951 1500] [2001 0] ")

ReplaceFramesSimple is part of stickboy's RemapFrames.

I'm assuming both have the same framerate and resolution. I'm purposely leaving out any audio. Extract it and add it back later on. Or maybe the script will work with audio but I don't know as I never include audio in my video projects.

[blaze077]

Just a note: the range [2001 0] will not work with ReplaceFramesSimple as it needs the exact frame numbers for the frame ranges. That range will throw an out-of-bounds error. You would have to replace 0 with whatever the last frame number of your video clip is.

[Groucho2004]

There is an updated version of RemapFrames by cretindesalpes.
I built 32 and 64 versions with this latest code. You can find it here.

[Bexley]

Originally Posted by manono

Assuming one is used more often than the other (Film.avi in this example), then just replace the parts needing replacing like so:

p=AVISource("film.avi")
i=AVISource("video.avi")

ReplaceFramesSimple(p,i,Mappings="[3951 1500] [2001 0] ")

ReplaceFramesSimple is part of stickboy's RemapFrames.

I'm assuming both have the same framerate and resolution. I'm purposely leaving out any audio. Extract it and add it back later on. Or maybe the script will work with audio but I don't know as I never include audio in my video projects.

I'll give that a try, but just at a glance it doesn't look like it's going to be much less awkward. Yes, both sources are exactly the same frame rate and resolution after conversion. Unfortunately, this particular show is about 50/50 so there isn't really one that dominates.

What I was kind of hoping for is some sort of function where I could plug in a list of splice points (maybe using the last frame of each segment) and use an n+1 function to start a new segment at the next frame without having to manually put in every beginning and end frame. That would eliminate a lot of time and mistakes from keying in so many frame numbers. Then maybe a ++ syntax to join them instead of the clunky AlignedSplice() that introduces more time and more chance of mistakes when it gets really long. If I knew enough about scripting I'm sure I could write something like this, but I'm just not that good.

If I were only doing a few sources I wouldn't worry too much about it, but I have hundreds of hours of material to convert this way, so any time savings is a big deal.

[burfadel]

Cut and encode in Staxrip. Make the first cut point, name the output file A_1.mkv (mkv for a reason), click next, change the _1 to _2, delete current cut point, make new one, proceed through the file. Repeat for the next clip, but call them 'B'. Encode, then in the MKVToolnix GUI, open A_1.mkv, append B_1, append A_2 to that, and so on.

[StainlessS]

Or use ClipClop in place of RemapFramesSimple

p=AVISource("film.avi")
i=AVISource("video.avi")
SCMD="
    1 951,1500  # scene 2   # clip 1 (i), frames 951 -> 1500
    1 2001,0    # scene 4   # clip 1 (i), frames 2001 -> last frame
"
SHOW=True
ClipClop(p,i,scmd=SCMD,show=SHOW) # replace frames/ranges in p (clip 0) with those in i (clip 1).
#ClipClop supports up to 255 replacement clips:- http://forum.doom9.org/showthread.php?t=162266

ReplaceFrameSimple will do just as well.

EDIT: Or alternative Sawbones/FrameSurgeon

Import("FrameSurgeon.avs")

src=AVISource("film.avi")
FX1=AVISource("video.avi")
FX2=src.FFT3DFilter(Sigma=2.0)  # Some additional mod (clip 2)
FX3=src.FFT3DFilter(Sigma=4.0)  # Some additional mod (clip 3)

SCMD="
    FX1 951,1500     # Replace src 951->1500 with same range from FX1 clip.
    FX2 1600         # Replace src frame 1600 with same frame from FX2 clip
    FX3 1700,1705    # Replace src frames 1700->1705 with same frames from FX3 clip
    CP  1800         # CopyFromPrevious frame 1800 (replace src frame 1800 with src frame 1799)
    CN  1900         # CopyFromNext frame 1900 (replace src frame 1900 with src frame 1901)
    I1  1920         # Interpolate frame 1920, using as source frames src 1919 and 1921.
    I3  1930         # Interpolate 3 frames 1930->1932, using as source frames src 1929 and 1933.
    FX1 2001,0       # Replace src 2001->Last frame with same range from FX1 clip.
"
SHOW=True
src.FrameSurgeon(SCmd=SCMD,Show=SHOW,FX1=FX1,FX2=FX2,FX3=FX3)

Does frame interpolation of up to 9 frames.
SawBones/FrameSurgeon:- http://forum.doom9.org/showthread.php?t=173158
EDIT: Commands above other than FX1, are demo only, and not needed in your case.

[manono]

Originally Posted by blaze077

Just a note: the range [2001 0] will not work with ReplaceFramesSimple...

Yes, of course you're right. Thank you for the correction.

[2001 last frame number]

[raffriff42]

Originally Posted by Bexley

What I was kind of hoping for is some sort of function where I could plug in a list of splice points (maybe using the last frame of each segment) and use an n+1 function to start a new segment at the next frame without having to manually put in every beginning and end frame.

I like to use a spreadsheet for this sort of thing:

(note column D is calculated from previous row, column F)

When pasted into the script, it looks like this (adding a BlankClip head and tail)

BlankClip(p, length=15)
\++Trim(	p	,	0	,	950	)
\++Trim(	I	,	951	,	1500	)
\++Trim(	p	,	1501	,	2000	)
\++Trim(	I	,	2001	,	0	)
\++BlankClip(p, length=60)

To get the frame numbers, I don't type -- I use VirtualDub's copy source frame number to clipboard, after assigning it a hotkey.

[manono]

Originally Posted by Bexley

I'll give that a try, but just at a glance it doesn't look like it's going to be much less awkward.

You wind up having to list half or less of the start and end frames. Fewer chances of mistakes and much easier to find and fix when you do make mistakes, without those mistakes cascading down through every start and end frame afterwards. Already much less 'awkward'. I couldn't live without this function in my own work.

Then maybe a ++ syntax to join them instead of the clunky AlignedSplice() that introduces more time and more chance of mistakes when it gets really long.

They're already 'joined'. Much less time involved and many fewer chances of making mistakes.

If I were only doing a few sources I wouldn't worry too much about it, but I have hundreds of hours of material to convert this way...

So do I, and I could never do it the way you're doing it at the moment.

[Bexley]

Thanks guys. I'll give some of these options a try and report the results.

[Bexley]

Originally Posted by raffriff42

I like to use a spreadsheet for this sort of thing:

(note column D is calculated from previous row, column F)

When pasted into the script, it looks like this (adding a BlankClip head and tail)

BlankClip(p, length=15)
\++Trim(	p	,	0	,	950	)
\++Trim(	I	,	951	,	1500	)
\++Trim(	p	,	1501	,	2000	)
\++Trim(	I	,	2001	,	0	)
\++BlankClip(p, length=60)

To get the frame numbers, I don't type -- I use VirtualDub's copy source frame number to clipboard, after assigning it a hotkey.

Ding ding ding!!! We have a winner! LOL

It's amazing how we sometimes can't see the trees for the forest. This solution is so obvious, but it never crossed my mind. I just did a 1hr episode in about 1/3 of the time it took before, with no mistakes.

Thanks all!

[StainlessS]

Still seems like a lot of work to me.

Using Sawbones (link previously given), you could eg

src=AVISource("film.avi")
FX1=AVISource("video.avi")
return StackHorizontal(src,FX1)

and load avs script into VirtualDubfilterMod.

Scan though clip and mark first and last frame where replacing src(film) range with FX1(video) range, and then
press CTRL/SHIFT/1, to insert FX1 range command into notepad command window.
No need for any spreadsheet, nor jumping back and forth between windows, nor copying and pasting individual frame numbers.
Easy peasy.

Final script doing the replacements

Import("FrameSurgeon.avs") # was missing

src=AVISource("film.avi")
FX1=AVISource("video.avi")
CMD="D:\Command.txt"
#ClipClop(src,cmd=CMD,FX1=FX1) # wrong
FrameSurgeon(src,cmd=CMD,FX1=FX1)

EDIT: You only mark the video(FX1) ranges. (SawBones inserts the ranges into notepad, which you save as Command.txt)

[Bexley]

Originally Posted by StainlessS

Final script doing the replacements

src=AVISource("film.avi")
FX1=AVISource("video.avi")
CMD="D:\Command.txt"
ClipClop(src,cmd=CMD,FX1=FX1)

I'm having trouble getting this to work. I have the FX1 ranges saved to command.txt, but when I try to load the final script above I get VirtualDub error: Invalid arguments to function "ClipClop"

[StainlessS]

Oops sorry, should have been

Import("FrameSurgeon.avs")

src=AVISource("film.avi")
FX1=AVISource("video.avi")
#src=colorbars.Killaudio     # test src
#FX1=src.showFrameNumber     # test FX1

CMD="Command.txt"
FrameSurgeon(src,cmd=CMD,FX1=FX1)

EDIT: And make sure that the END RANGE frame number inserted is correct in sawbones, can be (in Vdub) optionally
either Range End Frame Exclusive(Vdub mode) or Range End Frame Inclusive(avs mode) (see Sawbones docs).

EDIT: The result in NotePad Command should of course be Avisynth style Range End Frame Inclusive, the option mentioned
above is only to suite how you best feel comfortable when marking frames in VDub, ie VDub normal style (where you mark
END FRAME + 1, and is converted to END Frame in NotePad [by subtracting 1]), or Avisynth mode where you mark the End
Frame without any conversion for NotePad Command.

[Bexley]

Originally Posted by StainlessS

Oops sorry, should have been

Import("FrameSurgeon.avs")

src=AVISource("film.avi")
FX1=AVISource("video.avi")
#src=colorbars.Killaudio     # test src
#FX1=src.showFrameNumber     # test FX1

CMD="Command.txt"
FrameSurgeon(src,cmd=CMD,FX1=FX1)

EDIT: And make sure that the END RANGE frame number inserted is correct in sawbones, can be (in Vdub) optionally
either Range End Frame Exclusive(Vdub mode) or Range End Frame Inclusive(avs mode) (see Sawbones docs).

EDIT: The result in NotePad Command should of course be Avisynth style Range End Frame Inclusive, the option mentioned
above is only to suite how you best feel comfortable when marking frames in VDub, ie VDub normal style (where you mark
END FRAME + 1, and is converted to END Frame in NotePad [by subtracting 1]), or Avisynth mode where you mark the End
Frame without any conversion for NotePad Command.

OK this worked great after I resolved a few dependencies of FrameSurgeon. You're right, this is pretty easy. Thanks!

[hello_hello]

Bexley,
I'm not sure why you couldn't avoid the intermediate file creation, and given you've got to find the start/end frames for each section, have you tried using the AVS Cutter? There's also one built into MeGUI under the Tools menu. It looks different to the stand-alone version but having only used MeGUI's cutter, I don't know what the differences might be feature-wise. MeGUI's version doesn't do much more than help you specify trims, but using it isn't unlike opening a video with VirtualDub and specifying edit points. When you're done, it adds the trims to the script like so:

__film = last
__t0 = __film.trim(0, 6123)
__t1 = __film.trim(6124, 15185)
__t2 = __film.trim(15186, 20491)
__t3 = __film.trim(20492, 35708)
__t4 = __film.trim(35708, 37783)
__t5 = __film.trim(37784, 47291)
__t0 ++ __t1 ++ __t2 ++ __t3 ++ __t4 ++ __t5

You could create a script, use the AVS Cutter to add the trims, then modify them so each section is taken from a differently filtered version of the video. Creating intermediate files seems a bit pointless in this case. Assuming the filtering from your previous thread is what you're using:

DGDecode_mpeg2source("D:\something.d2v")
A = Last
F = A.TDeint(mode=1).RemoveDirtMC(20).LRemoveDust(17,2).ChangeFPS(60000, 1001)
V = A.Yadif(mode=1).LRemoveDust(17,2).SmoothFPS2(60000,1001)

# Uncomment the Yadif line below for adding the trims to the script initially.
# After adding the trims remove the __film = last line completely.
# Use Notepad's replace function to replace all remaining instances of "__film" with nothing (deleting them).
# Prefix each .Trim() with "F" or "V" as appropriate.
# Delete or comment the Yadif line below again.

# Yadif(mode=1).ChangeFPS(60000, 1001)

__t0 = F.trim(0, 6123)
__t1 = V.trim(6124, 15185)
__t2 = F.trim(15186, 20491)
__t3 = V.trim(20492, 35708)
__t4 = F.trim(35708, 37783)
__t5 = V.trim(37784, 47291)
__t0 ++ __t1 ++ __t2 ++ __t3 ++ __t4 ++ __t5 

Lanczos4Resize(720,480)

If you used QTGMC for de-interlacing, aside from the fact your eyes will be very grateful, you might be better off enabling it's denoising instead of using the other filters for removing dust/dirt etc.

Edit: I suggested another method that converted the frame rate of each section individually but I've deleted it as it has the potential to change the duration a little.


Are you still separating fields and weaving etc?
You mentioned encoding with x264 and burning a bunch of episodes to Bluray discs, but I'd be trying to keep it progressive. I've never created a Bluray compliant video disc in my life so I know very little about it, but Bluray apparently supports 1280×720/59.94p and if that's the only way to keep it progressive it'd be one time I'd give serious consideration to upscaling (ie cropping to 4:3, resizing to 960x720 and adding borders for 1280x720).

Thinking about it, are you sure the film sections are interlaced? If not, maybe it'd be better to convert them directly from 25fps progressive to 59.94fps, rather than bob de-interlace first, although I'd possibly still use TDeint to fix any unexpected combing, and (optionally) tell it to use QTGMC for that. For example:

DGDecode_mpeg2source("D:\something.d2v")
A = Last
Deinterlaced_With_QTGMC = A.QTGMC().SelectEven()
F = A.TDeint(edeint=Deinterlaced_With_QTGMC).RemoveDirtMC(20).LRemoveDust(17,2).ChangeFPS(60000, 1001)
V = A.QTGMC().LRemoveDust(17,2).SmoothFPS2(60000,1001)

[Bexley]

Originally Posted by hello_hello

Bexley,
I'm not sure why you couldn't avoid the intermediate file creation, and given you've got to find the start/end frames for each section, have you tried using the AVS Cutter? There's also one built into MeGUI under the Tools menu. It looks different to the stand-alone version but having only used MeGUI's cutter, I don't know what the differences might be feature-wise. MeGUI's version doesn't do much more than help you specify trims, but using it isn't unlike opening a video with VirtualDub and specifying edit points. When you're done, it adds the trims to the script like so:
[snip]

You could create a script, use the AVS Cutter to add the trims, then modify them so each section is taken from a differently filtered version of the video. Creating intermediate files seems a bit pointless in this case. Assuming the filtering from your previous thread is what you're using:
[snip]

If you used QTGMC for de-interlacing, aside from the fact your eyes will be very grateful, you might be better off enabling it's denoising instead of using the other filters for removing dust/dirt etc.

Edit: I suggested another method that converted the frame rate of each section individually but I've deleted it as it has the potential to change the duration a little.


Are you still separating fields and weaving etc?
You mentioned encoding with x264 and burning a bunch of episodes to Bluray discs, but I'd be trying to keep it progressive. I've never created a Bluray compliant video disc in my life so I know very little about it, but Bluray apparently supports 1280×720/59.94p and if that's the only way to keep it progressive it'd be one time I'd give serious consideration to upscaling (ie cropping to 4:3, resizing to 960x720 and adding borders for 1280x720).

Thinking about it, are you sure the film sections are interlaced? If not, maybe it'd be better to convert them directly from 25fps progressive to 59.94fps, rather than bob de-interlace first, although I'd possibly still use TDeint to fix any unexpected combing, and (optionally) tell it to use QTGMC for that. For example:
[snip]

Yes, I'm keeping it interlaced, because one of my main goals is to preserve the smooth look of video from the original material. My region free player will play it just fine, except it butchers the video and makes it look like it's all film, and that's what I don't like. I want the video to look like video, and the film to look like film. I'm bobbing with Yadif, converting to 59.94 with either SmoothFPS2 or ChangeFPS, resizing, denoising/whatever, and then weaving. I tried several ways to do it in a single script, and it actually wasn't that hard, but AVS kept crashing under the load of running multiple complex functions. Video uses SmoothFPS2, LRemoveDust, and LSFMod, while Film uses ChangeFPS, TemporalDegrain and RemoveDirtMC. AVS just can't handle 30 or more of those going at once, and I found the intermediate file method to be the simplest. I have plenty of storage and a fairly fast box, so it's not a problem.

The target medium is no longer BD (which I've decided is a waste of money and shelf space), and it's now my home media server. But I'm still encoding BD-compliant 480i streams in case I ever do want to burn them for some reason.

[hello_hello]

Unless you're using a multithreaded Avisynth or a buggy plug-in, Avisynth should be able to handle it fine (my record number of trims in a single script is over 100 when I panned and scanned a 4:3 video as 16:9). If you have two versions of the same video in a script with different filtering applied to each, and you use Trim to take segments from each, I'm pretty sure only the required frames are processed. They're not both processed simultaneously as such.

I'm not sure why the video would look less like video if you leave it progressive. Forgetting the frame rate conversion for the moment, generally when you view interlaced PAL/NTSC on a progressive display it'd be bobbed de-interlaced so you're seeing 50/60 progressive frames per second. That's a big part of what gives it the "video" look. Frame rate conversion aside (and I've no idea what your region free player is doing) aren't you effectively bob de-interlacing and then throwing away half the fields so on playback it can be bobbed de-interlaced again?

I've encoded quite a few old BBC shows myself (Blakes 7 included) and while I didn't convert the frame rate, the video bobbed to 50fps looks like video (I used QTGMC because once you do there's no going back) and the film converted to 50fps by duplicating frames looks like film. I can't imagine why they wouldn't, but each to their own....