Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th September 2011, 04:42   #1  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Prune() v1.14 - Trim & Splice plug - 27 Jul 2021

Code:
Prune() A Trim Plugin for Avisynth v2.5, v2.60/+, x86 & x64. by StainlessS.
Req VS2008 CPP Runtimes.

http://forum.doom9.org/showthread.php?t=162446

Video:- Planar, YUY2, RGB32, RGB24. v2.60 dll all Valid Colorspaces incl HBD where Avs+.

Audio:- FLOAT, INT16, INT32, for Audio Fade, ALL types supported if NOT fading (Pass through untouched).

Prune is a simple plugin to trim() multiple source clips and splice the results into a new clip.
Supports up to 256 source clips, with unlimited number of trims/splices into output clip. 
Prune can fade Audio (to reduce clicks between splices) for supported audio formats.
The plugin will do AlignedSplice only.

v1.10, Implements 'Nickname's, Pseudonyms for the source clips, ie instead of '17 1000,2000' you could use 
eg 'Holidays_Aug_2011 1000,2000', to splice frames 1000 to 2000 of the clip 17 into the output clip. 
Supports Nicknames of up to 31 characters in length so you could use eg 'UncleBobsBirthdayBash 10000,14560', or 
'Fred_And_Bobs_Wedding_Part_1 3485,6023'.


Prune(Clip R0, clip R1, ... , clip R255, string 'Cmd',string 'SCmd', bool 'Show', bool 'Ver',int 'dv', bool 'NoErr',
    float 'Fade', bool 'FadeIn', bool 'FadeSplice', bool 'FadeOut', String 'Nickname')

Compulsory args:

        Rx,     Clip, No Default
                Source clips, at least one required. The first clip R0, up to R255 are supported, 256 clips in all.

Optional args:

        Cmd,    string, Default= "", Not set.
                Command file, use eg "PruneCmd.txt" for command file in your *.avs script directory.

        SCmd,   string, Default= "", Not set.
                Command String, as for command file, but commands supplied in a string rather than a file.

        Show    bool, Default= false.
                true = Show info.

        ver     bool, Default=false.
                true = Show version.

        dv      int, Default 0, possible values, 0 - 4 (Out of range values will default to 4==Debug==FULL)
                Generates info/warning/error messages during the filter setup stage (before 1st frame is displayed).

                0 = No info
                1 = Errors sent to DebugView window (Need DebugView Utility)
                2 = Warnings & Errors sent to DebugView window
                3 = Parsing Info, Warnings & Errors sent to DebugView window. 
                4 = Debug info, Parsing Info, Warnings & Errors sent to DebugView window (dll Testing, or just for your curiosity). 

                In DebugView, filter for "Prune:" to skip all the extraneous stuff generated by other plugins/programs.
                (On "Menu:Edit-Filter/Highlight", and enter "Prune:", all ClipClop messages are prefixed by "Prune:".

                http://technet.microsoft.com/en-gb/sysinternals/bb545027

        NoErr,  bool, Default false

                false = Stop on ANY command error. (usual mode)
                     You will be informed in DebugView which (if any) errors can be ignored if switched to NoErr==true.

                true =  Suppress trivial errors where possible.
                     Only use after you have viewed and dismissed as unimportant any DebugView errors displayed when NoErr==false.
                     Not all errors are ignorable.

        Fade    float, Audio Fade Duration. Default 1.0 (1 millisecond).
                0.0 (off) to 25.0 = 25.0 milliseconds.
                Will be limited internally to 25.0 milliseconds or half frame duration, whichever is lesser.

        FadeIn  bool, Default false.
                false, no audio fade-in at start of output clip.
                true, fade-in audio at start of output clip.  (will switch internally to false if Fade==0.0)

        FadeSplice  bool, Default false.
                false, no audio Fade-Out/In at splices in output clip.
                true, Audio Fade-Out/In at splices in output clip. (will switch internally to false if Fade==0.0)

        FadeOut  bool, Default false. 
                false, no audio fade-Out at end of output clip.
                true, fade-Out audio at end of output clip. (will switch internally to false if Fade==0.0)

        Nickname String, default "" (not set) [Added v1.10].
                Nicknames representing a clip index eg 'Hols 37,2045' instead of '5 37,2045'.                


Prune is a simple plugin to trim() multiple source clips and splice the results into a new clip,
has Audio fade (to reduce clicks between spices) for supported audio formats.
The plugin will do AlignedSplice only.
All, the source clips must be in the same colorspace, same size as the first R0, source clip.
The audio must also be the same as the first R0 clip.

Prune will accept up to 256 source (Rx) clips, they are not named.
(Hopefully that is enough for those of you who are very demanding [You know who you are])  ;)
Source clips start at R0 and go up to R255.

Either the command file (Cmd) or the Command string (SCmd) must also be supplied.
*** NOTE ***,  if both Cmd and SCmd are supplied, the commands in the Command File are processed
first followed by those in the SCmd String, So if both are processed, all the trims in the command file
will be spliced together and those in the SCmd will be added later. It might be wise to use the SCmd
string when setting up your trims and then transfer them to the command file when happy. Then
continue adding trims to the end of the clip using the SCmd string, etc.


The Command file (Cmd) and the Command String (SCmd) will have a series of ranges to Trim, 
and will have the below format.

Rx(s,e)


Where the 'R' is optional and can be omitted.

where x is clip index number eg R7 is clip 7, (the first one is R0).

Where the '(' and ')' are optional and if skipped then the opening '(' must be replaced by Comma ',' or ' ' (SPACE).

Where the ',' (COMMA) can be replaced with a ' ' (SPACE) [only needed if 'e' is specified].

Where the ')' is strictly speaking, unnecessary, whether the '(' is used or not.

s is the start frame and MUST be supplied.

e is optional and if missing, then the range will consist of only 1 frame.

        If e=0, then the range will continue to the last frame of the Rx clip.

        If e is -ve (negative) then it will represent a -ve frame count eg -2 means 2 frames.

These are all valid:

R0 (35,42)      # This is a comment
R1 65 374
2 1000
3 3700 5000
4 (36734,0)     # My helicopter is full of penguins
5,23644,-3
R6,20,30)
R6(56,123

If the command file contained the above commands then clips R0 to R6 have all to be supplied to Prune.

The Cmd file commands MUST be line separated (ie one command per line).
The SCmd commands can be either Line Separated as the cmd commands or use a ';' SEMICOLON separator, as in

SCmd="R0(35,42) # My Zeppelin is full of lead ; R1 65 374
        R2,1000,1200; R2(1400,1500)
        R3(2000,-100)  # My Spaceship is replete with aardvarks"


In SCmd, A SEMICOLON ';' would not be acceptable in a comment, it would signify a new line and command pending
(ie PARSE error, it would look for a command following it).

  The term 'Rx' clip originated in sister plugin ClipClop where it meant 'replacement clip x',
it is kept in Prune for mutual compatiblity with ClipClop and so that future plugins might be able
to parse both Prune & ClipClop's command files including 'Rx nnn,nnn' type commands. 

-----------------------

v1.10, Nickname string is roughly the same format as SCMD, SEMICOLON (;) separated if on same line
or line separated, and can also contain comments. A Nickname, MUST start with either an underscore ('_'),
or an Alphabetic character ('A'-'Z' or 'a' - 'z'). The remainder of the nickname can contain underscore,
alphabetic or numeric ('0' - '9') characters, as in "_Testing_123_Testing" or "AnyOldName_432" or "Fred".
The length limit on Nicknames is 31 characters.
There are a set of RESERVED names that CANNOT be redefined by Nickname, "Rnnn" or "rnnn" where 'nnn' represents
1 or more numeric digits. In a command file/string, you can use eg "R0" to "R255" (or '0' to '255') to refer to
the clips whether or not they have nicknames applied.
As well as "(", "," and SPACE " " separators, a nickname allows "=". Below some valid examples:-

NickName = "
    Xmas_Crackers=1 #  This is a comment ; Vaca,2 # Costa Del Sol July 2006 ; Hols_Part_1(3) # Never again 
    Hols_Part_2(4 # Never say 'Never Again'.
    ZX(5)
    QP (6)   
    XXX 7 # Patient: My hand hurts when I do this. ; ttt = 8 # Doctor: Then stop doing that!
    ds123dgsdg__dld 9
    ww(10
    yy=11)   
"

Nicknames do not have to be in ascending order, but might be best to keep them that way.

Using above Nickname example,

"Hols_Part_1 2000,3000" would replace source frames 2000 to 3000 with clip 3 frames 2000 to 3000, when used in a command
file or command string. Nicknames are case insignificant ('A' == 'a').


----------------------

If NoErr is true, then the plugin will try to avoid throwing an error for 'suspect' commands, not all
errors will be ignored.
Commands that do not fit in the valid formats described above will not be ignored (PARSE error). 
-ve start frame will not be ignored (PARSE error, it could be clipped to 0, but I REFUSE to allow it).
If an Rx clip has been omitted (missed out) then all Rx commands can be ignored.
If a range fully or partially falls outside of the source clips, then those can be ignored (invalid part of the range).

It is recommended that you DO use the linked DebugView utility, to check out warnings,
Prune() will try to warn you if you need to re-evaluate your,,, evaluations. 
There is no need to be alarmed by the NoErr option, you do not have to use it.
Dont always use NoErr = true, unless you really really know what you
are doing (or your name is 'Cretindesalpes' :)

---------------------------------------------------------------------------------------------
The CMD and SCMD parsing is a 2 pass process, reason being, we need to know how many frames will
be generated, in order to allocate arrays to hold the trim data, So the 1st pass counts generated
frames, and the second pass allocates and fills in the arrays with the trims. Both passes will
produce output in DebugView, CMD, SCMD, CMD, SCMD.
---------------------------------------------------------------------------------------------
In v1.02, trim/splice 'coalescence' has been implemented for adjacent trims from same source clip,
ie if a command file had the following in it:

0 (10,12)
0 (13,15)
0 (16,18)

then frames 10 through 18 would be amalgamated or coalesced as though from a single trim/splice.
This would be more conservative on memory requirements in Prune, and also avoid fading of audio between 
splices where there was clearly no need for such.

If you wanted (for whatever reason) to KEEP some frames in a clip based purely on some runtime
filter, eg YDifferenceFromPrevious, then you might be able to use something like this:

    #--------------------------------------------------------------------------------------------------------
    # Template, to KEEP frames where YDifferenceFromPrevious is greater than 0.1 (delete those that are less than).
    # The '|| current_frame==0' part of the condition is required for the first frame as it always results in 0
    # using YDifferenceFromPrevious on frame zero.
    # Could use other conditionals to keep/delete for other applications.

    Avisource("In.avi").ConvertToYV12() # Ensure Planar for YDifferenceFromPrevious (not necessary for Prune pass).
    assumeFPS(250.0)    # Fast as we can
    Writefileif("In_cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0",""" "0," ""","current_frame",append=false)
    #--------------------------------------------------------------------------------------------------------

And for the 2nd pass for Prune(),

    #--------------------------------------------------------------------------------------------------------
    Avisource("In.avi")     # ConvertToYV12(), Not Necessary here.
    Prune(cmd="In_Cmd.txt")
    #--------------------------------------------------------------------------------------------------------

The first Writefile pass might produce a command file listing for Prune(), like this:

    0,0
    0,1
    0,3
    0,4

Where in this case frame two was omitted. All the initial zeros specify clip zero as the source clip for each of
these single frame trim/splices, and the second number is the frame number. When fed into Prune(), the first two
frames 0 and 1 will be coalesced into a trim and likewise the frame 3 and 4 into another trim as if these were specified
in the command file as:

   0,0,1    # first trim
   0,3,4    # second trim

I have tested the speed at which Prune() can do it's two pass parse on a command file of 30,000 frames, 42 of which were
omitted by the writefile script and the input of 30,000 single frame trims was coalesced to 8 trims overall
with an overhead of about 1.5 seconds on my less than powerful machine. In most cases, I think you are unlikely
to be wanting to feed in 30,000 or more trims to Prune, but the above example shows it can be done without effort.
Above timing was with dv=0, ie DebugView output off (a couple of seconds longer with full debug on).

Just a little idea of where this might be useful, security cameras where you want just the interesting bits
and omit the masses of static frames that are of no interest but you might want to raise the condition above 0.1 as
used above to avoid false alarms.

--------------

Almost all of the hard work of Prune is done during setting up the filter. During the rendering
stage, Prune() does not do anything terribly clever, it just selects the frame to use from
pre-calculated arrays, what it does do, it does fast as possible. Prune() has an overhead of approx
1 trim or splice filter, no matter how many trim/splices are involved, if you are processing eg
30000 single frame trim/splices it should be somewhat faster than if you had to do it in script.

---------------------------------------------------------------------------------------------

GetParity is implemented in the plugin (Thanks, Cretindesalpes).

---------------------------------------------------------------------------------------------
Sister, Range/Frame Replacement plugin ClipClop() [with Nicknames too].
http://forum.doom9.org/showthread.php?t=162266

DebugView utility:
http://technet.microsoft.com/en-gb/sysinternals/bb545027

---------------------------------------------------------------------------------------------
StainlessS.
Here,jpg showing Prune FadeSplice:- http://forum.doom9.org/showthread.ph...32#post1715332
EDIT: here


FadeSplice implemented due to this thread:- http://forum.doom9.org/showthread.php?t=162907

You might also find ClipClop() of interest, it is a Clip range/frame replacement plugin.
http://forum.doom9.org/showthread.php?t=162266

Also Get this, DebugView
http://technet.microsoft.com/en-gb/s...rnals/bb545027

See MediaFire in sig below this post:-
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 10th December 2021 at 13:50. Reason: Update
StainlessS is offline   Reply With Quote
Old 5th September 2011, 20:00   #2  |  Link
tin3tin
Registered User
 
tin3tin's Avatar
 
Join Date: Mar 2005
Posts: 366
Wow, looks interesting.
__________________
DVD slideshow GUI(Freeware).
tin3tin is offline   Reply With Quote
Old 6th September 2011, 18:33   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Bug in v0.0Alpha, Not really too concerned about it, dont think it would ever be executed anyway,
so will not issue update at present. Here it is:
Code:
void __stdcall Prune::GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env)
{	// start and count are in samples

	// Check if something Dreadfully Ghastly has happened. (No idea if any of this is necessary.)
	if(start > outEndA[nTrims-1]) {
		return;						// Yep, something Dreadfully Ghastly.
	} else if(start < 0) {
		if(start + count <= 0)
			return;					// Yep, something Nastily Naughty.
		count = count + start;		                // v0.01 BUGFIX, was "count = count - start"
		start = 0;
	}
	if((start + count - 1) > outEndA[nTrims-1]) {
		count = outEndA[nTrims-1] - start + 1;		// Yep, something Horrendously Horrid.
	}

	int rx,s,e;

	s=FindTrimA(start);					// trim number holding first audio sample 
	e=FindTrimA(start+count-1);				// trim number holding the last audio sample
	
	for(; s <= e; ++s) {
		__int64 trim_count	=	outEndA[s] + 1 - start;		// Samples from this trim.
		if(trim_count > count)
			trim_count = count;
		__int64 trim_offset =	start - outStartA[s];			// Output sample relative START of this trim

		trim_offset += trimFrameStartA[s];				// Relative Rx(frame)

		rx = trimRx[s];
		((rx == 0) ? child :  rClips[rx])->GetAudio(buf,trim_offset,trim_count,env);
		start = start + trim_count;
		count = count - trim_count;
		buf = (char*)buf + vi.BytesFromAudioSamples(trim_count);
	}
}
Bug is hi-lited in red.

I was testing the Prune_ColorsBars_test.AVS in AvsEdit with
Media Player Classic, and the audio for some reason seems to
lag the trim switchover points. dont know why this should do
it, dont know if media player classic is the problem or not.
Anyway, dont think anything to do with Prune, as it plays
just fine in VirtualDub and also if rendered to a lossless
format and played (even in Media Player Classic).
EDIT: See Here:-
http://forum.doom9.org/showthread.ph...59#post1524259


Anybody with Audio experience notice problems in the source?
(never touched Audio before).

Also, anybody think that UnAligned Splice would be at all useful,
I would never want it and would rather forget about even attempting it.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 6th September 2011 at 22:02.
StainlessS is offline   Reply With Quote
Old 10th September 2011, 22:53   #4  |  Link
tin3tin
Registered User
 
tin3tin's Avatar
 
Join Date: Mar 2005
Posts: 366
That's really interesting stuff you're doing Stainless. What are your longer term plans? To use those plugins to pass NLE UI data into avisynth? Or an EDL importer?
__________________
DVD slideshow GUI(Freeware).
tin3tin is offline   Reply With Quote
Old 15th September 2011, 12:28   #5  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by tin3tin View Post
That's really interesting stuff you're doing Stainless. What are your longer term plans? To use those plugins to pass NLE UI data into avisynth? Or an EDL importer?
Nothing quite so grand, (had to guess that NLE UI was something
like Non Linear Editor User Interface, and EDL was Edit Decision List), I dont use AVSPmod or anything like that, have heard good things about AVSPMod and will try it out one day, they could of course use the plugins in NLE's but it was not my intention.
I had already a group of plugins for editing frames and trims, but
they were not in shape for publication, I just decided to clean them up & make them more concrete. I had a clip that required an awful lot of single frame edits (about 2000 hand edits) using eg "predicted from previous" etc, and would like to have had a Rotoscoping utility to fix those that could not be adequately fixed by other means. I will at some point do the extract/replace
plugin to pull out 3 frames for Rotoscoping and put the central one back, (as mentioned on ClipClop thread), and hopefully someone will use that plugin to implement a Rotoscoping utility,
if not, then I guess I might have a bash at it in VB.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 19th September 2011, 09:05   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
New v1.00 with bugfix for previously posted problem. No other code alterations,
does not seem to need any. Perhaps the final version.
See first post.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 15th October 2011, 03:24   #7  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
My bad, now I feel like an idiot
Anyway, I still think the readme can be improved to make it tidier or more clear so your tools don't scare out the people, the most novices.
Thanks a lot for these plugins : )
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 15th October 2011 at 03:36. Reason: a typo error of mine
Dogway is offline   Reply With Quote
Old 15th October 2011, 12:25   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by Dogway View Post
My bad, now I feel like an idiot
Anyway, I still think the readme can be improved to make it tidier or more clear so your tools don't scare out the people, the most novices.
Thanks a lot for these plugins : )
Well, my writings have never been confused with that of William Shakespear, but I did not know they were so very very bad,
I think there must be more of a Stephen King influence on them.

If you would like to contribute an edited version of the text, that would be appreciated, it looks fine to me, but then, I'm quite easily pleased.

EDIT:- Can you at least tell what you found confusing. Thanks.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th October 2011 at 14:29.
StainlessS is offline   Reply With Quote
Old 16th October 2011, 19:17   #9  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,352
lol I just thought you could get more feedback if the OP was more welcome. Im not native speaker so I can't help much but the first insight on what prune does is in your second edit. There you explain one situation where it can help out. In my case it was only 2 clips, and I wanted chunks from one and chunks from the other interleaving both. So I wasn't sure if this was going to do that. Just gave it a shot. Probably I think over too much and the title is self explanatory. Sorry for that. From your first post I also got to know audiograph, cool tool, so thanks for that too.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 5th November 2011, 08:04   #10  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
@Dogway,

You can come out from behind the sofa now, this next beta is I hope not as scary as the last one.
Take a look at it and if you find anything particularly terrifying, then please let me know.

Oh, and if comments in source eg "// Check if something Dreadfully Ghastly has happened",
made your blood run cold, then it is just a little sick programmer humour.
(Should really have posted this answer on the 31 of last month, oooooOOO!).

New Beta v1.01beta2 in next post.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th November 2011 at 08:44.
StainlessS is offline   Reply With Quote
Old 5th November 2011, 08:14   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
New version Prune() v1.01Beta2

EDIT: Link removed.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th November 2011 at 21:34. Reason: Update
StainlessS is offline   Reply With Quote
Old 15th November 2011, 22:23   #12  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Having just posted the below code fragment in another thread, I thought it might be a good
idea to also post it here, so I dont forget it.



Code:
avisource("d:\avs\avi\1.avi").ConvertToYV12() # The test clip was RGB

Writefileif("d:\avs\avi\1.txt","YDifferenceFromPrevious>=0.1 || current_frame==0",""" "0," ""","current_frame",append=false)
The above code would produce a Prune() command file to keep all frames with a difference in the Luma from
the previous frame of 0.1 or greater. The condition to keep could be changed for another application.
In the above, it would be used to keep frames that were different and throw away duplicates.
You may only want to use this as a last resort if you can find no other way to do something.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th November 2011 at 21:41.
StainlessS is offline   Reply With Quote
Old 17th November 2011, 21:47   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
New Version Prune() v1.02.

Prune is a simple plugin to trim() multiple source clips and splice the results into a new clip.
Supports up to 256 source clips, with unlimited number of trims/splices into output clip.
Prune can fade Audio (to reduce clicks between splices) for supported audio formats.
The plugin will do AlignedSplice only.
See first post.

Here a little excerpt from update notes that have been added with this version.

Code:
In v1.02, trim/splice 'coalescence' has been implemented for adjacent trims from same source clip,
ie if a command file had the following in it:

0 (10,12)
0 (13,15)
0 (16,18)

then frames 10 through 18 would be amalgamated or coalesced as though from a single trim/splice.
This would be more conservative on memory requirements in Prune, and also avoid fading of audio between 
splices where there was clearly no need for such.

If you wanted (for whatever reason) to KEEP some frames in a clip based purely on some runtime
filter, eg YDifferenceFromPrevious, then you might be able to use something like this:

    #--------------------------------------------------------------------------------------------------------
    # Template, to KEEP frames where YDiffFromPrevious is greater than 0.1 (delete those that are less than).
    # The '|| current_frame==0' part of the condition is required for the first frame as it always results in 0
    # using YDifferenceFromPrevious on frame zero.
    # Could use other conditionals to keep/delete for other applications.

    Avisource("In.avi").ConvertToYV12() # Ensure Planar for YDifferenceFromPrevious (not necessary for Prune pass).
    assumeFPS(250.0)    # Fast as we can
    Writefileif("In_cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0",""" "0," ""","current_frame",append=false)
    #--------------------------------------------------------------------------------------------------------

And for the 2nd pass for Prune(),

    #--------------------------------------------------------------------------------------------------------
    Avisource("In.avi")     # ConvertToYV12(), Not Necessary here.
    Prune(cmd="In_Cmd.txt")
    #--------------------------------------------------------------------------------------------------------

The first Writefile pass might produce a command file listing for Prune(), like this:

    0,0
    0,1
    0,3
    0,4

Where in this case frame two was omitted. All the initial zeros specify clip zero as the source clip for each of
these single frame trim/splices, and the second number is the frame number. When fed into Prune(), the first two
frames 0 and 1 will be coalesced into a trim and likewise the frame 3 and 4 into another trim as if these were specified
in the command file as:

   0,0,1    # first trim
   0,3,4    # second trim

I have tested the speed at which Prune() can do it's two pass parse on a command file of 30,000 frames, 42 of which were
omitted by the writefile script and the input of 30,000 single frame trims was coalesced to 8 trims overall
with an overhead of about 1.5 seconds on my less than powerful machine. In most cases, I think you are unlikely
to be wanting to feed in 30,000 or more trims to Prune, but the above example show it can be done without effort.
Above timing was with dv=0, ie DebugView output off (a couple of seconds longer with full debug on).

Just a little idea of where this might be useful, security cameras where you want just the interesting bits
and omit the masses of static frames that are of no interest but you might want to raise the condition above 0.1 as
used above to avoid false alarms.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 18th November 2011, 17:21   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
From v1.02 update:

Quote:
Originally Posted by StainlessS View Post
Code:
#--------------------------------------------------------------------------------------------------------
    # Template, to KEEP frames where YDiffFromPrevious is greater than 0.1 (delete those that are less than).
    # The '|| current_frame==0' part of the condition is required for the first frame as it always results in 0
    # using YDifferenceFromPrevious on frame zero.
    # Could use other conditionals to keep/delete for other applications.

    Avisource("In.avi").ConvertToYV12() # Ensure Planar for YDifferenceFromPrevious (not necessary for Prune pass).
    assumeFPS(250.0)    # Fast as we can
    Writefileif("In_cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0",""" "0," ""","current_frame",append=false)
    #--------------------------------------------------------------------------------------------------------

And for the 2nd pass for Prune(),
    #--------------------------------------------------------------------------------------------------------
    Avisource("In.avi")     # ConvertToYV12(), Not Necessary here.
    Prune(cmd="In_Cmd.txt")
    #--------------------------------------------------------------------------------------------------------

Just a little idea of where this might be useful, security cameras where you want just the interesting bits
and omit the masses of static frames that are of no interest but you might want to raise the condition above 0.1 as
used above to avoid false alarms.
ref: above
Also, if your security cam footage has some kind of timecode on it, you could blank it (temporarily)
for the WriteFileIf pass (using eg blankclip/resize/overlay) and remove for the Prune pass.
Also, if it did not have a timecode, then you could add one
during the Prune pass so you could examine interesting snips from the edited footage using the original clip.

Here is a snippet stolen from Gavino:
http://forum.doom9.org/showthread.ph...42#post1539542

EDIT:-
Code:
function DeleteFrames2(clip c, string condition, string fileName) {
  keepCondition = "!(" + condition + ")"
  WriteFileIf(c, fileName, keepCondition, """ "0," """, "current_frame", append=false)
}
}
The above snip, provides a more flexible interface than using the "WriteFileIf" directly but for deleting frames.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 18th November 2011 at 21:04.
StainlessS is offline   Reply With Quote
Old 18th November 2011, 17:43   #15  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by StainlessS View Post
Here is a snippet stolen from Gavino
That's fair, as I stole the underlying method from your earlier idea.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 18th November 2011, 19:40   #16  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
OK Gavino, we are then square.

Stealing it back again, but with modification:-

Command Generator functions
PruneGenny.avsi
Code:
# PruneGenny.avsi
Function PruneKeepFrames(clip c, string fileName, string condition, bool "Fast") {
# Conditional KEEP frames Command File generator for Prune()
# MUST call with correct colorspace for condition eg YDifferenceFromPrevious requires a Planar colorspace.
# May require additional condition for end frames for eg YDifferenceFromPrevious and YDifferenceToNext.
#  Any-DifferenceFromPrevious,
#     Gives 0 for frame zero and so you may want to add to condition " || current_frame==0" to include frame 0.
#  Any-DifferenceToNext,
#     Gives 0 for last frame and so you may want to add to condition " || current_frame==framecount-1 " to include last frame.

    c=(Default(Fast,true))?c.AssumeFPS(250.0):c         # Fast as we can if Fast = true (Default true)
    WriteFileIf(c, fileName, condition, """ "0," """, "current_frame", append=false)
} 

Function PruneDeleteFrames(clip c, string fileName, string condition, bool "Fast") {
# Conditional DELETE frames Command File generator for Prune()
# MUST call with correct colorspace for condition eg YDifferenceFromPrevious requires a Planar colorspace.
# Is EXACT opposite of PruneKeepFrames() for same conditional.
# NOTE, any additional end frame condition as used in PruneKeepFrames, will also be inverted
#   eg " || current_frame==0", frame 0 would be deleted.

    c=(Default(Fast,true))?c.AssumeFPS(250.0):c         # Fast as we can if Fast = true (Default true)
    condition = "!(" + condition + ")"
    WriteFileIf(c, fileName, condition, """ "0," """, "current_frame", append=false)
}

###########################
# Below :HINT ON USE:
/* ################
  Avisource("In.avi").ConvertToYV12()     # Planar Required for AverageLuma and YDifferenceFromPrevious
#####  pick one of below [can replace 0.1 with whatever threshold you like]
  #PruneKeepFrames(".\cmd.txt","AverageLuma>=0.1")
  #PruneDeleteFrames(".\cmd.txt","AverageLuma>=0.1")
#####  Or with additional conditional for end frame [YDifferenceFromPrevious ALWAYS 0.0 on frame 0, YDifferenceToNext ALWAYS 0.0 on last frame]
  #PruneKeepFrames(".\cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0")           # Always keeps first frame 0 (else delete)
  #PruneDeleteFrames(".\cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0")         # Always deletes first frame 0 (else keep)
  #PruneKeepFrames(".\cmd.txt","YDifferenceToNext>=0.1 || current_frame==framecount-1")      # Always keeps last frame (else delete)
  #PruneDeleteFrames(".\cmd.txt","YDifferenceToNext>=0.1 || current_frame==framecount-1")    # Always deletes last frame (else keep)
  Return last

#####  And Pass2
  Avisource("In.avi") #.ConvertToYV12()  # Planar Required for AverageLuma and YDifferenceFromPrevious NOT REQUIRED HERE
  Return Prune(cmd=".\cmd.txt")
################ */
Template function calls for above command generators
Code:
# Import(".\PruneGenny.avsi"
Avisource("In.avi").ConvertToYV12()     # Planar Required for AverageLuma and YDifferenceFromPrevious
# pick one of below [can replace 0.1 with whatever threshold you like]

#PruneKeepFrames(".\cmd.txt","AverageLuma>=0.1")
#PruneDeleteFrames(".\cmd.txt","AverageLuma>=0.1")

# Or with additional conditional for end frame [YDifferenceFromPrevious ALWAYS 0.0 on frame 0, YDifferenceToNext ALWAYS 0.0 on last frame]

#PruneKeepFrames(".\cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0")           # Always keeps first frame 0 (else delete)
#PruneDeleteFrames(".\cmd.txt","YDifferenceFromPrevious>=0.1 || current_frame==0")         # Always deletes first frame 0 (else keep)
#PruneKeepFrames(".\cmd.txt","YDifferenceToNext>=0.1 || current_frame==framecount-1")      # Always keeps last frame (else delete)
#PruneDeleteFrames(".\cmd.txt","YDifferenceToNext>=0.1 || current_frame==framecount-1")    # Always deletes last frame (else keep)
  Return last
And Pass2
Code:
    Avisource("In.avi") #.ConvertToYV12()  # Planar Required for AverageLuma and YDifferenceFromPrevious NOT REQUIRED HERE
    Return Prune(cmd=".\cmd.txt")
EDIT: Below from another thread: http://forum.doom9.org/showthread.php?t=165345
Where a user wanted to delete based on a averageLuma of cropped area
Code:
# PASS 1
source=avisource("D:\avs\avi\1.avi").ConvertToYV12()

X       =   40          # X-Coord of Crop area, For your Area of Interest.
Y       =   40          # Y-Coord of Crop area
W       =   32          # Width  of Crop area, YV12, MOD4
H       =   32          # Height of Crop area, YV12, MOD4

METRICS =   True        # True To Show Metrics (NO Pass1 CMD file Created)
DIM     =   True        # True will dim NON Area Of Interest when viewing Metrics
Thresh  =   128.0       # 0.0 -> 255.0, Threshold for frame drops. SET using view metrics
ALIGN   =   7           # METRICS SubTitle Alignment 7=TopLeft, 1=BotLeft (As KeyPad)

# -----------------------

cropped=source.Crop(X,Y,W,H)    # Your area of interest

source = (DIM) ? source.levels(0,1.0,255,0,128).Overlay(cropped,x=X,y=Y,opacity=1.0) : source

Return (METRICS)\
        ? source.ScriptClip(""" TmpAL=cropped.AverageLuma() Subtitle(String(TmpAL) + ((TmpAL>Thresh)?" DROP":""),align=ALIGN)""" ) \
        : cropped.PruneDeleteFrames("cmd.txt","AverageLuma>Thresh",Fast=True)
Here just as a reminder.

EDIT: And Pass2
Code:
    #--------------------------------------------------------------------------------------------------------
    avisource("D:\avs\avi\1.avi") # .ConvertToYV12(), Not Necessary here.
    Prune(cmd="Cmd.txt")
    #--------------------------------------------------------------------------------------------------------
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 10th December 2021 at 14:53.
StainlessS is offline   Reply With Quote
Old 18th November 2011, 20:29   #17  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Ha! You've just made me realize I had the condition the wrong way round on my DeleteFrames2 function - it was keeping the frames instead! Off now to change it...

BTW instead of adding something like AssumeFPS(250), what I do for this sort of thing is just run VirtualDub's 'video analysis pass' on the script, then it really does run as fast as truly possible.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 18th November 2011, 21:15   #18  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by Gavino View Post
Ha! You've just made me realize I had the condition the wrong way round on my DeleteFrames2 function - it was keeping the frames instead! Off now to change it...

BTW instead of adding something like AssumeFPS(250), what I do for this sort of thing is just run VirtualDub's 'video analysis pass' on the script, then it really does run as fast as truly possible.
Have adjusted my ill gotten gains to reflect their new content.
How would I dare to correct you on the discrepancy.

Yes, can use VD 'video analysis pass', above AssumeFPS would
not hurt though in that case and could use even MPlayer.
IIRC, VDMod screws up if you try to feed it 300FPS+,
(Or maybe just ignores it, cant remember).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 23rd March 2012 at 01:09.
StainlessS is offline   Reply With Quote
Old 23rd March 2012, 01:09   #19  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Video Analysis pass jumps back to beginning of file on completion, so if you
want to inspect metrics when done, you cant, this is the real reason I mainly use
AssumeFPS(250.0), but I forgot why.

EDIT: Above for VirtualDubMod, VirtualDub2 no longer jumps back to start so can use Video Analysis pass with that without Assumefps(250.0).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 10th December 2021 at 14:29.
StainlessS is offline   Reply With Quote
Old 27th March 2012, 13:28   #20  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
New version v1.10, see 1st post.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Reply

Tags
editing, range, splice, trim

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:39.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.