Trimmaker 0.06: a program to help making trims

[tsp]

I have made a small program to assist in making the trims. It takes a text file with the numbers of the frames that is to be cut out as an input and makes an avs file with the necessary trims. It can also just include the frames from the text file. A possible use of this program is to use writefileif() to detect the static part of the video and make a logfile with the noise frames as descript in this thread (http://forum.doom9.org/showthread.php?s=&threadid=82066).

You can get the install file here with all the nessecary files here (http://wave.prohosting.com/~3x1999/trimmaker.rar) (1.7MB)
or the exe and readme only here (http://wave.prohosting.com/~3x1999/trimmaker.zip) (17 KB) or here (http://www.tsp.person.dk/trimmaker.zip)

Changelog:

0.01 10/9-2004 Initial version.

0.02 11/9-2004 Added the ability to output the junkpart and included a save as option.

0.03 11/9-2004 Fixed a bug so the program now support video that are longer than 32000 frames.

0.04 11/9-2004 Added the ability to set a minimum length of good parts and to include text from the input file.

0.05 13/9-2004 Fixed a bug concerning the sorting of the frame numbers and added the ability to assign a string to the start of the good or bad parts and to output both.

0.06 21/9-2004 Trimmaker now remembers the last used settings. It is now possible to ignore offset if there are less than x bad frames in a row.

Any comments or suggestions are welcome.

[esby]

mmm in what language did you do it?

And why is it 1.6mb for 'just' generating trim list.?

maybe it's me but i don't like having a setup.exe to run to test new program


esby

Edit:
I just opened the zip (and not the rar)
I guess the rar must contain the needed dll.

[tsp]

mmm in what language did you do it?

And why is it 1.6mb for 'just' generating trim list.?


Well visual basic 6. It allways result in such large setupfiles.

[mkanel]

tsp,

I found one small bug. The minimum length of good parts only works when outputting good frames, not bad frames.

Another issue, if the first frame of a video is junk then preview output from start (Ctrl-F5) in VirtualDubMod writes a zero as the first junk frame, after processing it snaps back to frame zero and again writes zero now as the last junk frame. When the good trims are written the last trim becomes ++ Trim(1,0) basically repeated the entire video. This isn't a problem as long as the first frame of the video is a good frame. Tsp, could you have trimmaker not use the last junkframe entry from log.txt if it's zero?

One more feature request. With the include text option Trimmaker can now make a useable avs. Could you have trimmaker generate an avs with both the good frames and the junk frames. Both trim sets could be assigned a variable so that the avs might look like this.

Avisource=("C:\some.avi")
#showframenumber
Good=trim( 94, 237)++\
trim( 285,0)
Junk=trim( 9, 12).subtitle("GOOD",x=-1,size=34)++trim( 13, 16).subtitle("JUNK",x=-1,size=34)++trim( 17, 20).subtitle("GOOD",x=-1,size=34)++Trim(0,- 1).blankclip++\
trim( 16, 19).subtitle("GOOD",x=-1,size=34)++trim( 20, 25).subtitle("JUNK",x=-1,size=34)++trim( 26, 29).subtitle("GOOD",x=-1,size=34)++Trim(0,- 1).blankclip
Good
#Junk

This would make it really easy to toggle back and forth between the good frames and the junk frames if you needed to perform some manual tweaking.

Thanks. Mike.

[tsp]

mkanel: Fixed the 2 bugs and added the suggested feature. Thanks for all the suggestions and debugging.

[jorel]

hy all!
few minutes after you post trimmaker, i was trying to download the files. today again 2 times and i can't get it with or without download managers.
can you help please?
anyone have the same problem?
thankyou tsp! :)

[tsp]

Jorel: you can try this (http://www.tsp.person.dk/trimmaker.zip) place for the exe only. Is there anyone who knows a good place to host this program?

[K-]

lycos.co.uk can host small files, they have a daily cap on 15mb I think or on your isp webspace if you have any.

[mkanel]

I wouldn't bother with this unless you find trimmaker useful to you and think you'll use it regularly. If your just checking out trimmaker you probably should start with a very simple avs script like this:AVISource("D:\Some.avi").converttoyv12

output="d:\log.txt"
WriteFileIf(output,"AverageChromaU()>200\
||(AverageLuma>215"\
,"current_frame",append=false)

Obviously you need to change the path and file name to your video, and the path to the log.txt. The above avs script should then write most blue screen and static/snow scenes to log.txt which you can feed to trimmaker to see how it works. The above script will make some mistakes in choosing junk frames but it should give you an idea of what trimmaker does. If it's worth pursuing you can then work on fine tuning the junk definitions. All I'm looking to trim are the gaps in a video from a captured analog tape. I suppose if your creative you might try to find a way to output other kinds of junk frames for trimmaker such as advertisments.


Enough of that here's what I really wanted to post


Trimmaker is working well for me so I've scripted a template for AviSynthesizer to create my junk finding avs. I don't use the freestanding AviSynthesizer but the one built into VirtualDubMod. If you open "D:\Folder\Subfolder\myvideo.avi" in VDM with this template (trimmaker.avst) you'll get this "MyVideo--TrimMaker.avs" in the same subfolder. If you run this avs in VDM(preview from start crtl-F5) it will create this "MyVideo1 TrimLog.txt". If you open "MyVideo--TrimMaker.avs" again or refresh it it will keep creating incremenal log files of the same name "MyVideo2 TrimLog.txt", etc. The output of the TrimLog.txt is this:

AviSource("D:\Folder\SubFolder\MyVideo.avi")
#ShowFrameNumber
0
1
2
3
4
Good

Which can now be fed to trimmaker to yield this file "MyVideo1 TrimLog.avs" containing:

AviSource("D:\Folder\SubFolder\MyVideo.avi")
#ShowFrameNumber
Good=trim( 5,0)
Junk=trim( 0, 4).subtitle("JUNK",x=-1,size=34)++trim( 5, 8).subtitle("GOOD",x=-1,size=34)++Trim(0,- 1).blankclip
#Junk
Good

After opening "MyVideo1 TrimLog.avs" in VDM you can now switch back and forth between displaying Good for good trims and Junk for junk trims by changing which variable is commented at the end of the AVS script. When viewing "Junk" I would also uncomment #ShowFrameNumber to help with manually tweaking the good trims.

In TrimMaker make sure to check these boxes: output badframes, output goodframes, and include text from input in output. Also use append before trim, "Junk=" and "Good=" in their respective boxes.

One likely error is having VDM tell you "Avisynth open Failure: The script's return value was not a video clip" If you see that you need to manually add either "Good" or "Junk" to the end of your avs script. Those last two lines are written with WriteFileEnd and WriteFileEnd isn't written until you close the video in VDM (ctrl-w), you close VDM, or refresh the script. If someone has a better way to write this so that this doesn't happen let me know.

Here's the template, use a text editor such as notepad to create TrimMaker.avst in the template folder that's located with your VirtualDubMod.exe file. Paste the following code into it.
#ASYNTHER TrimMaker
video="%f"
AviSource(video).converttoyv12

##################### Increments the name for the trimlog.txt #############
count = name(1,LeftStr(video,strlen(video)-4))
output=LeftStr(video,strlen(video)-4)+ string(count) + " TrimLog.txt"

function name(int count, string prefix)
{
return exist(prefix + string(count) + " TrimLog.txt")\
?name(count+1,prefix) : count
}
##################### Increments the name for the trimlog.txt #############

WriteFileStart(output, """ "AviSource(" """, "chr(34)","video","chr(34)", """ ")" """)
WriteFileStart(output, """ "#ShowFrameNumber" """,append=true)

################## Searches for and writes junk frames #######################
WriteFileIf(output,"AverageChromaU()>200\
||(AverageLuma>215\
&&AverageLuma(DEdgeMask(last,thY1=140,thY2=140).expand())>1)"\
,"current_frame")
################## Searches for and writes junk frames #######################

WriteFileEnd(output, """ "Good" """)
WriteFileEnd(output, """ "#Junk" """)

Converttoyuy2 #Hopefully you can remove this last line to speed up processing.


I've written it with my favorite junk finding parameters which require masktools plugin for avisynth to be installed. I hope it's clear in the template above where to put your prefered junk parameters.

[esby]

mmmm
What is the exact usage of trim maker?

Trimming out bad part of a video?
Or replacing them by static content?

Because, i usually do trim to:
* Make AMV.
* do separate processing on some part of a video, and reassemble the whole after.
And for these two usage, I don't really need a trimMaker.
I am more asking out of curiosity than anything else.

esby

[tsp]

What is the exact usage of trim maker?

Trimming out bad part of a video?
Or replacing them by static content?

Mkanel and I use trimmaker to cut the bad parts out.
The program just converts a textfile with frame numbers to an avs script with trims that include and/or exclude those.
This is useful when you use an avisynth script to detect the bad or otherwise interesting frames and output these to a textfile.

* Make AMV.
What is AMV?

[esby]

mmmm I think I'm getting to get what you are doing.
You are working with footage right? And you don't really care if you 'lose' some frames, since you have no overall sync to keep due to subs or separate audio somewhere, if I ain't wrong.


Animated Music Video.

Taking some parts of anime/video and mixing them together with various effects and with a choiced music.
Try to look on ermac page, you'll find a selection of his amv,
http://www.ermacstudios.org/


esby

[mkanel]

@esby

I've been capturing analog tape to my PC, often there are gaps in the tape where nothing is recorded, these show up as static or a blue screen. These gaps are often several seconds long but can be as little as one frame. It's a pain to search for these so I use avisynth to find these bad spots and write their frame numbers to a file. Trimmaker is a utility to assemble all these frame numbers into trim commands for avisynth. Avisynth can roughly provide the services of trimmaker (this thread (http://forum.doom9.org/showthread.php?s=&threadid=82236)) but trimmaker is easier to use and better. There might be other uses for this but I don't know of any. Any frame that you can define as good or bad for avisynth could be found and processed this way. For me this utility is very useful but if you're not capturing tapes it's probably useless to you.

I hope that helps clear things up but if I've just made things less clear let me know and I'll try again.

Mike.

[tsp]

Here is a faster script to detect statics in analog captures. It has been tested with vhs captures.


input="c:\test.avi"
logfile="c:\testlog.txt"
source=AVISource(input).AssumeFPS(100).converttoyv12.Kerneldeint(1,threshold=10)#Change this to your favorit deinterlacer

global Fmask = DEdgeMask(source,thY1=79,thY2=79,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
global Pmask = source.crop(source.width/2-12,0,24,source.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
source
WriteFileIf(last, logfile,\
"AverageLuma(Pmask)>0.001 && AverageLuma(Fmask)>0.05 && (AverageLuma(Fmask.trim(current_frame-1,-1))>0.05||UDifferenceFromPrevious()>2)"
\, "current_frame")

[mkanel]

Nice work on the static detection tsp. I'm now using your script but because my tape player hangers static to blue after a few frames I add my blue frame detector to your script.input="c:\test.avi"
logfile="c:\testlog.txt"
source=AVISource(input).AssumeFPS(100).converttoyv12.Kerneldeint(1,threshold=10)#Change this to your favorite deinterlacer

global Fmask = DEdgeMask(source,thY1=79,thY2=79,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
global Pmask = source.crop(source.width/2-12,0,24,source.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
source
WriteFileIf(last, logfile,\
"AverageChromaU(source)>200 || AverageLuma(Pmask)>0.001 && AverageLuma(Fmask)>0.05 && (AverageLuma(Fmask.trim(current_frame-1,-1))>0.05||UDifferenceFromPrevious()>2)"
\, "current_frame")


For a speed improvement I add two tweaks. Instead of deinterlacing I just throw out the even (or odd) fields. The second tweak is a little odd in that you no longer view the video to be processed but instead see a couple of blank rectangles but it does help the speed. The changes are below in red

.input="c:\test.avi"
logfile="c:\testlog.txt"
source=AVISource(input).AssumeFPS(100).converttoyv12.bob.selectodd.converttoyv12#Change this to your favorite deinterlacer

global Fmask = DEdgeMask(source,thY1=79,thY2=79,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
global Pmask = source.crop(source.width/2-12,0,24,source.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
version.blankclip(framecount(source))
WriteFileIf(last, logfile,\
"AverageChromaU(source)>200 || AverageLuma(Pmask)>0.001 && AverageLuma(Fmask)>0.05 && (AverageLuma(Fmask.trim(current_frame-1,-1))>0.05||UDifferenceFromPrevious()>2)"
\, "current_frame")

You get a little extra speed by turning off preview in VirtualDubMod (F9 and F10). There's nothing to see after these tweaks anyway.

One more feature request tsp, would it be possible to have Trimmaker remember the previously used setting, such as which boxes are checked and what variable names the trims are assigned to?

[tsp]

version 0.06 released with the suggested features added.

mkanel if you throw out one of the fields remember to lower the threshold for the averageluma.

Another good matrix(maybe slight faster than the last one used) to use for DEdgemask is this:

global Fmask = DEdgeMask(source,thY1=79,thY2=79,thC1=140,thC2=140,matrix="-1 -1 -1 0 4 0 0 -1 0" )

[mkanel]

tsp,

I just noticed one more small bug, when adding an offset to cut X number of frames after bad frames the offset is properly used with the Good trims but ignored in writing the Junk trims.

Thanks for the last changes. Mike.

[ADLANCAS]

Nice work tsp and mkanel!
I've just made a very quick test and it works good.

When I have time (I hope soon) I'll test more in deep.
I don't know if I got it, but maybe could be added all .avs examples (static scenes, blue screen, others) that can be used to generate text file, just to make the things easier.

Alexandre

[mkanel]

@ADLANCAS,

I'm glad it worked for you. I don't have a place to post images of static or blue screen. I'm attaching a jpeg here but in the past my attachments haven't shown up in the post so I don't know if this will work. You can also see tsp's previously posted examples of static, his tape player doesn't output blue screen. tsp's example of static (http://www.tsp.person.dk/frame1.jpg)

I don't know which static detection parameters you used, I wish I could disown my post 4 posts previous, it was fast when I tested it on short videos but became slower and slower on a long (2 hour video). I have new parameters that are even faster and so far much more accurate. I'm using tsp's basic idea of testing just a small vertical slice of the video for static to speed the process and then if static is suspected running the test on the full frame. The 3 main changes from tsp's parameters are 1) I also need to check for blue screen. 2) This script only displays the small vertical slice of the frame to speed the script. 3) I basically rotate his matrix 90 degrees and test for vertical lines, I found that testing for horizontal lines occasionally triggered a false positive for junk frames in high motion scenes due to interlacing artifacts. I'm finally starting to understand tsp's method of using masktools (not just copying his work) and I'm more confident in this script, I may again be proven wrong again but the thresholds seem good, static frames are well above the threshold to trigger detection and good frames fall well below the threshold to weed out false positives.

I would recommend trying tsp's parameters 5 posts previous to this one or my new parameters here: s=AVISource("C:\Some.avi")
partial=s.crop(width(s)/2-8,0,32,height(s)).converttoyv12.separatefields.selecteven.assumefps(50)
FullMask=s.converttoyv12.separatefields.selecteven.assumefps(100)\
.DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-1 2 -1 -1 2 -1 -1 2 -1" )
partial
output="C:\log.txt"
PartialMask = DEdgeMask(Partial,thY1=45,thY2=45,thC1=140,thC2=140,matrix="-1 2 -1 -1 2 -1 -1 2 -1" )
WriteFileIf(last, output,\
"AverageChromaU(partial)>200||(AverageLuma(PartialMask)>.1 && AverageLuma(FullMask)>.1)"
\, "current_frame",append=false)

#ScriptClip("Subtitle ((String(averageluma(PartialMask))),x=0,size=12) ")
#ScriptClip("Subtitle ((String(averageluma(FullMask))),x=0,y=25,size=12) ")

[tsp]

mkanel don't you avoid the false positive frames caused by interlacing when you only use one field? Also your noise are really clean and in the first frame it seems as the white lines are vertical but my source is from 1982 on an vhs-tape causing the noise to be more noisy. I will have to test the vertical edge detect to see if it can find the statics in my source.

[mkanel]

Originally posted by tsp
mkanel don't you avoid the false positive frames caused by interlacing when you only use one field?

I was surprised to see the occasional false positive with only one field also. I get far fewer false positives with one field than with two. I think you get the occasional narrow horizontal line when you remove one field thereby joining two same field lines that were previously slightly separated in space. With two fields you get adjacent horizontal lines that were produced at at slightly different times. I could probably also cure this by slightly tweaking the settings for horizontal line detection but at least for my source it seemed more sure to just check for vertical lines.

Mike.