TrimSubs - Script to cut text subtitles + AvsP macros

[vdcrim]

This Python script parses a specified Avisynth script for a line with uncommented Trims, and cuts an input text subtitle file accordingly so the subtitles match the trimmed video. It's intended as a companion to vfr.py, script that creates chapter files based on Trims and that also can cut audio files. It can be specially useful for Matroska ordered chapters.

There are three ways of specifying the line of the avs used:
· Parse the avs from top to bottom (default) or vice versa, and use the first line with Trims found.
· Use a line with a specific comment at the end, e.g: Trim(0,99)++Trim(200,499) # cuts
· Directly specifying the Trims line number, starting with 1.

A frame rate or timecode file (v1 or v2) is required, except for MicroDVD subtitles. A new trimmed timecode v2 file can be generated optionally.

Supported subtitle formats: ASS, SSA, SRT, SUB (MicroDVD).

usage: TrimSubs.py script.avs                                                            
                   [-h [{doc}]] [-V] [-v] [-r] [-l LABEL] [-g LINE] [-f FPS]             
                   [-t [OTC]] [-i [INPUT]] [-c ENCODING] [-o OUTPUT]  

Info arguments:
  -h [{doc}], --help [{doc}]
                        Show a help message and exit. Add 'doc' to 
                        include also the documentation
  -V, --version         Show program's version number and exit

Required arguments:
  script.avs            Avisynth script containing Trims

Optional arguments:
  (--input or --otc parameter is required)

  -v, --verbose         Show detailed info
  -r, --reversed        Parse the avs from bottom to top
  -l LABEL, --label LABEL
                        Use the Trims from the line in the avs that ends in 
                        a commentary with LABEL
  -g LINE, --line LINE  Use the Trims from the line nº LINE
  -f FPS, --fps FPS     Frame rate or timecode file (v1 or v2). If omitted,
                        search for a timecode file or default to 24000/1001
  -t [OTC], --otc [OTC]
                        Output a new timecode file. Path optional
  -i [INPUT], --input [INPUT]
                        Input subtitle file. If INPUT is not specified, 
                        search for a valid input file
  -c ENCODING, --encoding ENCODING
                        Input subtitle file encoding
  -o OUTPUT, --output OUTPUT
                        Custom path for the output subtitle file

TrimSubs.py (Python script, needs Python 3.2 and PySubs)
TrimSubs.exe (Windows executable, without dependencies)
TrimSubs repository


I've made some AvsP macros to easy the use of vfr.py and TrimSubs. These links may be out of date, latest version is always here:

· AvsP macro - Insert and format Trims

Contains the following macros:
· Insert Trims from bookmarks (single-line)
· Insert Trims from bookmarks (multi-line)
· Format MeGUI Trims (single-line)
· Format MeGUI Trims (multi-line)

· AvsP macro - Insert Trims from Matroska chapter file

This can be useful to redo a previously non-ordered chapters encode to ordered, if the original avs is no longer available. Currently only constant frame rate is supported.

· AvsP macro - Create/join timecodes from Trims

Assign a FPS or timecode to every Trim in a specified line and create a new timecode that can span all the video range or only the Trims. It's meant to be used when obtaining a VFR video as a result of deinterlacing/IVTC various sections of the video in different ways, or joining several videos of different FPS.

. AvsP macro - Trim timecode

This functionality is already included on TrimSubs, but it's useful to have it as a stand-alone macro as well.

· AvsP macro - Matroska chapters from Trims (needs vfr.py and TrimSubs)

Wrapper for vfr.py and TrimSubs.py. Generate Matroska chapters from Avisynth Trims. Additionally cut audio and text subtitles to match the trimmed video.

· AvsP macro - Chapter names from file

Replace default chapter names in the Matroska chapter files on a given directory with the ones provided in a specified file, one chapter name per line.

The macros must be put in your "AvsPmod\macros" directory. Note that the order of appearance in the macro menu can be customized by prefixing "[number]" to the filenames. Please read the complete info and change (if desired) the preferences on the first lines of each script.


Changelog:

2011-12-30: initial release
2012-01-29: see post #2

[vdcrim]

TrimSubs updated to 0.2:
· Added --line parameter
· Support for negative second member of the Trim pair

'Matroska chapters from Trims' updated to v2:
· Added custom parsing order and label feature. Needs vfr.py 0.8.6.1+

'Create-join timecodes from Trims' updated to v2:
· Support for negative second member of the Trim pair

Added new macros:
· Insert Trims from bookmarks (multi-line)
· Format MeGUI Trims (single-line)
· Format MeGUI Trims (multi-line)
· Insert Trims from Matroska chapter file
· Chapter names from file

[Yellow_]

hi, I wonder if you could suggest a way of doing something similar.

I'm using AVS2pipemod + Imagemagick making big files that are high on memory and disk space usage, the whole video has to be loaded before writing image sequences out with IM. I'm using IM as it's 16bit RGB from the Dither Plugin to 16bit half float EXR.

So I'm trying to code a python macro for AVSPmod to divide the video into chunks by trimming at the bookmarks set by intervals, but creating a new .avs file instead of a multiline / single line list in one .avs as per your scripts. I've manually created the .avs scripts previously.

Looking to get an .avs script for each bookmark so that I can batch process the avs files created through AVS2pipemod sequentially.

[vdcrim]

Originally Posted by Yellow_

Looking to get an .avs script for each bookmark so that I can batch process the avs files created through AVS2pipemod sequentially.

I assume you already have the bookmarks ready (probably by using the 'Bookmarks at Intervals' macro). Try this:


AvsP macro - Divide script [latest version]

'''
Divide an Avisynth script into multiple avs according to AvsP bookmarks.  
The first and last frame are automatically added to the bookmarks if not 
already present.
'''

# If True, save the new scripts to the same directory and with the same 
# name as the input avs (if exists).
use_same_avs_dir = False

# If True, save the scripts always to this location, e.g. ur"C:\Scripts", 
# using the following basename.  If basename is empty take it from the 
# input avs.  If there's not avs, prompt for a name:
use_avs_dir = False
avs_dir = ur""  
base_name = ur""


# ------------------------------------------------------------------------------


from os import makedirs
from os.path import isdir, basename, join
import codecs

bm_list = avsp.GetBookmarkList()
if not bm_list:
    avsp.MsgBox("There is not bookmarks", 'Error')
    return
avs = avsp.GetScriptFilename()
if not (use_same_avs_dir and avs):
    if use_avs_dir and avs_dir:
        if not isdir(avs_dir):
            makedirs(avs_dir)
        if not base_name:
            if avs:
                base_name = basename(avs)
            else:
                base_name = avsp.GetTextEntry('Introduce a basename for the new scripts', 
                                              'avs_trim', 'Divide script')
                if not base_name:
                    return
        avs = join(avs_dir, base_name)
    else:
        avs = avsp.GetSaveFilename('Select a directory and basename for the new scripts')
        if not avs:
            return
if avs.endswith('.avs'):
    avs = avs[:-4]

bm_list.sort()
if bm_list[0] != 0:
    bm_list[:0] = [0]
frame_count = avsp.GetVideoFramecount()
if bm_list[-1] == frame_count - 1:
    bm_list[-1] = frame_count
else:
    bm_list.append(frame_count)
digits = len(str(len(bm_list) - 1))
text = avsp.GetText()
avs_list = []
for i, bm in enumerate(bm_list[:-1]):
    avs_path = u'{}_{:0{}}.avs'.format(avs, i+1, digits)
    avs_list.append(avs_path)
    with codecs.open(avs_path, 'w', 'utf-8') as f:
        f.write(text + '\nTrim({},{})\n'.format(bm, bm_list[i+1] - 1))
return avs_list

[Yellow_]

ok, that's wicked, thank you so much.

I'd spent 5 hours on and off and got nothing like your code. My python skills or lack of, need serious improvement. :-)

I tested it with a 423 frame video ie: 0 - 422, bookmarks set by interval of 200 frames and recieved 3 .avs files, with trims (0, 199) , (200,399) and (400,420)

So I think I'm just loosing a few frames off the end?

I'm going to try and understand your code to see if I can resolve it, but could take a while. ;-)

[vdcrim]

Originally Posted by Yellow_

So I think I'm just loosing a few frames off the end?

Yep, last frame was missing. Fixed

[Yellow_]

many many thanks, hope this macro is useful to others too.

I'd like to update a post on my blog, http://www.http://blendervse.wordpre...d-linear-exrs/ from a while back, with your macro to resolve the problem I described, is that ok with you, if I credit you and link to this thread?

thanks again.

[vdcrim]

Sure. Nice blog btw, I think I've already stumbled upon it before.

[Yellow_]

Cheers. :-)

[Yellow_]

hi again, long time :-)

The script is great, but at the moment it adds the Trim() to the bottom of the script however, I wonder if it's possible for the trim(?,?) to be added at the cursor position, the reason I'd like to be able to trim a source before further processing so having control over where the trim is added would be great.

[vdcrim]

Try this new version (raw link). The latest version is always there. If the 'split at the current cursor position' option is used then the script is only evaluated until the line the cursor is on, so posterior filters that change the framecount don't affect the splitting.

Also, the list of created scripts is saved as a global variable avs_list. Besides creating a batch file you can also start the processing of these files with a macro like this:

# run macro in new thread

import os.path
import subprocess

for avs in avs_list:
    if subprocess.call(['program.exe', '--arg1', 'x', '--arg2', 'y', avs]):
        avsp.MsgBox(_('Error processing "{0}"').format(os.path.basename(avs)), 
                    _('Error'))
avsp.MsgBox(_('Done!'), _('Info'))

[Yellow_]

Great, thanks, I have a proble though, when I try using the Divide macro with either using the bookmarks or by specifying a frame step, I get a pause, then an error message 'Error loading the script". :-(

[vdcrim]

Originally Posted by Yellow_

Great, thanks, I have a proble though, when I try using the Divide macro with either using the bookmarks or by specifying a frame step, I get a pause, then an error message 'Error loading the script". :-(

The pause is the macro evaluating the script in order to get the total number of frames, and the message means that there's an error in the script. My best guess is that the last line of your script (or the line the cursor is on if 'split at the current cursor position' is checked) is an assignment. In that case the script doesn't have a return value. I'm going to append 'last' to the text that is evaluated to workaround this.

[Yellow_]

Hi, thanks, although the script I was testing on had heavy processing further down, I'd thrown a return last in immediately after the initial ffmpegsource2(...) in order to test the trim addition without incurring heavy processing section of the script at this time. So correct in your analysis.

many thanks, will try again.

[vdcrim]

Instead of just appending 'last' I added a new option to always use the last evaluated expression, even if it's assigned to a variable. Also fixed the text that is evaluated when the cursor is on a multiline comment.

[Yellow_]

hi, I'm still not getting anywhere, every option causes error can't load script. :-( I must be doing something wrong.

Here's my script:

LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\mvtools2.dll")
LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\fft3dfilter.dll")
LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\removegrain.dll")
LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")
LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\dfttest.dll")

SetMemoryMax(512)

source=ffmpegsource2("videofile.mkv", threads=1)

denoised=MCTD(dfttest(source, sigma=4.0, lsb=true), chroma=true, settings="low")

Dither_convert_yuv_to_rgb(denoised, matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709")
Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )

I'd like to create multiple avs files of the above with a trim(?,?) added to the end of the line:

source=ffmpegsource2("videofile.mkv", threads=1).trim(?,?)

For each bookmark set or interval set etc.

The reason for this is that I intend piping batches of around 350 to 500 16bit interleaved frames from thousands via avs2yuv to imagemagick for encoding to 16bit linear exr's, any more than about 500 frames at a time will cause memory errors on the machine I'm using for this currently. I'm manually adding trim command and trim points to multiple avs scripts currently and would like to automate it.

The problem with avs2yuv type route I believe is that all frames must be accounted for before processing starts and this is simply too many for memory + temp harddisk space, so some sort of batching required.

But alternatively if it were possible to do this directly from AVSPmod in a single avs script in a similar way to 'save to mp4' from the tools menu by calling avs2yuv or avs2pipemod and feeding batches of so many frames directly from AVSPmod negating need for multiple scripts, would this be possible because I find the more complex the processing required, the smaller the batch increments and therefore increased number of batch scripts needed to prevent memory errors.

Thanks for your time making changes to date.

[vdcrim]

Originally Posted by Yellow_

every option causes error can't load script

You have to put the cursor on the 'source=...' line and then use the macro checking both 'Split at the current cursor position' and '... using the last evaluated expression' options. Works for me with your script. However the generated scripts weren't correct, that's fixed now.

The problem with avs2yuv type route I believe is that all frames must be accounted for before processing starts and this is simply too many for memory + temp harddisk space, so some sort of batching required.

But alternatively if it were possible to do this directly from AVSPmod in a single avs script in a similar way to 'save to mp4' from the tools menu by calling avs2yuv or avs2pipemod and feeding batches of so many frames directly from AVSPmod negating need for multiple scripts, would this be possible because I find the more complex the processing required, the smaller the batch increments and therefore increased number of batch scripts needed to prevent memory errors.

The 'save to mp4' tool is in fact a general script encoder that executes a batch file generated from a template file. It won't give much benefit over just writing yourself the batch, and you would still have to create the trimmed scripts. What you can do is use a macro to write your batch file, or just use the snippet I posted above instead of the batch.

Another way is to pipe directly from AvsPmod to ImageMagick. In that case all could be done from a macro, without creating temporary avs scripts and a batch file. I already made something like that on this Create GIF with ImageMagick macro. Of course the disk space/memory issue would still be present, so the data shouldn't be piped in just one go like in that macro. Send me your command line and I'll take a look, but I can't promise you anything. EDIT: just found out that there's an example on the Dither docs, should be enough.

[Yellow_]

Thanks for your reply and suggestions, leave it with you if you get time, just wanted to say the command line in the Dither docs was provided by myself so its the one I would have provided had I responded before your edit. :-)

Thanks again.

[vdcrim]

I just posted a macro script on the AvsPmod thread. Let's keep the discussion there.

[real.finder]

hi, first thank you for wonderful macros

I wonder if you could made macro that create x264 qpfile from the mkv chapters with support v1 & v2 timecodes and cfr too