Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th August 2009, 08:21   #41  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Quote:
Originally Posted by Gavino View Post
Assuming the timecodes in your subtitle file are relative to the original clip, then TextSub should come before Trim.
Wow, that's just clever!

I'm starting to get the hang of this, thanks very much for your help.
fvisagie is offline   Reply With Quote
Old 15th August 2009, 16:17   #42  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Threshold suppresses only rec_time output

The DVInfo doc says:

Quote:
With "threshold">0 the output is switched on if there is a difference of the recording date between current and last frame which is bigger than "threshold" seconds. After "autoframes" frames the output is switched off. That way you get the recording date only at scenechanges.
Similarly to tacman123 I'm trying to write a CSV file that logs current_frame and rec_time, once for each change in rec_time.

I'm using this statement:
Code:
DVInfo(input_file, "current_frame + chr(44) + chr(32) + rec_time", threshold=0.5, autoframes=1)
However, there's output on every frame, even when rec_time doesn't change. The output string in such cases effectively consists of an empty rec_time, e.g.:
Code:
32,
fvisagie is offline   Reply With Quote
Old 15th August 2009, 17:49   #43  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
In my view that's a bug, or at best you could say the documentation is highly misleading.

Instead of the output being 'switched off', it simply sets rec_time to an empty string. (I've checked the source code.)
That's fine in the default case (where the output consists solely of rec_time), but not in the general case.

Here's a potential workaround:
Code:
DVInfo(input_file, """rec_time != "" ? current_frame+chr(44)+chr(32)+rec_time : "" """, threshold=0.5, autoframes=1))

Last edited by Gavino; 15th August 2009 at 19:10. Reason: Simplified workaround - ScriptClip unnecessary
Gavino is offline   Reply With Quote
Old 17th August 2009, 09:17   #44  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Ah, that's what I've been trying to get working (without ScriptClip), thanks .
fvisagie is offline   Reply With Quote
Old 21st August 2009, 12:07   #45  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
DVInfo corrupting script variable?

Hi All,

I've started setting up the following framework to track timecode changes across frames (note the assigment of tc_time to old_tc):

Code:
input_file = "Test with sub_type2.avi"

#
# Script execution section
#

# "Constant" declarations
output_suffix = ".timecodes.txt"
output_file = string(input_file + output_suffix)
field_separator = ", "


OpenDMLSource (input_file)

WriteFileStart (output_file, """ "current_frame" + field_separator + "tc_time" + field_separator + "old_tc" + field_separator + "next_tc" """)

# Top-level script scope variable declarations
debug = false
next_tc = "59:59:59.99"
old_tc = "59:59:59.99"

#
# Runtime execution section
#

WriteFile (output_file, "string(current_frame) + field_separator + string(tc_time) + field_separator + string(old_tc) + field_separator + string(next_tc)")
ScriptClip("""
	next_tc = old_tc
#	old_tc = LeftStr(tc_time, 6) + RightStr(tc_time, 5)
	old_tc = tc_time
	debug ? Subtitle(string(current_frame) + field_separator + string(tc_time) + field_separator + string(old_tc) + field_separator + string(next_tc)) : last
	""")
DVInfo (input_file, """ "" """)
The text file output is:

Quote:
current_frame, tc_time, old_tc, next_tc
0, 00:55:48-08, 00:55:48-08, 59:59:59.99
1, 00:55:48-09, 00:55:48-09, D~ŽB~ŽB~ѬB~(
2, 00:55:48-10, 00:55:48-10, 00:55:48-10
3, 00:55:48-11, 00:55:48-11, 00:55:48-11
4, 00:55:48-12, 00:55:48-12, 00:55:48-12
5, 00:55:48-13, 00:55:48-13, 00:55:48-13
6, 00:55:48-14, 00:55:48-14, 00:55:48-14
7, 00:55:48-15, 00:55:48-15, 00:55:48-15
old_tc (which is assigned to next_tc before the latter is printed) is ALWAYS corrupted on the second frame, and incorrectly overwritten to the value of the current timecode on subsequent ones. I thought an intermediate variable would help protect old_tc:

Code:
	next_tc = old_tc
	temp_tc = tc_time
	old_tc = temp_tc
but it seems AVS assigns strings by reference and this doesn't help.

From the source it seems DVInfo assigns its runtime variables by reference. If I forcibly create a copy of tc_time in old_tc as follows:

Code:
	old_tc = LeftStr(tc_time, 6) + RightStr(tc_time, 5)
#	old_tc = tc_time
old_tc isn't corrupted, and in that case the text file output is:

Quote:
current_frame, tc_time, old_tc, next_tc
0, 00:55:48-08, 00:55:48-08, 59:59:59.99
1, 00:55:48-09, 00:55:48-09, 00:55:48-08
2, 00:55:48-10, 00:55:48-10, 00:55:48-09
3, 00:55:48-11, 00:55:48-11, 00:55:48-10
4, 00:55:48-12, 00:55:48-12, 00:55:48-11
5, 00:55:48-13, 00:55:48-13, 00:55:48-12
6, 00:55:48-14, 00:55:48-14, 00:55:48-13
7, 00:55:48-15, 00:55:48-15, 00:55:48-14
as would be expected.

Is it correct to deduce that in the first code block DVInfo retains a pointer to old_tc and corrupts it between the first two frames and incorrectly overwrites it subsequently?
If so, is there someone willing and able to do a quick fix of DVInfo for me? In the meantime is there a computationally cheaper way of creating my own local copy of tc_time?

Many thanks,
Francois

Last edited by fvisagie; 21st August 2009 at 12:11.
fvisagie is offline   Reply With Quote
Old 21st August 2009, 17:55   #46  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by fvisagie View Post
Is it correct to deduce that in the first code block DVInfo retains a pointer to old_tc and corrupts it between the first two frames and incorrectly overwrites it subsequently?
Not exactly, but something similar.
What is happening is that DVInfo (in its GetFrame function) uses local stack space for the strings, which is thus only in scope during the fetching of a single frame. When you save tc_time in old_tc, and then try to use it during the next frame, you end up with a reference that is no longer valid and has been potentially overwritten by something else.

Now, that something else will usually be the new tc_time, since the new GetFrame will end up with the same stack address if called in a loop by Avisynth's client application. That you only get 'garbage' on frame 1 is probably due to the very first GetFrame (for frame 0) being called differently by the application.

Is this a bug or is it a feature?
I would argue that DVInfo should not use local storage for its run-time variables. In nearly all cases, local storage will suffice but a problem arises in your case where you have a run-time script with side-effects (something which must be done carefully, but I believe is perfectly 'legal').
Quote:
If so, is there someone willing and able to do a quick fix of DVInfo for me? In the meantime is there a computationally cheaper way of creating my own local copy of tc_time?
If necessary, I could have a go, but it would be better if the original author could address this (and the other bug you reported in post #42) - are you still out there, WarpEnterprises?

In the meantime your workaround is a valid one.
More efficient would be just old_tc = LeftStr(tc_time, 11).
Gavino is offline   Reply With Quote
Old 22nd August 2009, 13:15   #47  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Quote:
Originally Posted by Gavino View Post
More efficient would be just old_tc = LeftStr(tc_time, 11).
Ah, excellent, thanks very much!

The script is basically complete (bar that improvement and some final testing), and I'm looking forward to posting the results soon!
fvisagie is offline   Reply With Quote
Old 22nd August 2009, 14:11   #48  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
If you're concerned about speed, and you are only interested in the file output, so don't need to see the video result, you could replace DVInfo(input_file, """ "" """) by

DVInfo (Crop(0,0,2,2), input_file, """ "" """)

DVinfo is quite slow, since it creates a new instance of Subtitle for every frame (even with an empty string!). Because of the way Subtitle works, it is substantially faster if given a smaller frame.
The speed-up will be much more significant than tweaking the expressions in your script (even in the run-time part).
Gavino is offline   Reply With Quote
Old 22nd August 2009, 15:14   #49  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Quote:
Originally Posted by Gavino View Post
If you're concerned about speed, and you are only interested in the file output, so don't need to see the video result, you could replace DVInfo(input_file, """ "" """) by

DVInfo (Crop(0,0,2,2), input_file, """ "" """)
That makes sense, thanks - I couldn't get the previous similar suggestion to work without Subtitle(). This shows why..
fvisagie is offline   Reply With Quote
Old 22nd August 2009, 15:39   #50  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by fvisagie View Post
I couldn't get the previous similar suggestion to work without Subtitle().
My previous suggestion of
Code:
StackVertical(Crop(0, 0, 0, 20).DVInfo(file, ...), Crop(0, 20, 0, 0))
should also work and speed things up when you want to get the video output too. Unless you are using additional options to position or size the text differently, in which case you would need to adjust the '20' accordingly.
Gavino is offline   Reply With Quote
Old 22nd August 2009, 21:01   #51  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
tc2Scenes: timecode-based scene file generator

Hi All,

After much blood, sweat and tears, the timecode-based scene file generator script has been completed and so far it seems to work well.

What it's about:

Quote:
# tc2Scenes loads the specified DV AVI (Type 1 or 2) file and generates a scene file from timecode discontinuities.
# tc2Scenes works for unedited DV files, or edited files where the editor used direct mode or similar to preserve original timecodes in unmodified frames.
# For tc2Scenes to work correctly, its output must be rendered linearly by playing, encoding, whatever is more convenient.
# The scene file is generated in ASS v4.00+ subtitle script format, with one subtitle per scene.
# Subtitles start with the corresponding scene, and the DATECODE and timecode of the scene's first frame are written to their text.
# tc2Scenes has been tested on 25fps PAL material, but hopefully it was written generally enough to work for other integer frame rates.
# Currently tc2Scenes does not validate timecode values.

# Possible usage restrictions:
# AvsP: To improve performance this script produces highly cropped video. Trying to Fill or Fit its preview to the AvsP window will likely result in an error.
# Instead, either do not Fill or Fit the preview, or reduce the Crop at the bottom of the script.
The current version is 1.2 and contains the following changes:
Quote:
# 2009/08/28 1.2 Added output path command line option for batch file
# AVS script now receives full output filename from calling script instead of just suffix
# ASS style corrected from Default to SceneChange
# Shortened default subtitle duration to last frame below 0.5 s
# Some other minor changes
How to run it:
* As a stand-alone AVS script by editing it for the appropriate input and output file specifications; or
* via Import() from another script which sets the input and output file specifications for it; or
* From the attached batch file

The latter is quite convenient for launching the scene file generator via a Windows shell extension handler:
Quote:
1. Copy the attached script and tc2Scenes batch file to somewhere on your system and make sure the LOCATION environment variable in the batch file correctly points to the _script's_ location
2. In Windows Explorer, open Tools.Folder Options.File Types, in the Registered file types list select AVI and click Advanced
3. Click New, in the Action text box fill in "Generate scene file", in the "Application used..." box fill in the full path and name of tc2Scenes.bat and click OK on all dialog boxes
4. In Windows Explorer, right-click your AVI file and select "Generate scene file"
What it produces where:
The batch file creates a stub AVS script called tc2Scenes_stub.avs which sets input and output file specifications and loads tc2Scenes.avs with Import(). The batch file then invokes ffmpeg with the stub AVS script as input. tc2Scenes.avs outputs scene changes to an ASS subtitle script format text file. The batch file also logs start and end times to a processing time logfile.

When no output path is provided, the stub script is created in the current directory, and the output file and processing time logfile are created in the same folder and with the same base filename as the input file. When a valid output path is provided, all files are created in the specified folder with the base filename tc2Scenes.

Writing to a separate output path is sometimes convenient, and also provides a modest performance gain when the destination is on a different disk.

If you prefer to use an alternative to ffmpeg for rendering the script, either modify the batch file, or terminate ffmpeg when/if it starts running and render the stub script with your preferred player or editor instead.

You're free to use this as you see fit (or as you do not! ). If you do use it and come up with suggestions for improvement, kindly implement and share them with me? . The main ones I can think of at this point are:
* Validating timecode values read (shouldn't be too hard, I just didn't have time now)
* Considering adding DATECODE-based scene detection. Depending on what's needed, adding detection of DATECODE discontinuities may be essential. As it stands, tc2Scenes detects timecode discontinuities. These are usually introduced by the editor. However, on-camera scene cuts created by stopping and resuming recording have continuous timecodes. Any such cuts retained as is in the final project can only be identified by DATECODE discontinuity.

Kind regards,
Francois
Attached Files
File Type: txt tc2Scenes.avs.txt (6.7 KB, 137 views)
File Type: txt tc2Scenes.bat.txt (3.3 KB, 83 views)

Last edited by fvisagie; 28th August 2009 at 15:26. Reason: Updated for 1.2
fvisagie is offline   Reply With Quote
Old 29th August 2009, 16:39   #52  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
@WarpEnterprises: "error reading DV frame" with large VirtualDub files

Hi WarpEnterprises,

Congratulations on and thanks for a great utility (DVInfo)! Unfortunately I'm having a problem with it on the file that matters, the output of my project.

Results so far:
Quote:
* Works: Type 1 files I've tried
* Works: large (36GB) Type 2 files created by DVDate
* Works: small (0.5GB) Type 2 files created by VirtualDub
* Doesn't work: large (36GB) Type 2 file created by VirtualDub, for which DVInfo reports "error reading DV frame" on each frame.
Both small and large Type 2 VirtualDub files were created with version 1.9.6 in Direct stream copy mode (and the Cedocida v0.2.0 DV codec FWIW).

The only significant-looking difference I've discovered so far between the large Type 2 files created by DVDate and VirtualDub as reported by GSpot are:

~36GB Type 2 file created by DVDate:
Quote:
Note: 31.3 GB unneeded bytes at end of file
DV Type 2 AVI
Multipart OpenDML AVI (5 parts)
(255491 frames in first part, 0 frames follow)
~36GB Type file created by VirtualDub:
Quote:
Note: 30.0 GB unneeded bytes at end of file
DV Type 2 AVI
Multipart OpenDML AVI (3 parts)
(14044 frames in first part, 240558 frames follow)
There seem to be significant differences between the number of parts and between the number of frames in the first part, for roughly the same file size.

It seems a possible worst case work-around might be to convert the VirtualDub output to Type 1 and back with DVDate. Also I could play with different VirtualDub settings on larger and larger files to see if any work for DVInfo, although either of these this will take hours and I have very little time right now.

If you have any suggestion off the top of your head for configuring VirtualDub to create large Type 2 files in a format that DVInfo will understand, please, I would love to hear about it!

Many thanks,
Francois
fvisagie is offline   Reply With Quote
Old 29th August 2009, 20:12   #53  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
It turns out this problem appears as soon as the VirtualDub Type 2 output file size exceeds 2GB, which is its default threshold for OpenDML indexing.

It also turns out that it is possible to process a VirtualDub file >2GB in DVInfo, if it is first converted to Type 1 by DVDate. If DVDate is used to convert that back to another Type 2 file again, that is also readable by DVInfo as expected - no DVDate files have caused problems yet.

So that creates a potentially useful comparison - the original >2GB Type 2 file created by VirtualDub which DVInfo can't read, and the DVDate Type 2 version which it can. I attach the RIFF headers of both as reported by DVDate.

I have played with the AVI output-related settings in VirtualDub, but so far haven't succeeded in producing a file that DVInfo can read directly. I also attach the available VirtualDub settings. If there are any changes you think may help things work, please let me know!
Attached Images
 
Attached Files
File Type: txt DV AVI_type2 VirtualDub.txt (8.4 KB, 34 views)
File Type: txt DV AVI_type2 DVDate.txt (8.7 KB, 31 views)
fvisagie is offline   Reply With Quote
Old 2nd June 2010, 21:48   #54  |  Link
WarpEnterprises
C64
 
WarpEnterprises's Avatar
 
Join Date: Apr 2002
Location: Austria
Posts: 830
Maybe a little late - but here is a small update:
- really switching off the output
- fixed silly bug when working with some big type-2 AVIs
- moved the storage of the strings to class scope

http://avisynth.org/warpenterprises/...o_20100602.zip
WarpEnterprises is offline   Reply With Quote
Old 3rd June 2010, 08:34   #55  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 588
Quote:
Originally Posted by WarpEnterprises View Post
Maybe a little late
Not at all, and still very useful. I hope to update the scene file generator some time and this will come in very nicely, thanks!
fvisagie is offline   Reply With Quote
Old 13th July 2010, 20:48   #56  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,717
With the new version, VirtualDub crashes every time the script containing DVInfo is closed, pointing to ntdll.dll. I cannot test the previous version as I don't have it on my HD anymore

It can be reproduced with a simple script like this (source is already bobbed):
Code:
AVISource("w:\dv\bobbed\nauha1.avi")
video="w:\dv\combined\nauha1.avi"
SeparateFields()
SelectEvery(4,0,3)
Weave()
DVInfo(video,"rec_time",rec_format="%d-%m-%y %H:%M:%S",fix_format="%d",threshold=999999,autoframes=250,font="Arial",x=500,y=500,text_color=$FFFFFF,size=24)
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is online now   Reply With Quote
Old 29th September 2015, 08:19   #57  |  Link
an_private
Registered User
 
Join Date: Sep 2015
Posts: 1
Error found

Error in source was found - space for indexes was limited to 4029 (line 125 in avi.h). Normally, size of one chunk for indexes is up to 65536 byte. One index record used 8 bytes -> up to 8192 indexes can be in one chunk.
Corrected, after correction all files is working correctly.
https://drive.google.com/open?id=0B4...GlwYXZTQWV1c1k
an_private is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:30.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.