HDVSplit 0.75 - utility for capturing HDV with scene split by timestamp

[paviko]

Hi!
I've not found any free utility that is able to capture from HDV camcorders and split to scenes like it is common for DV camcorders. So here it is my approach to HDV capturing - HDVSplit. Utility that allows Windows users to capture and split their HDV footage to scenes.
To download and read "How To", look at my home page (http://strony.aster.pl/paviko).

TODO:
- Simple cutting of m2t files with GOP accuracy

Version changes:
Version 0.75 - 05.03.2006
- fixed crash on exit when MPEG2 decoder could not be found
- added detection of dropped frames/packets during capture
- added elapsed time of capturing session
Version 0.7 - 19.02.2006
- preview added (3rd party decoder needed, e.g. free ffdshow)
Version 0.55 - 05.02.2006
- saving user preferences to file
- possibility to choose date and time format
Version 0.53 - 25.01.2006
- fixed serious bug with line artifacts in frames introduced in 0.51
Version 0.51 - 08.01.2006
- added option to synchronize audio and video (thanks to domain for pointing it out!)
Version 0.5 - 31.12.2005
- new feature: direct capture from camcorder
- fixed incorrect split when non-video packet appeared during search for split
- hack for strange behaviour (noticed with HC1) - sometimes wrong timestamp is written to tape ?!
Version 0.1 alpha - 10.12.2005
- first release

Regards

[bb]

This is good news for the HDV camp. Unfortunately I don't own an HDV camcorder, so I cannot test your program. Maybe someone else can post his / her experience?

bb

[paviko]

New version is out.

Version 0.51 - 08.01.2006
- added option to synchronize audio and video (thanks to domain for pointing it out!)

Not much interest here, but I'll keep posting info about new version. Maybe HDV will be more common later. Other threads related with HDVSplit:
http://www.sonyhdvinfo.com/showthread.php?t=3577
http://www.sonyhdvinfo.com/showthread.php?t=3728

[TachyonEagle]

thanks! great free utility!
v 0.52 works flawlessly with sony hdr-fx1e ;)

update : and now it seems that even with sony hvr-z1e , version 0.75 seems just a little bit confused when tape position is just after dv part has ended and there is yet no hdv timecode, anyway capture seems ok

[paviko]

Version 0.53 - 25.01.2006
Fixed serious bug with line artifacts in frames introduced in 0.51
In version 0.51 I've decreased internal buffer from 10MB to 4MB. Unfortunately it caused that some frames have artifacts, because some packets were lost during capturing.
Internal buffer is not used when capturing without scene split or when scenes are split from file. So this functions were not affected.

I really apologize for this bug :( In version 0.53 size of the buffer has been increased back to the size from 0.5 version.

Regards

[bb]

[...]I really apologize for this bug :( [...]
Such is life - no need to apologize. We have to be thankful that you share your program with us - even for free!

bb

[paviko]

Version 0.55 - 05.02.2006
With same simple new features:
- saving user preferences to file
- possibility to choose date and time format

Regards

[paviko]

Version 0.7 - 19.02.2006
- preview added (3rd party decoder needed, e.g. free ffdshow)

Preview will only work if there is MPEG2 decoder available. ffdshow - is very good, fast, great quality free decoder. Instruction how to configure it is at my homepage.
I don't recommend previewing if you don't have dual core system. It's very easy to lost packets transfered between camcorder and computer.

Regards

[Backwoods]

Very nice, thank you for the constant updates.

[krisq]

Superb stuff paviko, thank you :D

[paviko]

Version 0.75 - 05.03.2006
- fixed crash on exit when MPEG2 decoder could not be found
- added detection of dropped frames/packets during capture
- added elapsed time of capturing session

I hope new feature will be usefull for you as it is for me. No auto recapture of scenes with drops yet, but at least there is no need to watch and search for artifacts that came during false transfer. If they occur they are displayed in HDVSplit and for now manually you can recapture the tape/scene.
There is also one option - "Do not treat mid-scenes as dropped frames". If it is selected (default), then heuristic is used to not accidentialy treat undecodable fragments as dropped frames. This undecodable fragments can be caused by reusing tapes - mix of old scenes and new scenes.
The heuristic is not available when scene split is disabled, because there is no information about scenes.

Regards

[PrimeHD]

Hi Paviko,

Thank you for the fantastic utility, HDV Split.

I use it to capture HDV over firewire from my Sony Z1U, while monitoring the capture full-screen in real time with your HDVSplit.

One thing is missing though - waveform monitor.

Of course, DV Rack folks provides HDV upgrade and it does have waveform and all the bells and whistles. But these bells & whistles are costly, and a total overkill in my case.

I was wondering if you have a version of HDV Split with waveform monitor built-in?

[paviko]

Hi Alex.

I have never been using waveform monitor. After looking at wiki (http://en.wikipedia.org/wiki/Waveform_monitor), implementing it needs quite a lot of time.

Regards

[agilpwc]

Great program, used it this weekend to record direct to disc several hours of footage at a dog show.

Is it possible to get the source code?
I'd liked to make a couple simple improvements for myself or if not maybe you could add the features in?

1) option for a pause button that would stop recording, then when you hit it again, continue recording to the same file
2) an option for when you hit stop, it would automatically increment the name of the file, so you can hit record again without having to manually rename the output file.

Thanks Eric
agilityvideo at gmail dot com

[agilpwc]

You can add JVC GR-HD1 to your list of supported camcorders.

[agilpwc]

Feature request:

For HDV Split to start and stop recording when the record button is pushed on the camera and make them separate files.

Regards
Eric

[paviko]

Hi agilpwc.
1) option for a pause button that would stop recording, then when you hit it again, continue recording to the same fileGood point.
2) an option for when you hit stop, it would automatically increment the name of the file, so you can hit record again without having to manually rename the output file.
I think it can be done now when you select scene detection and choose count (with radio buttons) then every time you start new recording file has added count number.
For HDV Split to start and stop recording when the record button is pushed on the camera and make them separate files.That's quite interesting. I think it can be done. What you mean in second part of sentence "and make them separate files"?
You can add JVC GR-HD1 to your list of supported camcorders.
Thanks a lot for checking with JVC. Is scene splitting working or just capturing ?

Regards

[agilpwc]

Hi
On the JVC only capture works, not scene splitting or preview.

You were asking about the second part of the sentence 'to separate files'. I just meant that each push of record and stop on the camera would result in a separate clip and thus file on your harddrive.

Another interesting possibility, is in Virtual Dub capture tools, it has camera controls. You hit record in virtual dub and it activates the camera through firewire and starts recording on tape in the camera.

I would think somewhere in that same code would be the capability to reverse that and record on the computer when the record button on the camera is pushed.

I know that a signal is sent from the camera through firewire each time record is hit. As long as there is a tape in the camera.

Thanks for a very useful program.

[paviko]

Thank you agilpwc for information and suggestions. It's possible to check the status of camcorder so if it's recording you can also start recording automatically at computer. I think I've read in MSDN documentation that some signals are send when camcorder status has changes so looks like this feature is possible.

Regards

[Backwoods]

Would it be possible to include stop motion capture? Such as every click of the mouse creates a new frame? There doesn't seem to be any programs that can see my Sony FX1 and can do this.

[WBL]

Hello,

great tool for capturing!
I have a request: would it be possible to output to PS (mpeg) instead of TS (m2t) ?This way one does not need to use another program to convert the clips to mpeg files so these files are usable in other programs like vdub.

If such an implemetation is not easy to do does someone know a good converter which can do batch conversion as well and does not mess up the mpeg files like capdvhs does ?

thanks
Werner

[Backwoods]

Look into HDTVtoMPEG2, I'm pretty sure it can batch process.

[WBL]

no, HDTVtoMPEG2 cannot do batch process - it can load multiple files, but can only output to one file - in fact HDTVtoMPEG2 joins all clips to one file.

greetings
Werner

[Korben]

Hi Paviko
you can only capture with HDVSplit, but the splitting doesnt work.

[Backwoods]

What type of error are you getting? I've used the scene split feature without problems. Also are you trying to cap and scene split or scene split an already captured file?

[Korben]

For capturing I used "Scenes split" and "Synchronize audio and video", also the option "add to file name" is set on "Date and time".
My camera is detected properly, but the scene splitting doesnt work, the filename is test-noTime-0000.m2t.
Any ideas?

[Backwoods]

Hmm, what is the exact error and what camera are you using?

[paviko]

My camera is detected properly, but the scene splitting doesnt work, the filename is test-noTime-0000.m2t.
Any ideas?
Hello Korben. I was lazy and HDV parameters have been harcoded by me for Sony camcorders. For JVC it's not finding timestamp. In free time I'll change the code to be more universal.

Regards

[WBL]

Hello Paviko,

is it possible to add a little feature: when splitting to single scenes and using the counter it would be great if the couter itself is not located at the end of the filename it should be possible to place it within the filename. Example: when choosing the filename "video_####_name" HDVsplit should use the "#" and replace them by the counter so this gives files like:
video_0000_name.m2t
video_0001_name.m2t
and so on
The number of "#" written in the filename could also be used to define how many numbers are used for the counter. example: "video_##_name" gives a counter from 00 to 99 whereas "video_###_name" from 000 to 999

thanks in advance
Werner

[Korben]

Hi Paviko,
I am using JVC GR-HD1.

[billou2k]

hi! Has it been tested on the Sony hc3?

[Richard Iredale]

Yes, I think someone posted that it worked with the HC3 over on the other forum.

[paviko]

@Korben
I'll try add support for JVC and Canon in free time.

@billou2k
I have not tested it myself with HC3. But it's so long since HC3 release and noone reported that it's not working with HC3.

Regards

[Backwoods]

I can confirm the HC3 works with HDVSplit.

Paviko, any chance of HDVSplit being able to capture one frame at a time from cameras? For stop motion purposes. Such as, each time you click a button a frame will be recorded. Thanks.

[redindian]

does it work with JVC HD100? (HDV 720p/24/30)

[documentaryman]

Thanks for a great utility. But when importing captured files into Premiere Pro, I noticed that the source timecode was missing. This is a problem for editing. Is it very difficult to implement?

David

[tomos]

just started using this today. worked perfectly on my new sony HD cam.

thanks paviko :)

[Korben]

Thanks a lot Paviko.

[BabaG]

paviko,

great utility. one major thing missing for me and those i'm working
with: timecode. when i split the files from an existing master file i
lose the timecode. this would be a very valuable feature to those
of us in a multi user environment, or even for things like batch
capture. if i had timecode i could place all of the split files on a
timeline and create an edl which would double as a batch capture
list. that way i could easily delete files i think i might not need but
recapture them if i was wrong. and i could avoid having to capture
the entire tape again. i could just recapture the individual clips.
no timecode means i can't do this. it also means i can't make an
edl anyone else can use. please consider this. it would be a
great enhancement to an already great little app.

thanks,
BabaG

[jesus2099]

Hello Paviko!
I like the interface and simplicity of your soft very much.
I am a user of standard plain DV.

Would you think it would be difficult to make your tool compatible with DV ?

It would be so nice. :)

[paviko]

@Backwoods
Paviko, any chance of HDVSplit being able to capture one frame at a time from cameras? For stop motion purposes. Such as, each time you click a button a frame will be recorded.
I'm thinking about such feature :)

@redindian
does it work with JVC HD100? (HDV 720p/24/30)HDVSplit should work with JDV HD100 but without scene split and preview, only capture.

@BabaG and @documentaryman
But when importing captured files into Premiere Pro, I noticed that the source timecode was missing.
Timecode, like timestamp, is stored in .m2t file, so with proper tool you can read the timecode from .m2t file captured with HDVSplit or CapDVHS. So as I understand when you are capturing clips using e.g. Premiere it is storing timecodes, but in separate file and it's not able to read it from m2t file, right ? Or maybe you've meant something else ?
... if i had timecode i could place all of the split files on a
timeline and create an edl which would double as a batch capture
list. ...BabaG, you want HDVSplit to generate EDL file, e.g. CMX 3600 compilant ? I'm a bit lost, because how would you like to have timecode otherwise, in file name ?

@jesus2099
Would you think it would be difficult to make your tool compatible with DV ?
It would take some time to make it DV compatible and there are better program for DV already, also free - WinDV http://windv.mourek.cz/

Regards

[Backwoods]

Great to hear paviko, looking forward to it.

[BabaG]

thanks for the detailed response paviko. here's my situation with
regard to timecode.

i have a lot of full hdv mpegs captured mostly in premiere pro 2.0.
generally the timecode for the tapes was set specifically for each
tape. for example, tape 12 would start at hour 12, tape 13 at hour
13, etc. when i use hdvsplit to break the full-tape-file into its
individual scenes i find that all of the new files start at hour 0.
00;00;00;00 to be precise. are you saying that if we'd captured
these tapes using hdvsplit instead of premiere this would not
be the case? that would be great to know.

and as to the edl question, i was only using that as an example.
if all of my broken down files start at 00;00;00;00 i can't use
them to make a batch capture edl for premiere to recapture my
media. i have to capture the entire tape again and break it down
again. but if the broken down files retained their timecode such
as:

file-001.mpeg = 12;00;00;00
file-002.mpeg = 12;00;37;12
file-003.mpeg = 12;01;03;21
file-004.mpeg = 12;04;13;02
file-005.mpeg = 12;06;49;16

i could set them on a timeline in order by start time and export
an edl which would serve as a batch capture list for recapturing
the tape. by editing the edl i could recapture only select scenes
as well. if all of my files start at 00;00;00;00, however, they
bear no time reference to the original tape. i was just using this
as an example of how retaining the timecode for files would be
useful. i wasn't requesting edl functionality in hdvsplit.

on the other hand, and maybe it already does this, if hdvsplit
output a batch list when it captured, that would be a great thing!
it already can scene detect, isn't that true? if it keeps a list as it
captures and scene detects, that would be the best of everything.

thanks again,
BabaG

[paviko]

when i use hdvsplit to break the full-tape-file into its
individual scenes i find that all of the new files start at hour 0.
00;00;00;00 to be precise. are you saying that if we'd captured
these tapes using hdvsplit instead of premiere this would not
be the case? that would be great to know.
Nothing will change, because timecodes are inside .m2t file and are added by camcorder, not Premiere or HDVSplit. So proper timecodes are in all your captured files with Premiere (when output is m2t) or captured/splitted by HDVSplit.
BabaG, do you know where Premiere stores timecodes for a clip ? I think this problem is specific to video editor. Premiere probably is not reading timecodes from file, but is associating timecodes with clips only when they are captured from Premiere. I don't know Premiere, but maybe there is possibility to import EDL file or batch list and then Premiere can properly associate timecodes. Will have to install trial version and experiment.
Currently HDVSplit is not generating batch list, but I agree this would be usefull :)

Regards

[BabaG]

paviko,

i believe premiere does store timecode in its own place somewhere.
it may be in the file header someplace. there is dv software for
coverting various dv formats called dv conversion suite. it was
written by attila mezzei (hope i spelled right). his app offers the
possibility of 'add premiere timecode'. this leads me to believe the
code is in the file itself someplace premiere specifically looks at.
premiere also offers the possibility of changing the timecode
for a file. it's done by selecting the media in the project window
(not the timeline) and going to file--->timecode. i hope i'm
remembering that right. this also leads me to think it rewrites a
header or something. i have had email contact with attila and he
seems very helpful. you might be able to get some info from him.
here's a link to the folks distributing his app now:

http://www.focusinfo.com/solutions/catalog.asp?id=8

i read a post in the sony forum linked on your hdvsplit site that
describes the same issue i'm talking about. the poster described
how the first file of a splitting session retained the timecode but
all the subsequent files split out of the same master started at 0.
i'm not sure if the poster was using premiere or not.

so, if i capture with hdvsplit, the fornat the file is captured to is
.m2t? am i right in that? premiere captures to .mpeg file extension.
if i understand correctly, you're saying that capturing with hdvsplit
should retain the timecode as it scene detects and creates
individual files from a tape because it is saving in .m2t format.
is this correct? it would likely solve my problem. this seems to
contradict what the sony forum poster was saying, though. a batch
log as this is done would be great, though.

anyway, thanks for a great app and the quick attention you're giving us,
BabaG

[BabaG]

nope. just captured some scenes with scene detect on and
they are all read by premiere as starting at 0.

[paviko]

nope. just captured some scenes with scene detect on and
they are all read by premiere as starting at 0.
I've just said that timecodes are inside m2t file (e.g. use DGIndex and you see them while saving .d2v project) and Premiere can't read them. So we need to tell Premiere about timecodes somehow - e.g. utility that reads them from already captured files (no need to recapture) and placed them in batch list.
so, if i capture with hdvsplit, the fornat the file is captured to is
.m2t? am i right in that? premiere captures to .mpeg file extension.
if i understand correctly, you're saying that capturing with hdvsplit
should retain the timecode as it scene detects and creates
individual files from a tape because it is saving in .m2t format.
File extension can be anything. I was writing about .m2t as MPEG TS format. If you can capture from Premiere (with .mpeg extension) and then HDVSplit is able to split this file, then Premiere also uses MPEG TS and timecodes are inside files, but still not readable by Premiere :(

Will look at this DV Conversion Suite apps, thanks.

Regards

[BabaG]

thanks paviko.


use DGIndex and you see them while saving .d2v project


could you provide a bit of detail on how i do this to see the timecode?
i know it's a bit out of your way but i'm very interested in this and
would very much apreciate the help.


So we need to tell Premiere about timecodes somehow


been complaining about this for years.

thanks again,
BabaG

[documentaryman]

Paviko, it's most likely that Premiere Pro stores the timecode in a different location when it's writes the header. If you want I can send you a small captured Premiere file to examine.

[paviko]

could you provide a bit of detail on how i do this to see the timecode?
Probably it won't be much usefull for you, because the timecode is displayed live in DGIndex. But you can check that a tool can be written that will read in and out timecodes for certain clip (maybe I'll write it at this weekend). To see timecode:
1. Download DGIndex (or as I see it's named also DGMPGDEC)
http://neuron2.net/dgmpgdec/dgmpgdec148.zip
2. Start it.
3. Choose "File"/"Open"
4. Enter "*.*" and select .m2t file you want to open.
5. Detect PIDs (it's need to be done only once):
5a. Go to "Stream"\"Detects PIDs: PAT/PMT"
5b. Select 0x810 and click Set Video
5c. Select 0x814 and click Set Audio
6. Choose "File"\"Save project"
7. Click "Save"
The dialog will open with "Timestamp" label (in "Audio group"). The timestamp is changing while DGIndex is processing a file, so it's not much usefull (also it's cut to the seconds for easiness). This only shows that .m2t files have timestamps preserved.

Paviko, it's most likely that Premiere Pro stores the timecode in a different location when it's writes the header. If you want I can send you a small captured Premiere file to examine.
Thank you. But if it's writing timestamp inside mpg file, then it won't be compatibale with other editor.

Regards