Helix dtdrive multi-format multi-tool

[karl_lillevold]

Now that there is a codec section in binary release section on Helix Community that I manage, I realized I can make 'dtdrive' available for download. The package that I built is binary Win32-x86 only, but many of dtdrive's modules and dtdrive itself can be built from source from the Helix Community.

This tool does just about everything and the kitchen sink (except mux RM files, but that may be forthcoming), and I thought it would be useful for the advanced container format users in this forum.

For instance, you can convert AAC-RM to AAC

dtdrive file.rm -W file.aac


AAC-RM to M4A

dtdrive file.rm -W file.m4a


AAC to AAC-RM

dtdrive file.aac -S 1 -W file.rm

[if the resulting file is too short or too long, add option +d: dtdrive +d file.aac -S 1 -W file.rm, to recompute duration]

AAC to M4A and M4A to AAC

dtdrive file.aac -W file.m4a
dtdrive file.m4a -W file.aac



Demux RM A+V:

Audio: dtdrive -DV 0 -DA 2 av.rm -S 1 -W audio.rm
Video: dtdrive -DV 2 -DA 0 av.rm -S 1 -W video.rm


[-S 1 should always be used when writing out RealMedia file]

+ a whole lot more (conversions, decodes, transcodes, transmux) with a long list of formats (sorry, not Matroska). These were just the minimal set of examples I have tried myself. I am not really giving dtdrive justice by providing so few examples... If something does not work, let me know, but it is more than likely I can not help or explain right away at least.

Download
Helix Binary Downloads

Usage:

Important: dtdrive needs to find its DLLs. Use option -L or set env variable HELIX_LIBS.

Version 6.0.9.636-Release
usage:
dtdrive [+|-<Options>] [<Commands>] {[[<FSName>,]<Protocol>://]<File>}

=> Temporal Directives:
     Commands:
      -J <Time>     = skip to specified media time (ms)
      -T <Time>     = fetch packets for the specified amount of time (ms)

=> DLL Paths:
     Commands:
      -L <Path>     = load all components dlls from specified path
                       * overrides HELIX_LIBS environment variable setting
                       * can be overriden with more specific settings below
      -LP <Path>    = load plugins from specified path
                       * overrides HELIX_PLUGINS environment variable setting
      -LC <Path>    = load codecs from specified path
                       * overrides HELIX_CODECS environment variable setting
      -LE <Path>    = load encoder SDK from specified path
                       * overrides HELIX_ENCSDK environment variable setting

=> Output:
     Options: (+ = add, - = remove)
      p = dump packet content
      h = dump headers/processing stages
      v = verbose (show all buffer contents)
      e = dump event counters
      w = dump phase execution times
      d = if writing a file, recompute duration
      r = if writing a file, recompute preroll
      o = if writing a file, overwrite the existing file
      t = if writing a file, use relative timestamps
     Commands:
      -F <File>     = place output into specifed file
      -W <File.Ext> = write out file in format associated with extension
      -SL <MBytes>  = set the file size limit for written out files
      -TD <DirPath> = use the specified directory for intermediate files

=> Rate Selection:
     Options: (+ = add, - = remove)
      q = pass through switch-off packets
     Commands:
      -R <Rate>     = read only packets for the specified transfer rate (bps)
      -RM <Mode>    = perform dynamic rate switching in specified mode
                       0 = Constant Rete (no switching)
                       1 = Continusly oscillating (up/down)
                       2 = Ascending Rate (up to ceiling)
                       3 = Descending Rate (down to floor)
      -RMI <Time>   = rate change interval used when performning dynamic
                      rate switching (ms)
                       Default: 10000
      -RC <Rate>    = rate ceiling when performning dynamic
                      rate switching (bps)
      -RF <Rate>    = rate floor when performning dynamic
                      rate switching (bps)

=> A/V Synchronization & Scaling:
     Options: (+ = add, - = remove)
      k = when scaling video times (-VTS), output keyframes only
     Commands:
      -VO <Offset>  = offset in time video relative to audio (+/-ms)
      -VTS <Scale>  = Scaling to apply to video times
                      (in percent of normal playback speed)
      -MIN <fps>    = When scaling video times in keyframe-only mode,
                      use the MIN method with the specified fps
      -MAX <fps>    = When scaling video times in keyframe-only mode,
                      use the MAX method with the specified fps
      -LA <num>     = When scaling video times in keyframe-only mode
                      with MIN method, use this number of lookahead frames

=> Decoding:
     Options: (+ = add, - = remove)
      u = uncompress (decode) content
      f = when decoding, filter-out any non decoding streams
      s = when decoding, maximize speed on detriment of standards
      i = when decoding, set source properties in outgoing stream header
     Commands:
      -DA <Mode>    = decode audio streams via specified mode
                       0 = No Decode - passthrough/block based on option f
                       1 = Normal Decode
                       2 = Passthrough Decode
      -DV <Mode>    = decode video streams via specified mode
                       0 = No Decode - passthrough/block based on option f
                       1 = Normal Decode
                       2 = Passthrough Decode
                       3 = Depack Frames Only
      -DO <Mode>    = decode non-audio/video streams via specified mode
                       0 = No Decode - passthrough/block based on option f
                       1 = Normal Decode
                       2 = Passthrough Decode
      -DS <#> <Mode>= decode specified stream # via specified mode
                       0 = No Decode - passthrough/block based on option f
                       1 = Normal Decode
                       2 = Passthrough Decode
      -OSR <Samples>= format decoded audio to specified sample rate (Hz)
      -OCH <Count>  = format decoded audio to specified number of channels
      -OSS <Bits>   = format decoded audio to specified sample size (Bits)

=> Decryption:
     Options: (+ = add, - = remove)
      c = decrypt content into unprotected (clear) form

=> Encoding:
     Commands:
      -EA <Format>  = encode audio to specified format
      -EV <Format>  = encode video to specified format
      -EAR <Rate>   = when encoding audio, use specified bitrate (bps)
      -EVR <Rate>   = when encoding video, use specified bitrate (bps)
      -EART <%>     = when encoding audio, allow the encoded bitrate to
                      be within a percentage of the requested value.
                       0   = The encode MUST be at the specified bitrate
                       100 = The encode can be any bitrate
      -EARM <Rate>  = when encoding audio, this indicates the maximum allowable
                      encoded bitrate
      -EACF <list>  = when encoding audio, this indicates a list of codec
                      4cc codes that are acceptable for an
                      optimized transcode
                      (The list is a ';' seperated string of 4cc codes)
      -EAIF <4cc>   = when encoding audio, this indicates desired interleaver
      -EAICF <list> = when encoding audio, this indicates a list of
                      interleaver 4cc codes that are acceptable
                      for an optimized transcode
                      (The list is a ';' seperated string of 4cc codes)
      -OT           = enable optimized transcode. If a transcode is not
                      necessary then only a transformat will be done.

=> Environment:
     Commands:
      -S <Mode>     = run in server emulating mode
                       0 = run as client (local playback)
                       1 = run as server (streamed playback)

[bond]

cool stuff!

does it also handle to mux he-aac from .rm to .mp4 correctly with the sbr flag?

also when will we see rv9 with private track id in .mp4?

[S_O]

Where can I find dtdrive source code? I couldn´t find the project.
The new Helix Page has worst navigation I´ve ever seen. Even the old page was better! A general search-function would be very usefull, which searches the entire website. Also a complete overview over all projects would be very usefull (maybe it´s there, but I couldn´t find it).
Also where can I find a up-to-date AudienceDefaults.xls?

[karl_lillevold]

The site is hard to navigate, the ones responsible for site organization know, and are working to improve. The problem is there are so many projects, sub-projects, and dependencies. I have forwarded the suggestion about search and a main index / "Help I'm lost" page.

In the meantime, here is the Producer documentation. Not sure it includes the file you are looking for though.
https://producersdk.helixcommunity.org/docs/
If it's not there, please ask the Helix Producer Apps mailing list: http://lists.helixcommunity.org/mail...-producer-apps

Some have used Google with success to find the right information on helixcommunity.

All the projects are listed here:
https://helixcommunity.org/search/?t=allpublic
However, dtdrive is not a separate project as such, but a build target, what you have to do is follow the instructions on getting cvs up and running. Then run the 'build' command, and choose dtdrive as your build target and helix-client-all-defines are your profile. Your 'build' menu should look like this:

Build System Menu
-> Current Directory: D:\helix\test
[0] Set BIF branch (helix)
[1] Set Target(s) (dtdrive)
[2] Set Profile (helix-client-all-defines)
[3] run: build -trelease
[4] Toggle make depend & makefiles (-e -n)
[5] Toggle release (-trelease)
[6] Toggle 'make clean'  (-c)
[7] Toggle clobber (Dangerous!) (-C)
[8] Toggle halt-on-error (-p green)
[9] Toggle verbose mode (-v)
[10] Toggle static build (-tnodll)
[11] Checkout source for selected target now
[12] Set Tag/Branch/Timestamp
[13] Help Page (full help in build/doc/index.html)
[Q] Quit

[karl_lillevold]

I have updated the first post with some new information. When dtdrive is used to write out RM file, -S 1 should always be used, for maximum compatibility.

Option +d (recompute duration) is useful if the resulting file is too short (or too long)

Also, dtdrive can be used to convert between other formats than RealMedia, for instance from ADTS AAC directly to M4A.

P.S. Our MP4 (M4A) and AAC readers and writers are open source.

[S_O]

P.S. Our MP4 (M4A) and AAC readers and writers are open source.

Can they be used for the Helix Server?
I would really like streaming mp4 over rtsp with Helix Server.

c = decrypt content into unprotected (clear) form

Does that mean with this tool you can easily remove DRM protection of your new music store?

[S_O]

However, dtdrive is not a separate project as such, but a build target, what you have to do is follow the instructions on getting cvs up and running

What CVS instructions What CVS-Root, what modul? Is there any web interface available like on the old page where I can see what modules are available?

Also your site is very similar to sourceforge I think sourceforge is a good page, easy to navigate etc., but yours is a disaster. You don´t find anything and it´s horrible to navigate. Where can I find out what cvs-modules are available in a project? Where can I search the entire page for something (a file, a word, a sentence etc.)?

[karl_lillevold]

The CVS instructions are in the 2nd link upper right corner on the front page (Configuring CVS): https://common.helixcommunity.org/20...docs/cvsconfig
so even though I agree it's hard to find what you are looking for, you did not look all that thouroughly either ..

Option +c does not work unless you have the right key as well as very special components, which of course are not distributed or available from hc.org.

[S_O]

Unfortunately your answer isn´t very helpful. CVS checkout is working fine, for example I can checkout from the ribosome-project the modul "build". No problem. All I want to know what CVS-Root and modul for dtdrive, not general CVS instruction, like:

Root: cvs.helixcommunity.org:/cvsroot/helix-tools
Modul: dtdrive

You said dtdrive is a build target, not a own project, but to what project does it belong?
I don´t know what to checkout from CVS.
When I would like to checkout source from helix-producer I click on CVS and see this:

:ext:developername@cvs.helixcommunity.org:/cvsroot/helix-producer co modulename

But where to get a list of the available modules if there is no web-cvs-view?

Option +c does not work unless you have the right key as well as very special components, which of course are not distributed or available from hc.org.

Ohh.. Then we´ll have to wait until someone breaks it. iTunes DRM can be decrypted and M$ DRM can be too (at least version 2), if your music-store is successfull it won´t take too long.

[karl_lillevold]

Glad you got cvs working. dtdrive picks up its source files from a long list of projects, datatype, client, etc, so a manual checkout of all the sources is just not feasible. Once you run 'build' option 3, the build script gathers a list of all the source files and distribution modules needed, and checks them out, then compiles. You can even use build's option 11 to check out all the sources without building. 'build' also has a number of cmd line options, see '--help'.

This is not the right forum for HC build and checkout questions though. Try our IRC, someone can usually help there, at least during normal west coast working hours.

I know you won't believe it, but based on what I know from these other DRM systems, compared to ours, whoever tries will have a harder time breaking ours. No DRM is 100% secure though, but everything can easily be updated, if broken.

[Latexxx]

This is a little OT. Sorry, but there was no one to answer in irc.
When I try to run "build" I'll get:

F:\helix\sources\s>build
F:\helix\Python23\lib\xmllib.py:10: DeprecationWarning: The xmllib module is obs
olete. Use xml.sax instead.
DeprecationWarning)
Updating [common] BIF files...
running 'cvs -d :ext:latexxx@cvs.helixcommunity.org:/cvsroot checkout common/bui
ld/BIF' in 'F:\\helix\\build\\bif-cvs\\helix' (as = None)
Cannot access /cvsroot/CVSROOT
No such file or directory
No BIF branches found, please configure your .buildrc!

Edit: My .buildrc is
AddCVS("helix",":ext:latexxx@cvs.helixcommunity.org:/cvsroot")
AddBIFPath("common", "[helix]/common/build/BIF")

Edit2: I got it to work:
AddMultiCVS("helix",":ext:latexxx@cvs.helixcommunity.org:/cvsroot")

[karl_lillevold]

I think you need to change your AddCVS line to AddMultiCVS:

AddMultiCVS("helix",":ext:latexxx@cvs.helixcommunity.org:/cvsroot/")

AddBIFPath("common","[helix]/common/build/BIF")
AddBIFPath("client","[helix]/client/build/BIF")
AddBIFPath("producerapps","[helix]/producerapps/build/BIF")

I am afraid I don't know the build system well enough to know exactly why this is needed, but it changed when we moved to the new site and CVS checkout method ("ext" ssh based protocol).

If you connect to IRC now, there are plenty of people there

EDIT: missed your EDIT2

[Atamido]

Originally posted by karl_lillevold
+ a whole lot more (conversions, decodes, transcodes, transmux) with a long list of formats (sorry, not Matroska).

[bond]

lo karl

i read that real seems to use an old version of the mpeg-4 norm and therefore hardcodes audio samplerate to 1000 in mp4/m4a, which seems to be outdated

[karl_lillevold]

yes, we know about that bug, and it has been fixed. The field is supposed to be ignored by players though.

[bond]

ok

[Musaran]

Originally posted by karl_lillevold The site is hard to navigate, the ones responsible for site organization know, and are working to improve.

Hello.

I created an account on http://helixcommunity.org, but i can't manage to log-in, hence download.
I keep returning to login page, with a message warning to activate cookies, but I can't think of anything preventing cookies on my system.
And I can't find an e-mail to ask directly.

Is this a known temporary problem due to undergoing reorganisation, or is it just my system ?


It's a bit off-topic, but I wasn't sure where to ask this, ans I felt it didn't deserve a thread of it's own.

[karl_lillevold]

Hi Musaran: I am afraid I don't know what's wrong. Perhaps you can try an alternative browser first, then please e-mail admin@helixcommunity.org and include your helix username, and a brief description of the problem (copy and paste from your post). Feel free follow up here if you find out what the problem is, in case someone else runs into the same.

[iradic]

hi
is it possible with this tool to encode to mp4 with racp (wav -> mp4)?
if so could someone please post a cmd line how-to...

thanks

[karl_lillevold]

there are ways to encode WAV -> HE-AAC using dtdrive, but the version I released may not be setup to do that. If I find some time, I may look into it. However, you can download Helix Producer and the HE-AAC add-on, and achieve the same.

With the cmd line producer, batch encoding is very easy even without any GUI helper programs. Just unzip producer in its own folder, goto this folder, and type:

producer -i <folder_w_wav_files> -ad my_audience.rpad -dt

You can even set up foobar2000 CLI encoder with this command, to fully automate the process of batch converting audio files using producer..

Your audience for HE-AAC can look something like this, for HE-AAC at 64 kbps [save this to a file, call it my_audience.rpad:

<?xml version="1.0"?>
<audience>
  <avgBitrate type="uint">64000</avgBitrate>
  <maxBitrate type="uint">64000</maxBitrate>
  <streams>
    <audioStream>
      <pluginName type="string">rn-audiocodec-realaudio</pluginName>
      <codecName type="string">racp</codecName>
      <codecFlavor type="uint">0</codecFlavor>
      <streamContext type="bag">
        <presentationType type="string">audio-only</presentationType>
        <audioMode type="string">music</audioMode>
      </streamContext>
    </audioStream>
  </streams>
</audience>

This will give you HE-AAC in RM, which can be converted via dtdrive directly, or mkvtoolnix (rm -> mka -> aac -> mp4). This conversion can be chained to the producer command with && in the foobar CLI encoder. I should post instructions on how to do that...