m2t problems

[BabaG]

i've been going in circles for a while now. hope there's some
info here.

i have some m2t files captured in hdvsplit from an hdv source.
much of it is 29.97 interlaced footage (60i). i also have a bunch
of hdv at 25fps interlaced (50i). i'm trying to conform the 60i to
the 50i and keep running into roadblocks. i have cineform but it
doesn't seem to go from 60i to 50i or even 25p. everything
seems aimed at going to 24p. i tried hdtvtomepg2 in order to
make a dupe i could get into vdub but once in vdub i can't
output back to my m2t or mpeg format. it seems to want to got
only to avi of various flavors.

i realize i may have missed something but what's a way i can
achieve the stated objective of 60i m2t ---> 50i m2t? or possibly
60i m2t ---> 25p m2t? m2t is the operative format here.

thanks,
BabaG

[Blue_MiSfit]

You are going to have to re-encode the MPEG-2 Video. This means you need two things (in my world anyway).

1) Avisynth with DGDecode to decode and process the MPEG-2
2) An MPEG-2 Encoder - like CCE if you have $, or HC / QuEnc if you don't

If you aren't familiar with the DGIndex / DGDecode workflow into avisynth, it's pretty simple. Drop the M2T on DGIndex, and if you need to adjust the PIDs, do so. HDV source material will likely not have this problem.

Save a D2V - which is like a reference file that AviSynth uses to indirectly access the M2T.

Make a new AviSynth script with the following bits in it:

#Load Plugin(s)
LoadPlugin("path to dgdecode.dll")

#Load Source
MPEG2Source("path to d2v")

#Tff or Bff depending on what DGIndex tells you
AssumeTFF()

#Bob (to work with fields)
tdeint(mode=1)

#Convert to 50 fields per second with blending
ConvertFPS(50)

#Reinterlace
separatefields().selectevery(4,0,3).weave()


This AviSynth script should take a 60i HDV input, and spit out uncompressed 50i. This can then be loaded into your MPEG-2 encoder and encoded to match the HDV specs, the details of which I am not entirely certain. I think it's CBR 25mbps for 1080i.

Anyway, after encoding you will get the m2v, which is just the video. Mux to an m2t with the audio that was demuxed by dgindex. I think MPEG Streamclip can do this. Not sure though.
~MiSfit

[BabaG]

Blue_MiSfit,

wow! thanks for all the detail. i am familiar with the first half of this
process as i use dgindex to feed vdub sometimes. mpeg2 encoding
i've always done from within my apps, though, which seem to only
support such. don't see the codecs anywhere but in the apps.

is there a way to take these many steps and automate them together
into a single process or script? could this be done with an extended
avisynth script?

thanks again,
BabaG

[Pookie]

The following procedure will automate the process.

Apps Required

DGindex - http://neuron2.net/dgmpgdec/dgmpgdec148.zip
Tdeint - http://www.missouri.edu/~kes25c/TDeintv1.zip
FFmpeg - http://tirnanog.fate.jp/mirror/ffmpeg/ffmpeg.rev6199.7z note: Use this version specifically, it works with high bit rates and Avisynth file input

Save the following text with the name Template.txt
SetMemoryMax(384)

LoadPlugin("DGDecode.dll")
LoadPlugin("Tdeint.dll")

mpeg2source("__vid__")

#Tff or Bff depending on what DGIndex tells you
AssumeTFF()

ConverttoYUY2()

#Bob (to work with fields)
tdeint(mode=1)

#Convert to 50 fields per second with blending
ConvertFPS(50)

#Reinterlace
separatefields().selectevery(4,0,3).weave().ConverttoYV12()

Save the following with the name Process.bat

for %%a in (*.m2t) do DGindex -IF=[%%a] -FO=0 -OM=1 -OF=[%%~na] -AT=[template.txt] -EXIT

for %%b in (*.avs) do ffmpeg -i "%%b" -threads 1 -vcodec mpeg2video -dc 10 -g 15 -bufsize 640 -minrate 25000 -b 25000 -maxrate 25000 -aspect 16:9 -an -mbd 2 -qmin 2 -async 1 -y "%%~nb.m2v"

Save everything into the same folder with your M2T files (I know, it isn't neat, but it is simple). It will look like this:

DGDecode.dll
DGIndex.exe
ffmpeg.exe
generic_name.m2t
here_is_another.m2t
process.bat
TDeint.dll
template.txt

Doubleclick on "Process.bat", and let the encoding begin (This will take a while...)

When it is done, your directory listing will look like this. The new 25fps file will be named *.m2v

DGDecode.dll
DGIndex.exe
DGIndex.ini
ffmpeg.exe
generic_name MPA PID 814 DELAY -240ms.mpa
generic_name.avs
generic_name.d2v
generic_name.m2t
generic_name.m2v
here_is_another MPA PID 814 DELAY -64ms.mpa
here_is_another.avs
here_is_another.d2v
here_is_another.m2t
here_is_another.m2v
process.bat
TDeint.dll
template.txt

Post Processing Audio and Multiplexing

You can use Belight to fix the delays and change the fps of the audio file to fit the new M2V file.

http://www.free-codecs.com/download/BeLight.htm
http://www.free-codecs.com/download/BeSweet.htm (required for Belight)

Finally, use the following app to mux the audio and m2v back together http://82.10.220.174/Uploaded_Files/Doom9_Forum_files/MPEG_Multiplexer_(v1.0).7z
Choose the Mpeg2 settings under "Options" before you multiplex

[Blue_MiSfit]

Nice followup Pookie - complete with a handy batch file :D

I was wondering which MPEG-2 encoders support HD sizes and rates correctly - are there any others aside from QuEnc and its parents ffmpeg?

And its nice to know that MPEG Muxer can make M2Ts :)

~MiSfit

[Pookie]

Good karma to help a guy named BabaG :D


Unfortunately, the list of high bitrate Mpeg2 encoders is scarce at the moment. Hank said his next version of HC enc will have support for HD resolutions/bitrates. That is gonna rock. I believe TMPGenc has the ability, but you'll have to dig around for the MP@HL settings.


"And its nice to know that MPEG Muxer can make M2Ts"

Unfortunately, no. It will mux into a plain ol' MPEG2 PS file. I haven't found any free Transport Stream multiplexers out there. There's VLC, but it is real time muxing, and you have to run TStoATSC on the resulting file to make it really Transport Stream compliant (standard PIDs, etc). The closest to free is the Womble Mpeg2VCR app, which I've used many times to mux to TS files that play on hardware decoders. Sixty bucks, but well spent. I believe they have a time limited eval on their site. You can also convert PS to TS with VideoRedo, but you'll still have to mux your .M2v and .MPA to .MPG with the above app first.

[BabaG]

wow! most groovy. thanks Pookie and Blue_MiSfit. couple questions
to finish up:

once i've a folder full of 50i m2v's, can the belight and
MPEG_Multiplexer processes be batched? there are potentially
hundreds of files here. using hdv split on my tapes and getting
50 and upwards clips per tape. the conversion is great. will still be
huge amount of work if have to hand multiplex and correct each
of the clips after the conversion.

thanks again. this is really great!
BabaG

[BabaG]

testing now. made one goof in forgetting to install avisynth. thought
i already had it. now i'm getting the m2v's.

another question: what is meant by 'fix the delays' regarding use of
belight in changing frame rate from 2997 to 25.

thanks again,
BabaG

[Pookie]

When you drag your multitude of .mpa files into the main window of Belight, you'll see in the lower left portion of the app, under Besweet OTA, a checkbox that says Delay . Just make sure it is checked. Then under "Advanced", I THINK you have to check "Convert Frame rate 29970 to 25000". Post a question up on the audio section, or do a search, or test a single file by outputting the 29.97 and 25.0 fps versions, and mux with the .m2v


Unfortunately, the Mpeg Muxing can't be batched :( (somebody please prove me wrong !)

[Pookie]

Okay, I proved myself wrong.

Use Mplex.exe from this package.

http://people.freenet.de/CharlesWeb/download/mjpegtools_180_P4_Win32.7z

Copy Cygwin1 into the same folder as Mplex.exe http://alloy.mit.edu/alloy30/cygwin1.dll


Syntax:

mplex -f 3 -b 786 -r 19200 -o output.mpg input.m2v input.ac3

[BabaG]

wow this is great pookie. spent all day wednesday trying to figure
out how to write a script to automate this. found something called
autoitv3 and got most of the way through making a script but
didn't quite finish. have to be in front of the workstation to finish it.
your solution is MUCH simpler. i'm obviously not a programmer.

two questions:

i notice the input for audio says input ac3. mine are mpa files. does
this not matter and i still use input.ac3?

also, i see that cygwin seems to be a whole package of things. is the
cygwin1.dll file all i need and not the entire setup package and
procedure?

thanks again, this is very helpful.
BabaG

ps, will post the autoit script when i finish if anyone's interested.
with this info, though, i may not finish it if noone's interested.

[Pookie]

Yes, it should accept .mpa files as the audio track, and Cygwin1 is just a single DLL which you can download here:

http://alloy.mit.edu/alloy30/cygwin1.dll

Just put Mplex.exe and Cygwin1.dll into the same directory, and you should be set to go :)

mplex -f 3 -b 786 -r 25000 -o output.mpg input.m2v input.mpa

[BabaG]

pookie,

just tried the mplex command:

mplex -f 3 -b 786 -r 25000 -o output.mpg input.m2v input.mpa

and got error that '[mplex] Unable to open file input.m2v for
reading.'

i'm thinking maybe i'm just too dumb to be using an apropriate
wildcard or something, replacing 'input' with 'actual_file_name'?
how woulf this work in a batch process? or is it supposed to be
doing that?

also, haven't been able to get belight to batch the mpa files
extracted in the original process from dgindex. stops after the
first file. have a question posted in audio encoding about that.

and lastly, don't know much about wxp commandline but seems
odd to me that when i try to cd to the directory all this stuff is
now in, the prompt remains my user directory in documents and
settings. i expect the prompt to change to the location i've
cd'ed to.

thanks,
BabaG

[Pookie]

First, make sure there is a name correlation between your audio and video files (001.m2v 001.mpa). There has to be something to identify the video and audio as a pair for any automated process to work. The other option is that you've placed the video and audio pair into their own directory or folder.

Let's assume you have a folder with 2 pairs of video and audio files in it -

001.m2v 001.mpa
002.m2v 002.mpa

From the command line, you can use the for command to your advantage. for is a list processor, very powerful.

for %x in (*.m2v) do mplex -f 3 -b 786 -r 25000 -o %~nx.mpg %~nx.m2v %~nx.mpa

%x is being used as the variable that holds the name of the file in ( ). The %~nx means let's use only the base name of the file as our variable - 001, then 002 as the process continues running on our pairs of files. Had we written our command as for %y in ()...etc, then %~ny would be the necessary bit on the rest of the command.

If you're using a batch file, you need to double the % signs

Runthis.bat

for %%x in (*.m2v) do mplex -f 3 -b 786 -r 25000 -o %%~nx.mpg %%~nx.m2v %%~nx.mpa

There are a bunch of cool things you can do with for. Type for /? at the command prompt for the list of switches, and google for the multitude of uses in windows command line scripts.

[BabaG]

well, just got through all these conversions and have a problem.
took most of a week to do it all, and that was processing 24 hrs
a day on a dual opteron. very slow. generally, though, it's working
great. if you're still seeing this, have another question, though.

all (i think) of the files i mplex'ed are coming up short. they
don't seem to all end at a uniform length or anything, just seem
to be truncated, often by a lot.

i checked the length of an original m2t file.
checked m2v and mpa against it. came up ok.
checked the m2v and mp2. also ok.
checked the mpg resulting from the mplex process and it's only
around a quarter the length it should be. maybe a lot less.

all the other steps seem fine and in sync.
process of mplexing in batch seems to go ok (to my untrained eye).

final mpg's are short, though.

any thoughts?

thanks again for all your help on this pookie. it's been a great
help and has made something i couldn't even begin to think of
doing manually into a really viable process. (just wish i'd known i
had 19 tapes instead of the 4 i originally thought i had ;)

BabaG