Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > (HD) DVD & Blu-ray authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th June 2008, 13:49   #1  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
MKV to M2TS on Linux

The point of this post is to discuss converting HD content in MKV files to M2TS files for streaming to a PS3, via Mediatomb, on an Ubuntu Linux server.

The PS3 can be quite fussy on what video and audio formats are acceptable when it comes to streamed HD content. The main problem I have remaining is when the Video stream resolution isnít 1920x1080 or 1280x720 - my PS3 wonít show any video on playback.

Iíve read a lot on various forums and most solutions involve using Windows GUI applications (E.g. avisynth, MeGUI). As Iím using a Linux server to store my files for streaming - I would really like to create a script to do the job in one hit if possible and just leave my server to chug away. (And thus Windows apps arenít an option for me.)

Before I go too far I would say Iím novice in the field, (Novice = Noob + '3weeks reading forums like mad' + 'tinkering at home'), so Iím drawing together items from various threads to help other people starting out. Therefore I am happy to accept constructive criticism here to improve the content of this thread as that will only serve to help others down the line.

Hereís what I found so far - PS3 Valid Video Criteria:
  • Profile level: 4.1
  • Resolution: 1920x1080 or 1280x720
  • Codec: H.264
  • Iím sure there are other things Iíve missed off...

PS3 Valid Audio Criteria:
  • Codec: AC3, MP3. (However not DTS)
  • again Iím sure there are other things Iíve missed off...

I must credit sticky123 for a great blog post which is the foundation to this posting. I am hoping to take his/her work a step or two further.



There are three scenarios I would like to discuss and really need help with number 3:

Scenario 1) the MKV file contains video and audio streams which are both PS3 valid.


As you can see the (fantastic) application tsMuxer does the repackaging job without the need to re-encode any data stream. It can also force the Video profile level to 4.1 to ensure the PS3 is happy.


Scenario 2) here the Audio stream isnít PS3 compliant. i.e. stored in a DTS format. We can Ďbreakí the DTS stream out of the MKV file into a separate file on itís own and re-encode it to AC3 format, finally re-muxing (via tsMuxer) the new AC3 file and Video stream into a M2TS file.



Scenario 3) this time the Video stream isnít PS3 compliant. i.e. the resolution isnít quite right. Now this is where my knowledge/experience lets me down and why Iím looking for help from the rest of you. As far as Iím aware it should be, (he says laughably), a simple case of add black borders to the video stream to bring it up-to the correct resolution. (Obviously ensure the aspect ratio is maintained.) I would want to keep the Video data at the same quality/codec as the original source.



From what Iíve read so far there are some powerful (free) tools available such as mencoder, ffmpeg and x264. However all of them have more command line options than you can shake a stick at and finding the correct route to take has been rather daunting. Thus Iím asking for help.


Scenario 4) obviously itís possible for Scenario 2 & 3 to apply to one MKV file, but I didnít think I needed to go over that individually.




Iím hoping some of you will find this useful, and Iím hoping more that someone will be able to help me with Scenario 3. Once Iíve got a reasonable solution for that I will post my script for all to share.

Best Regards
Chuckle Worthy


Other things to discuss at a later date:
- subtitles. (I donít want to discuss subtitles at the moment, maybe once weíve got the video and audio sorted then we can take things a step further.)
- ff/rewind with HD resolutions doesnít work very well.
ChuckleWorthy is offline   Reply With Quote
Old 16th June 2008, 14:03   #2  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Application for step 2 in scenario 3 is DGAVCDec + Avisynth.

What are these MKV files and where are you getting them?
Guest is offline   Reply With Quote
Old 16th June 2008, 14:13   #3  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
Isn't Avisynth a Windows GUI application?

A pal of mine gave me a DVD with a load of film trailers on. I've just got a HD TV and I'm trying to be flash and show off the TV and video streaming etc. But as per - I'm not had too much success.

Last edited by ChuckleWorthy; 16th June 2008 at 14:19.
ChuckleWorthy is offline   Reply With Quote
Old 16th June 2008, 14:28   #4  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Quote:
Originally Posted by ChuckleWorthy View Post
Isn't Avisynth a Windows GUI application?
Oops, too early in the morning. Haven't had my coffee yet.
Guest is offline   Reply With Quote
Old 16th June 2008, 15:30   #5  |  Link
jeffy
Registered User
 
Join Date: Jan 2007
Posts: 943
@ChuckleWorthy: OT: Please, what software did you create the images in? Thank you.
jeffy is offline   Reply With Quote
Old 16th June 2008, 15:52   #6  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
Quote:
Originally Posted by jeffy View Post
Please, what software did you create the images in? Thank you.
If you're referring to the MKV files - I didn't create any, as I said, I got copy from a mate. Is there a way of interrogating the MKV files for that kind of information?
ChuckleWorthy is offline   Reply With Quote
Old 16th June 2008, 18:02   #7  |  Link
jeffy
Registered User
 
Join Date: Jan 2007
Posts: 943
Quote:
Originally Posted by ChuckleWorthy View Post
If you're referring to the MKV files - I didn't create any, as I said, I got copy from a mate. Is there a way of interrogating the MKV files for that kind of information?
The images Scenario 1, 2, 3. Sorry if I was not clear.
jeffy is offline   Reply With Quote
Old 16th June 2008, 18:14   #8  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
Quote:
Originally Posted by jeffy View Post
The images Scenario 1, 2, 3. Sorry if I was not clear.
Ohhh. Heh. I'll turn off my dullard setting...

I used Microsoft Visio 2003 Standard Edition - I always find a couple of pictures help explain things far easier then paragraphs of waffle.

Last edited by ChuckleWorthy; 17th June 2008 at 12:43.
ChuckleWorthy is offline   Reply With Quote
Old 23rd June 2008, 15:12   #9  |  Link
Grimmi Meloni
Registered User
 
Join Date: Feb 2003
Posts: 22
I have posted my script which does this in another thread. Feel free to use it.

EDIT: Sorry, I did not realize you already have most of the scenarios covered. Just ignore my post.

Last edited by Grimmi Meloni; 23rd June 2008 at 15:15.
Grimmi Meloni is offline   Reply With Quote
Old 25th June 2008, 21:21   #10  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
Okay I've been answering my own question as best as I can. This is my first adventure in this world, so after lots and lots of tinkering later - 'mencoder' seemed like the best way forward.

Like I said here's what I've come up with so far. It has produced some really good results IMO.

Again I'm happy for comments/suggestions/help etc.

Code:
#!/bin/bash
#**
#** m2tsEx.sh - a script for converting MKV files to M2TS for streaming to a PS3 
#** from a DLNA Media server.
#** Uses:
#**   - tsMuxer: http://www.videohelp.com/tools/tsMuxeR
#**       (I use the Win32 version of tsMuxer as I found the Linux version unreliable)
#**   - wine: http://www.winehq.org/
#**   - mkvtoolnix: http://www.videohelp.com/tools/MKVtoolnix
#**   - dcadec: http://www.videolan.org/developers/libdca.html
#**   - aften: http://aften.sourceforge.net/
#**   - mencoder: http://www.mplayerhq.hu/ 
#**   - x264: http://www.videolan.org/developers/x264.html
#**
#** v0.1 initial version - 24th June 2008
#**
#** Usage: m2tsEx.sh filename.mkv
#**
#** Functions ******************************************************************************** 
RED=$(    echo -e "\e[31;40m" )
GREEN=$(  echo -e "\e[32;40m" )
YELLO=$(  echo -e "\e[33;40m" )
BLUE=$(   echo -e "\e[34;40m" )
MAGENTA=$(echo -e "\e[35;40m" )
CYAN=$(   echo -e "\e[36;40m" )
RESET=$(  echo -e "\e[0m"     )

dmpstart() 	{ echo "--------------------------------------------------------" > "./process.log"; }
dmp() 		{ echo "`date`: $*" >> "./process.log"; }

#** colorful echo functions
msg()  { echo ""; echo ">> $*";                   dmp "$*"; }
rmsg() { echo ""; echo ">> ${RED}$*${RESET}";     dmp "$*"; }
gmsg() { echo ""; echo ">> ${GREEN}$*${RESET}";   dmp "$*"; }
ymsg() { echo ""; echo ">> ${YELLO}$*${RESET}";   dmp "$*"; }
bmsg() { echo ""; echo ">> ${BLUE}$*${RESET}";    dmp "$*"; }
mmsg() { echo ""; echo ">> ${MAGENTA}$*${RESET}"; dmp "$*"; }
cmsg() { echo ""; echo ">> ${CYAN}$*${RESET}";    dmp "$*"; }


#** Constants ******************************************************************************** 
BIN_DIR="/usr/bin"
LOCAL_BIN_DIR="/usr/local/bin"
WORKING_DIR="./wrking/"
SEVEN20="1280:720"
TEN80="1920:1080"
SEVEN20_WIDTH="1280"
TEN80_WIDTH="1920"
PRIMARY_LANGUAGE="eng"    #** Used in the mux.meta file, change this if you want... (obviously)  
SECONDARY_LANGUAGE="und"

#** Check the passed parameter is a MKV file ************************************************
dmpstart
cmsg "Starting..."
SRC_FILE="$1"
EXTENSION=`$BIN_DIR/basename $SRC_FILE | awk -F . '{print $NF}' | tr 'A-Z' 'a-z'`
BASENAME=$($BIN_DIR/basename "$SRC_FILE" .mkv)
DEST_FILE=$BASENAME.m2ts
if [ "$EXTENSION" != "mkv" ]
then
	rmsg "Not a Matroska File, exiting..."
	exit 1 
fi

#** EXTRACT DETAILS OUT OF THE MKV FILE ******************************************************
#** Get the info from the MKV file
cmsg "Extracting MKV Information..."
MKV_INFO=`$BIN_DIR/mkvinfo "$SRC_FILE"`

#** Check there are Streams within the MKV we're interested in
cmsg "Determine Track Id's..."
VIDEO_TRACK_NO=`echo "$MKV_INFO" | grep "V_MPEG4/ISO/AVC" -B10 | grep "Track number:" | awk '{ print $5 }'`
AC3_TRACK_NO=`echo "$MKV_INFO" | grep "A_AC3" -B10 -C3 | grep -E "Language: $PRIMARY_LANGUAGE"\|"Language: $SECONDARY_LANGUAGE" -B13 | grep "Track number:" | awk '{ print $5 }' | head -1`
DTS_TRACK_NO=`echo "$MKV_INFO" | grep "A_DTS" -B10 -C3 | grep -E "Language: $PRIMARY_LANGUAGE"\|"Language: $SECONDARY_LANGUAGE" -B13 | grep "Track number:" | awk '{ print $5 }' | head -1`

if [[ -z $VIDEO_TRACK_NO ]]
then	
	rmsg "No Video Stream found, exiting..."
	exit 1; 
fi

if [[ -z $AC3_TRACK_NO ]] && [[ -z $DTS_TRACK_NO ]] 
then
    rmsg "No AC3 or DTS Streams found, exiting..."
    exit 1;
fi

#** AUDIO TRACK INVESTIGATION ****************************************************************
#** Okay, lets find the lowest value Track Number, as that generally is the Main audio stream
#** I'll cheat by setting any null values to a high number, so the non-nulls will win the test
cmsg "Sort Out Audio Tracks..."
if [[ -z $AC3_TRACK_NO ]]
then
    AC3_TRACK_NO=999
fi
if [[ -z $DTS_TRACK_NO ]] 

then
    DTS_TRACK_NO=999
fi

if [[ $AC3_TRACK_NO -lt $DTS_TRACK_NO ]] 
then
	AUDIO_TRACK_NO=$AC3_TRACK_NO
	AUDIO_TRANSCODE=1 #FALSE
	gmsg "AC3 Audio Stream Found - Track Id: $AC3_TRACK_NO."
else
	AUDIO_TRACK_NO=$DTS_TRACK_NO
	AUDIO_TRANSCODE=0 #TRUE
	gmsg "DTS Audio Stream Found - Track Id: $DTS_TRACK_NO."
fi

#** VIDEO TRACK INVESTIGATION ****************************************************************
#** Get the Video Resolution etc
cmsg "Sort Out Video Resolution..."
SRC_WIDTH=`echo "$MKV_INFO" | grep "Pixel width:" | awk '{ print $5}'` 
SRC_HEIGHT=`echo "$MKV_INFO" | grep "Pixel height:" | awk '{ print $5}'` 
SRC_RESOLUTION="$SRC_WIDTH:$SRC_HEIGHT"
SRC_FPS=`echo "$MKV_INFO" | grep "V_MPEG4/ISO/AVC" -A3 | grep "fps" | awk '{ print $6 }' | cut -c2-100`

#** Calculate the Average Bit Rate 
SRC_SECONDS=`echo "$MKV_INFO" | grep Duration | awk '{ print $4}' | sed 's/s//' | awk -F . '{ print $1 }'`
SRC_SIZE=`echo "$MKV_INFO" | grep Segment, | awk '{ print $4}'` 
SRC_AVG_BITRATE="$[ ( ($SRC_SIZE * 8) / ($SRC_SECONDS ) / 1024 ) ]"
cmsg "Source Video Details - Resolution: $SRC_RESOLUTION, Duration: $SRC_SECONDS seconds, Average Bitrate: $SRC_AVG_BITRATE, Frames Per Second: $SRC_FPS."

#** Calculate the desired Resolution
if [[ -z $2 ]]
then	
	if [[ "$SRC_WIDTH" != "$SEVEN20_WIDTH" ]] && [[ "$SRC_WIDTH" != "$TEN80_WIDTH" ]] 
	then
		rmsg "Odd Resolution: $SRC_RESOLUTION. Please specify manually: 1920:1080 or 1280:720, exiting..."
		exit 1; 
	else
		if [ "$SRC_WIDTH" == "$SEVEN20_WIDTH" ]
		then
			DEST_RESOLUTION=$SEVEN20;				
		fi
		
		if [ "$SRC_WIDTH" == "$TEN80_WIDTH" ]
		then
			DEST_RESOLUTION=$TEN80;
		fi
	fi 
else
	#** Use the passed paramter for the new Resolution
	DEST_RESOLUTION=$2
fi

#** See if we need to transcode the video
if [[ $SRC_RESOLUTION == $DEST_RESOLUTION ]] 
then
	VIDEO_TRANSCODE=1 #FALSE
	mmsg "Video Stream - Track Id: $VIDEO_TRACK_NO. No Transcoding Required."

else
	VIDEO_TRANSCODE=0 #TRUE
	mmsg "Video Stream - Track Id: $VIDEO_TRACK_NO. Transcoding To $DEST_RESOLUTION Resolution." 
fi

#** START THE CONVERSION ******************************************************
#** Create the working temporary directory
mkdir -p ${WORKING_DIR}

#** Create the tsMuxer config file
echo "MUXOPT --no-pcr-on-video-pid --new-audio-pes --vbr " > ${WORKING_DIR}mux.meta

if [[ $VIDEO_TRANSCODE -eq 1 ]] 
then
	#** No Video Transcoding - just reference the source MKV file	
	echo "V_MPEG4/ISO/AVC, \"$SRC_FILE\", level=4.1, insertSEI, contSPS, track=$VIDEO_TRACK_NO, fps=$SRC_FPS, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta
else
	#** Video Transcoding - break out the source stream
	cmsg "Extracting H264 Stream..."
	$BIN_DIR/nice -n20 \
		$BIN_DIR/mkvextract tracks "$SRC_FILE" $VIDEO_TRACK_NO:"${WORKING_DIR}$BASENAME.h264"

	#** Start Video Transcoding 
	X264PASS1=""
	X264PASS1="${X264PASS1}pass=1:threads=1:turbo=1:bitrate=$SRC_AVG_BITRATE:frameref=1:bframes=3:"
	X264PASS1="${X264PASS1}b_pyramid:direct_pred=auto:weight_b:me=dia:mixed_refs:bime:"
	X264PASS1="${X264PASS1}trellis=1:nopsnr:nossim:subq=1:level_idc=41" #** This sets the Profile Level to 4.1
	
	X264PASS2=""
	X264PASS2="${X264PASS2}pass=2:threads=1:turbo=0:bitrate=$SRC_AVG_BITRATE:frameref=1:bframes=3:" #** I found a frameref of 3 was a no-no
	X264PASS2="${X264PASS2}b_pyramid:direct_pred=auto:weight_b:me=umh:mixed_refs:bime:"
	X264PASS2="${X264PASS2}partitions=all:8x8dct:"
	X264PASS2="${X264PASS2}trellis=1:nopsnr:nossim:subq=6:level_idc=41"
	
	cmsg "Video Transcoding - First Pass..."	
	$BIN_DIR/nice -n20 \
		$BIN_DIR/mencoder \
			"${WORKING_DIR}$BASENAME.h264" \
			-vf expand=$DEST_RESOLUTION \
			-ofps $SRC_FPS \
			-ovc x264 \
			-x264encopts $X264PASS1 \
			-nosound \
			-of rawvideo \
			-o /dev/null;
#			-o "${WORKING_DIR}${BASENAME}-pass1.h264" > ${WORKING_DIR}pass1.txt;

	# Second pass
	cmsg "Video Transcoding - Second Pass..."	
	$BIN_DIR/nice -n20 \
		$BIN_DIR/mencoder \
			"${WORKING_DIR}$BASENAME.h264" \
			-vf expand=$DEST_RESOLUTION \
			-ofps $SRC_FPS \
			-ovc x264 \
			-x264encopts $X264PASS2 \
			-nosound \
			-of rawvideo \
			-o "${WORKING_DIR}${BASENAME}-pass2.h264";
#			-o "${WORKING_DIR}${BASENAME}-pass2.h264" > ${WORKING_DIR}pass2.txt;

	#** Get rid of the first Pass statistics wosname
	rm -f "divx2pass.log"
	echo "V_MPEG4/ISO/AVC, \"${WORKING_DIR}${BASENAME}-pass2.h264\", level=4.1, insertSEI, contSPS, track=1, fps=$SRC_FPS, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta
fi

if [[ $AUDIO_TRANSCODE -eq 1 ]] 
then
	#** No Audio Transcoding - just reference the source MKV file
	echo "A_AC3, \"$SRC_FILE\", track=$AUDIO_TRACK_NO, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta
else
	#** Audio Transcoding - break out the source stream
	cmsg "Extracting DTS Stream..."
	$BIN_DIR/nice -n20 \
		$BIN_DIR/mkvextract tracks "$SRC_FILE" $AUDIO_TRACK_NO:"${WORKING_DIR}$BASENAME.dts" 
	
	cmsg "Transcoding DTS To WAV..."
	$LOCAL_BIN_DIR/dcadec -r -o wavall "${WORKING_DIR}$BASENAME.dts" > "${WORKING_DIR}$BASENAME.wav"
	
	cmsg "Transcoding WAV To AC3..."
	$BIN_DIR/aften "${WORKING_DIR}$BASENAME.wav" "${WORKING_DIR}$BASENAME.ac3"
	
	echo "A_AC3, \"${WORKING_DIR}$BASENAME.ac3\", track=1, lang=$PRIMARY_LANGUAGE" >> ${WORKING_DIR}mux.meta
fi

#** Get tsMuxer on the case...
cmsg "tsMuxing To M2TS File..."
$BIN_DIR/wine $LOCAL_BIN_DIR/tsMuxeR.exe ${WORKING_DIR}mux.meta $DEST_FILE

cmsg "Tidying Up..."
#rm -r -f $WORKING_DIR

cmsg "Done."

#***********************************************************************************
#** Big Thanks To The Following:
#**
#**   http://tinyurl.com/3e9x7n - This is the foundation to my script above - Huge thanks
#**   http://tinyurl.com/4prnev - I got the x264encopts from here
#**   http://tinyurl.com/6k6lsb - How to compile x264 and ffmpeg via Ubuntu
#**   http://tinyurl.com/6k5opc 
#**   Plenty of other sites too...
#** 
#***********************************************************************************
ChuckleWorthy is offline   Reply With Quote
Old 6th August 2008, 11:03   #11  |  Link
RyLoS
Registered User
 
Join Date: Jan 2003
Location: Parma (Italy)
Posts: 11
Very good work!
I can help you improve your script if you want.
For example, a lot of MKV in HD are not standard 1280x720 or 1920x1080....you can find many situations, like 1280x5xx etc etc.
From what i've seen your script strictly check and strictly wants to use those 2 standard definitions but PS3 can play also different ones without problems.
__________________
By RyLoS
RyLoS is offline   Reply With Quote
Old 23rd August 2008, 15:35   #12  |  Link
ChuckleWorthy
Registered User
 
ChuckleWorthy's Avatar
 
Join Date: Jun 2008
Posts: 13
We'll I've been tinkering and reading even more Doom9 forum posts. Therefore I've tweaked my original script quite alot to make life easier - I hope...

I think I may have cracked the whole FF/Rewind problem thanks to other threads.

Anyhow - as before - feel free to comment.
Cheers
Chuckle
Attached Files
File Type: zip m2ts.sh.zip (5.5 KB, 296 views)
ChuckleWorthy is offline   Reply With Quote
Reply

Tags
linux, mediatomb, ps3, tsmuxer

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.