HDTV in Linux?

[Karl Beem]

Anybody capturing and converting transport streams in Linux? If so, what tools do you use?

[CityK]

Convert to what?

For a nice easy gui based solution, have a look at Avidemux2 and its options.

[SatansChild]

For capturing HDTV (I'm useing a pcHDTV 3000 card) from digital cable I use one of two solutions depending on my needs.
1) Myth tv (you'll need a version > .20 to get decent dvb support)
Pros: Easy to use
You can use mythtranscode and the mythtv editor to quickly and easily chop out commercials almost losslessly
Cons: Mythtv and it's dependencies are big (qt and mysql mainly), Cable companies (at least comcast I don't have any experiance with others) don't include electronic program guide or channel (number or name) info in thier streams (trying to encourage prople to buy thier cableboxes) so setting up myth for them requires figureing out which channels are which and useing the channel editor to plug that information in (time consuming)
2) use azap to tune your card and simply cat /dev/dvb/adapter0/dvr0 > somefile.ts
Pros: Can capture a whole multiplex stream rather then tuneing to a single channel allowing you to capture multiple channels at one time with only one tuner card (only works if all the channels you want are on one multiplex).
No large dependencies just dvb kernel drivers and azap
Cons: Much More labor intensive
can't use myths editor so your only option (as far as I know) is to use ProjextX which works very well but the interface is slow and seaking in a file is slow (feels like my system is a pentium 2 not a duel opteron) mythtv's editor and VideoReDo (windows only I've never managed to get it working under wine) don't have this proble so it seems to be an issue with projectx on linux (projectx on windows is better but still not as good)

I store my HDTV rips in x264.mkv format so to convert I use one of three choices depending which filters I need
1) mencoder to export to raw h264 elimentiary stream MP4 box to make a .mp4 of it and mkvmerge to merge the original AC3 sound back with the video (3 steps)
2) transcode (www.transcoding.org) outputing yuv4mepg to a fifo end encoded by the x264 binary (outputs to mkv with newer versions) which gets the orriginal sound merged back (two steps)
3) avidemux2 outputs to a mp4 file (mkv not supported yet and it's not a priority accoring to the developers), mkvmerge to merge the .mp4 video with the original AC3. I get less then half the encodeing speed with avidemux2, but I belive this is due to the hard limit of 4 threads imposed by it whereas mplayer and the x264 binary both like to use between 6 and 8 threads. avidemux also offers less options for x264 encodeing then the other two so if you want it I would avoind it unless you must have a gui editor and/or the filters it supports.

If you would like examples of the scripts I use let me know.

[Karl Beem]

Thanks. I use capdvhs in WinXP via the firewire output of a Motorola (Comcast) DVR. I was wondering if this could be done in Linux.

[mic64]

SatansChild

I really would like to see some scripts for the encoding.
Cause all my tries with 1080i and 720p material wee not succesfull.
The endresult was stuttering.

So please post some examples here.

thanks
mic64

[SatansChild]

For 720p content I use this

#!/bin/bash
FILETITLE=`echo $1 |grep -o .*S[0-9][0-9]E[0-9][0-9]`
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=1:turbo=2:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o $FILETITLE.720p.HDTV.RAWES.264 $1
MP4Box -fps 23.976025 -add $FILETITLE.720p.HDTV.RAWES.264 $FILETITLE.720p.HDTV.VideoOnly.mp4
mplayer -dumpaudio -dumpfile $FILETITLE.ac3 $1
mkvmerge -o $FILETITLE.720p.HDTV.x264.mkv $FILETITLE.720p.HDTV.VideoOnly.mp4 $FILETITLE.ac3

For 1080i content I use
#!/bin/bash
FILETITLE=`echo $1 |grep -o .*S[0-9][0-9]E[0-9][0-9]`
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=1:turbo=2:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o $FILETITLE.720p.HDTV.x264.264 $1
MP4Box -fps 23.976025 -add $FILETITLE.720p.HDTV.x264.264 $FILETITLE.720p.HDTV.x264.mp4
mplayer -dumpaudio -dumpfile $FILETITLE.ac3 $1
mkvmerge -o $FILETITLE.720p.HDTV.x264.mkv $FILETITLE.720p.HDTV.x264.mp4 $FILETITLE.ac3

Both scripts leave me with a 720p 23fps h.264 video with ac3 sound in a mastroka container. The scripts take two parameters the first is the filemane I want to encode from (I use a $ShowName.S$seasonNumberE$EpisodeNumber.$whateverelse format for my filenames) and the target bitrate to encode to. I set the bitrate so I will end up with a file approximately 1/4 dvd size (usually around 3100 for a normal tv show). Encodeing takes around 9 hours on my duel opteron 265 machine. Quality is acceptable (Looks pretty good on a 720p plasma screen but compression artifacts become pretty obvious on a 1080p LCD) to me but if anyone has suggestions for a higher quality and/or faster encode at around the same quality feel free to let me know.

[mic64]

thanks, I will give this a try but is it correct,
that in line 2 and 3 of the mencoder line its always pass=3 ?

[Karl Beem]

How do you edit?

[SatansChild]

mic64

From the mencoder man page
pass=<1−3>


Enable 2 or 3-pass mode. It is recommended to always encode in 2 or 3-pass mode as it leads to a better bit distribution and improves overall quality.

1 first pass

2 second pass (of two pass encoding)

3 Nth pass (second and third passes of three pass encoding)

[Thasp]

Are you supposed to label both the 2nd and 3rd pass in a 3 pass encode like he did, with pass=3 for both, or was that just a typo?

[kdf]

For 720p content I use this

#!/bin/bash
FILETITLE=`echo $1 |grep -o .*S[0-9][0-9]E[0-9][0-9]`
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=1:turbo=2:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=ac,framestep=2,filmdint=dint_thres=256,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o $FILETITLE.720p.HDTV.RAWES.264 $1
MP4Box -fps 23.976025 -add $FILETITLE.720p.HDTV.RAWES.264 $FILETITLE.720p.HDTV.VideoOnly.mp4
mplayer -dumpaudio -dumpfile $FILETITLE.ac3 $1
mkvmerge -o $FILETITLE.720p.HDTV.x264.mkv $FILETITLE.720p.HDTV.VideoOnly.mp4 $FILETITLE.ac3

For 1080i content I use
#!/bin/bash
FILETITLE=`echo $1 |grep -o .*S[0-9][0-9]E[0-9][0-9]`
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=1:turbo=2:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o /dev/null $1
mencoder -vf pp=lb,filmdint=dint_thres=256,pp=ac,scale=1280:720,harddup -ovc x264 -x264encopts bitrate=$2:threads=auto:pass=3:frameref=15:bframes=5:nob_pyramid:direct_pred=auto:partitions=all:8x8dct:subq=7:mixed_refs:nodct_decimate -nosound -ofps 24000/1001 -noskip -of rawvideo -o $FILETITLE.720p.HDTV.x264.264 $1
MP4Box -fps 23.976025 -add $FILETITLE.720p.HDTV.x264.264 $FILETITLE.720p.HDTV.x264.mp4
mplayer -dumpaudio -dumpfile $FILETITLE.ac3 $1
mkvmerge -o $FILETITLE.720p.HDTV.x264.mkv $FILETITLE.720p.HDTV.x264.mp4 $FILETITLE.ac3

Both scripts leave me with a 720p 23fps h.264 video with ac3 sound in a mastroka container. The scripts take two parameters the first is the filemane I want to encode from (I use a $ShowName.S$seasonNumberE$EpisodeNumber.$whateverelse format for my filenames) and the target bitrate to encode to. I set the bitrate so I will end up with a file approximately 1/4 dvd size (usually around 3100 for a normal tv show). Encodeing takes around 9 hours on my duel opteron 265 machine. Quality is acceptable (Looks pretty good on a 720p plasma screen but compression artifacts become pretty obvious on a 1080p LCD) to me but if anyone has suggestions for a higher quality and/or faster encode at around the same quality feel free to let me know.

Thanks ! much appreciated !

[Thasp]

One thing to remember is that not all sources are interlaced just because they are aired 1080i - a simple uncomb would work. pp=lb is a deinterlacer, and deinterlacing video that was not interlaced can cause undesired artifacts.

However, using filmdint with scale, or using filmdint without pp=lb, will cause artifacts due to a bug in the filter which seems to've gone unfixed.

After a large run of test encodes using nearly every deinterlacer, ivtc, uncomb filter in the manual, I found this worked best for my -vf. It gives mod8 cropping - I doubt any source will require more. It uses lanczosresize which is sharper, and for an HD source much nicer looking than the default bicubic, which has a large influence on the quality of the output.

pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup

Make sure to use -ofps 24000/1001 at the end before -of instead of -noskip.

Also, you can skip the step of going to mp4box then to mkv with the newest mkvmerge. If you do it with a basic command line, you will have distortion, or improper audio sync. Use this command line, and you'll have in sync, undistorted matroska output with audio.

mkvmerge -o finaloutput.mkv -a 0 -D -S audio.ac3 --default-duration 0:24000/1001fps -d 0 -A -S mencoder_output.264 --track-order 0:0,1:0

If you are encoding x264 from a 1080i input source, I find this is a good tradeoff between speed and quality, without the artifacts of using pp=lb on a source that merely requires uncombing.


mencoder video.mpg -oac copy -ovc copy -of rawaudio -o audio.ac3
mencoder video.mpg -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=1:turbo=2:bitrate=3200:threads=3:direct_pred=spatial:subq=1:me=dia:bframes=3:b_pyramid:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mencoder video.mpg -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=2:bitrate=3150:threads=3:direct_pred=spatial:subq=7:partitions=all:8x8dct:me=umh:frameref=5:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mkvmerge -o finaloutput.mkv -a 0 -D -S audio.ac3 --default-duration 0:24000/1001fps -d 0 -A -S video.264 --track-order 0:0,1:0

[Karl Beem]

I repeat. How do you edit out the commercials?

[SatansChild]

I repeat. How do you edit out the commercials?
If you've captured with mythtv you can use it to chop out your commercials if not you can use ProjectX or avidemux, all of which will edit out commercials allmost losslesly.

[kdf]

mencoder video.mpg -oac copy -ovc copy -of rawaudio -o audio.ac3
mencoder video.mpg -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=1:turbo=2:bitrate=3200:threads=3:direct_pred=spatial:subq=1:me=dia:bframes=3:b_pyramid:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mencoder video.mpg -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=2:bitrate=3150:threads=3:direct_pred=spatial:subq=7:partitions=all:8x8dct:me=umh:frameref=5:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mkvmerge -o finaloutput.mkv -a 0 -D -S audio.ac3 --default-duration 0:24000/1001fps -d 0 -A -S video.264 --track-order 0:0,1:0


Thanks for this, im trying it out now.. I have slightly modified your code just to make it more reusable by using bash inputs like SatansChild's script.

mencoder $1 -oac copy -ovc copy -of rawaudio -o audio.ac3
mencoder $1 -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=1:turbo=2:bitrate=$2:threads=2:direct_pred=spatial:subq=1:me=dia:bframes=3:b_pyramid:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mencoder $1 -sws 9 -vf pullup,crop=1904:1064:8:8,scale=1280:720:0:0:4,harddup -oac copy -ovc x264 -x264encopts pass=2:bitrate=$2:threads=2:direct_pred=spatial:subq=7:partitions=all:8x8dct:me=umh:frameref=5:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1:keyint=240:keyint_min=24 -ofps 24000/1001 -of rawvideo -o video.264
mkvmerge -o finaloutput.mkv -a 0 -D -S audio.ac3 --default-duration 0:24000/1001fps -d 0 -A -S video.264 --track-order 0:0,1:0

save that as a file called "encode720p" then "chmod +x encode720p" then to use simply

./encode720p inputfile bitrate.

Should end up with the finaloutput.mkv file if everything works.

[mic64]

Hi

thanks for the scripts guys.
They´re really good and with a good bitrate (>4000) the encodes look really really good.

Maybe it should be in another thread, but anyone got a mencoder x264 commandline which makes a PS3 compatible encode?
for aac we can use faac and for mp4 mp4creator.