Problems transcoding dvgrab output

[cayblood]

I'm having trouble transcoding a bunch of files recorded by dvgrab during a live capture session. I have multiple cameras, each of which is being captured by dvgrab under ubuntu linux. I'm having dvgrab capture in raw dv format so that I can monitor the capturing with ffplay, so what I end up with are a bunch of 1GB files like this:

cam1_001.dv
cam1_002.dv
cam1_003.dv
etc.

I want to take these files and transcode them into a smaller more compressed format that is a single file, such as h.264 at a high-quality level for archival purposes.

What do you guys think is the best way to do this? I'd really like to come up with a scriptable command-line solution for this. I'm still not very familiar with transcoding under linux.

Thanks,
Carl

[nm]

What do you guys think is the best way to do this? I'd really like to come up with a scriptable command-line solution for this. I'm still not very familiar with transcoding under linux.
Try MEncoder. It is very well documented and there are plenty of scripts around that use it for H.264 encoding with x264. You could start by looking at MEncoder threads in this forum. Most of the issues (how to avoid AVI output, how to deinterlace, ...) have already been discussed.

[cayblood]

Thanks nm, the biggest issue I'm running into right now is how to splice these dv files together before passing them into mencoder. Does anyone know how I can do this without having to create an intermediate file? I'm running into an issue where if I try to concatenate too many dv files together I go beyond the file size limit, so I'd like to somehow pipe the concatenated dv files through mencoder so I can avoid intermediate files.

[nm]

This might work:
cat cam1_001.dv cam1_002.dv cam1_003.dv | mencoder -vf YOUR_FILTERS -of rawvideo -ovc x264 -x264encopts YOUR_X264_OPTIONS -o out.264 -

[cayblood]

Thanks nm, I'm still having problems when trying to cat the files together:

confreaks@station1:/media/disk-1/001/temp$ cat cs1cam1_001.dv cs1cam1_002.dv cs1cam1_003.dv | mencoder -mc 0 -of rawvideo -ovc x264 -x264encopts subq=4:bframes=2:b_pyramid:weight_b -o out.264 -
MEncoder 2:1.0~rc1-0ubuntu13.1 (C) 2000-2006 MPlayer Team
CPU: Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz (Family: 6, Model: 15, Stepping: 6)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled with runtime CPU detection.
Reading from stdin...
success: format: 0 data: 0x0 - 0x0
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Win32 LoadLibrary failed to load: avisynth.dll, /usr/lib/win32/avisynth.dll, /usr/local/lib/win32/avisynth.dll
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
Cannot seek backward in linear streams!
Seek failed
============ Sorry, this file format is not recognized/supported =============
=== If this file is an AVI, ASF or MPEG stream, please contact the author! ===
Cannot open demuxer.

Exiting...

[nm]

You should upgrade MPlayer/MEncoder and x264 to the latest SVN revisions.

MEncoder can't recognize the input data format. Have you tried normal encoding (one input file, no piping)? If that works, find out what demuxer and codec is used and try to force them manually (see mplayer -demuxer help, mplayer -vc help).

[cayblood]

Thanks for the help, nm. Here is the output from transcoding one video:

confreaks@station1:/media/disk-1/001/temp$ mencoder -mc 0 -of rawvideo -oac faac -ovc x264 -x264encopts subq=4:bframes=2:b_pyramid:weight_b -o out.264 cs1cam1_001.dv
MEncoder 2:1.0~rc1-0ubuntu13.1 (C) 2000-2006 MPlayer Team
CPU: Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz (Family: 6, Model: 15, Stepping: 6)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled with runtime CPU detection.
success: format: 0 data: 0x0 - 0x3e7fc180
libavformat file format detected.
VIDEO: [dvsd] 720x480 0bpp 29.970 fps 28771.2 kbps (3512.1 kbyte/s)
[V] filefmt:35 fourcc:0x64737664 size:720x480 fps:29.97 ftime:=0.0334
==========================================================================
Opening audio decoder: [pcm] Uncompressed PCM audio decoder
AUDIO: 48000 Hz, 2 ch, s16le, 1536.0 kbit/100.00% (ratio: 192000->192000)
Selected audio codec: [pcm] afm: pcm (Uncompressed PCM)
==========================================================================
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
==========================================================================
Opening video decoder: [dshow] DirectShow video codecs
Win32 LoadLibrary failed to load: qdv.dll, /usr/lib/win32/qdv.dll, /usr/local/lib/win32/qdv.dll
Warning: DS_Filter() could not open DirectShow DLL. (DLL=qdv.dll, r=0x89253c0)
Failed to create DirectShow filter
ERROR: Could not open required DirectShow codec qdv.dll.
You need to upgrade/install the binary codecs package.
Go to http://www.mplayerhq.hu/dload.html
VDecoder init failed :(
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffdv] vfm: ffmpeg (FFmpeg DV decoder)
==========================================================================
VDec: vo config request - 720 x 480 (preferred colorspace: Planar 411P)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
VDec: using Planar 411P as output csp (no 0)
Movie-Aspect is 1.36:1 - prescaling to correct movie aspect.
SwScaler: reducing / aligning filtersize 1 -> 4
SwScaler: reducing / aligning filtersize 5 -> 4
SwScaler: reducing / aligning filtersize 1 -> 1
SwScaler: reducing / aligning filtersize 9 -> 8

SwScaler: BICUBIC scaler, from yuv411p to yuv420p using MMX2
SwScaler: using 4-tap MMX scaler for horizontal luminance scaling
SwScaler: using 4-tap MMX scaler for horizontal chrominance scaling
SwScaler: using 1-tap MMX "scaler" for vertical scaling (YV12 like)
SwScaler: 720x480 -> 720x480
x264 [info]: using SAR=10/11
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
Pos: 291.5s 8738f (100%) 15.50fps Trem: 0min 63mb A-V:0.000 [1830:186]
Flushing video frames

Video stream: 1831.363 kbit/s (228920 B/s) size: 66743560 bytes 291.558 secs 8738 frames

Audio stream: 186.341 kbit/s (23292 B/s) size: 6789760 bytes 291.499 secs
x264 [info]: slice I:36 Avg QP:23.00 size: 48602
x264 [info]: slice P:4084 Avg QP:26.00 size: 11952
x264 [info]: slice B:4618 Avg QP:27.87 size: 3504
x264 [info]: mb I I16..4: 26.4% 0.0% 73.6%
x264 [info]: mb P I16..4: 3.3% 0.0% 1.8% P16..4: 38.1% 23.9% 7.2% 0.0% 0.0% skip:25.7%
x264 [info]: mb B I16..4: 0.4% 0.0% 0.1% B16..8: 14.6% 1.1% 2.5% direct:21.4% skip:59.9%
x264 [info]: kb/s:1831.3

Can you tell me where I can figure out which demuxer and codec is being used? Also, what if it has a problem because each dv file has a small wrapper around it? How can I tell it to remove the wrapper from the incoming catted stream?

Thanks,
Carl

[nm]

Information about demuxers and decoders can be seen here:
libavformat file format detected.
VIDEO: [dvsd] 720x480 0bpp 29.970 fps 28771.2 kbps (3512.1 kbyte/s)
[V] filefmt:35 fourcc:0x64737664 size:720x480 fps:29.97 ftime:=0.0334
==========================================================================
Opening audio decoder: [pcm] Uncompressed PCM audio decoder
AUDIO: 48000 Hz, 2 ch, s16le, 1536.0 kbit/100.00% (ratio: 192000->192000)
Selected audio codec: [pcm] afm: pcm (Uncompressed PCM)


Also, what if it has a problem because each dv file has a small wrapper around it? How can I tell it to remove the wrapper from the incoming catted stream?
Yes, headers and other container-specific issues may cause problems here. I just remembered that MEncoder should have support for direct multiple file input nowadays. Forget my foolish advice on piping concatenated data and just give MEncoder a list of input files: mencoder ... -o out.264 input1.dv input2.dv ...

[skygod]

Thanks nm, I'm still having problems when trying to cat the files together:

I use avimerge. Make one really big file and then work with it, usually I make DVD's - using mpeg2enc.

[cayblood]

Success! Thanks so much for your advice, nm.

Skygod, avimerge is useless to me because I'm combining so many files that the combined file ends end up being larger than the filesystem allows, and I'd also rather not waste the extra time creating a large file when I'm just compressing it in the long run anyway.