New tool to change SAR in H264 bitstream

[roozhou]

I write a bitstream filter for ffmpeg. It can modify sps in H264 bitstream. So far it can change sar, fps, crop information, level, ref frames, cfr/vfr flag, fullrange flag, colorprim, transfer and colormatrix. Since the length of bitstream may be changed, I cannot make an in-place modifier.

ffmpeg git 29946
Download from sf (win32 only)

Patches can be found on svn.

Note this is a demuxer/muxer only ffmpeg build.

cmdline:

ffmpeg -i infile -vcodec copy -acodec copy -vbsf h264_changesps=arg1[/arg2[/arg3]] outfile

args can be:
sar=x:y ---- Set sar to x:y, 0:0 will remove sar info
fps=x:y ---- Set fps to x/y in cfr mode or set timebase to y/x in vfr mode
level=x ---- Set level to x. (e.g. 30 for L3.0, 41 for L4.1)
crop=x:y ---- Crop x pixels from right and y pixels from bottom (e.g. 1920x1080 should use crop=0:8)
ref=x ---- Set number of reference frames to x
cfr ---- Set fps mode to cfr
vfr ---- Set fps mode to vfr
fullrange ---- Use full range (0~255)
limitedrange ---- Use limited range (16~235)

colorprim=x ---- Set color primary to x (integer)
1="bt709", 2="undef", 4="bt470m", 5="bt470bg", 6="smpte170m", 7="smpte240m", 8="film"

transfer=x ---- Set transfer to x (integer)
1="bt709", 2="undef", 4="bt470m", 5="bt470bg", 6="smpte170m", 7="smpte240m", 8="linear", 9="log100", 10="log316"

colormatrix=x ---- Set color matrix to x (integer)
0="GBR", 1="bt709", 2="undef", 4="fcc", 5="bt470bg", 6="smpte170m", 7="smpte240m", 8="YCgCo"

removesei ---- Remove SEI from bitstream. This can be used to remove user data and HRD timing info.
removeaud ---- Remove access unit delimiters, which is useless outside BD player.
removedupheader ---- Remove duplicated sps/pps. In some AnnexB bitstreams sps/pps appear in every I frame. We can safely remove them.

multiple args should be separated by /

Output format is determined by extension.
Supported input and output containers:
ts,mp4,mkv,flv --> mp4,mkv,flv
ts,mp4,mkv,flv --> raw,avi(with -vbsf h264_mp4toannexb)
raw,avi --> raw,avi

It also supports remuxing aac from adts/rm/mp4/mkv/avi to adts/mp4/mkv/avi, and you don't need -absf aac_adts2asc.

Two optional args are added to h264_mp4toannexb:
norepeatheader ---- Write sps/pps only in the first frame
avihack ---- Hack on nal start code, saving ~1 byte per frame on avi. It has no effect on raw output. Avihack implies norepeatheader

A new option -muxerfps is added to ffmpeg. Use this to set container fps for avi or default duration for mkv. This is useful for vfr video (e.g. 120000/1001 for 24/30/60 fps mixed content) or mp4/mkv/flv -> avi muxing.

Example:
convert avc in mkv to raw:
ffmpeg -i input.mkv -vcodec copy -an -vbsf h264_mp4toannexb=norepeatheader output.h264

convert avc in mp4 to avi, set container fps to 29.97:
ffmpeg -i input.mp4 -vcodec copy -acodec copy -vbsf h264_mp4toannexb=avihack -muxerfps 30000/1001 -vtag H264 output.avi

Another new option -bom can be used to add BOM to extracted text files(e.g. ass subtitles).

Example:
extract ass subtitle from mkv
ffmpeg -i input.mkv -vn -an -scodec copy -bom utf-8 output.ass

[Lam3rD]

Great, thanks a lot for this. It makes things a lot simpler

edit: about crop - why 2 and not all 4 values - left/top/right/bottom ?

[roozhou]

Because ffmpeg does not support left & top crop.

[Keiyakusha]

Is there other restrictions for crop? For 1280x720 mkv it crashes for me with crop=640:360 or crop=16:16. And in some cases when it crashes after muxing, result is unplayable...

[nurbs]

Maybe it's only supposed to crop the padding in a non mod16 video. So when you have 1920x1080 it's actually 1920x1088 with 8 pixels cropped at the bottom.

[elguaxo]

Thanks.

[b66pak]

@roozhou thanks a lot...i was waiting for this...
_

[Blue_MiSfit]

Fantastic! I've had to use a hideously slow in-house developed perl script for tasks like this. Hopefully a real application will do this more quickly

~MiSfit

[b66pak]

some testing:

mkv input with wrong fps

ffmpeg -i sample.mkv -vcodec copy -an -vbsf h264_changesps=fps=24000:1001 -t 20 sample.mp4

the log

F:\1>ffmpeg -i sample.mkv -vcodec copy -an -vbsf h264_changesps=fps=24000:1001 -t 20 sample.mp4
FFmpeg version SVN-r21473, Copyright (c) 2000-2010 Fabrice Bellard, et al.
  built on Jan 28 2010 00:30:17 with gcc 4.4.1
  configuration:
  libavutil     50. 7. 0 / 50. 7. 0
  libavcodec    52.48. 0 / 52.48. 0
  libavformat   52.47. 0 / 52.47. 0
  libavdevice   52. 2. 0 / 52. 2. 0
  libswscale     0. 9. 0 /  0. 9. 0
[matroska @ 0x3ecd10]MAX_READ_SIZE:5000000 reached
[matroska @ 0x3ecd10]Estimating duration from bitrate, this may be inaccurate

Seems stream 0 codec frame rate differs from container frame rate: 47.95 (5000000/104271) -> 23.98 (5000000/208542)
Input #0, matroska, from 'sample.mkv':
  Duration: 01:24:02.51, start: 0.000000, bitrate: N/A
    Stream #0.0(eng): Video: h264, yuv420p, 1280x720, PAR 1:1 DAR 16:9, 47.62 fps, 23.98 tbr, 1k tbn, 47.95 tbc
    Stream #0.1: Audio: 0x0000, 48000 Hz, 6 channels
Output #0, mp4, to 'sample.mp4':
    Stream #0.0(eng): Video: 0x0021, yuv420p, 1280x720 [PAR 1:1 DAR 16:9], q=2-31, 24k tbn, 23.98 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame=  480 fps=  0 q=-1.0 Lsize=   16803kB time=19.94 bitrate=6904.5kbps
video:16794kB audio:0kB global headers:0kB muxing overhead 0.056911%

the output mp4 is 24000/1001 fps and play OK...



raw input (previous mkv demuxed to raw) with wrong fps

ffmpeg -i sample.h264 -vcodec copy -an -vbsf h264_changesps=fps=24000:1001 -t 20 sample.mp4

the log

F:\1>ffmpeg -i sample.h264 -vcodec copy -an -vbsf h264_changesps=fps=24000:1001 -t 20 sample.mp4
FFmpeg version SVN-r21473, Copyright (c) 2000-2010 Fabrice Bellard, et al.
  built on Jan 28 2010 00:30:17 with gcc 4.4.1
  configuration:
  libavutil     50. 7. 0 / 50. 7. 0
  libavcodec    52.48. 0 / 52.48. 0
  libavformat   52.47. 0 / 52.47. 0
  libavdevice   52. 2. 0 / 52. 2. 0
  libswscale     0. 9. 0 /  0. 9. 0
[h264 @ 0x3ecd10]max_analyze_duration reached
[h264 @ 0x3ecd10]Estimating duration from bitrate, this may be inaccurate

Seems stream 0 codec frame rate differs from container frame rate: 47.95 (959/20) -> 23.98 (959/40)
Input #0, h264, from 'sample.h264':
  Duration: N/A, bitrate: N/A
    Stream #0.0: Video: h264, yuv420p, 1280x720, 23.98 fps, 23.98 tbr, 1200k tbn, 47.95 tbc
Output #0, mp4, to 'sample.mp4':
    Stream #0.0: Video: 0x0021, yuv420p, 1280x720, q=2-31, 24k tbn, 23.98 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
[mp4 @ 0xa20050]pts has no value
frame=  480 fps=  0 q=-1.0 Lsize=   16802kB time=20.02 bitrate=6874.8kbps
video:16796kB audio:0kB global headers:0kB muxing overhead 0.031623%

the output mp4 is 24000/1001 fps but is not playing...
_

[Blue_MiSfit]

Can output to elementary stream be added? It doesn't look like this is supported yet..

~MiSfit

[b66pak]

also the .mp4 output no longer play in QT (is opened as audio only)...
_

[roozhou]

Originally Posted by b66pak

also the .mp4 output no longer play in QT (is opened as audio only)...
_

I set the display width/height of video track in mp4 to 0/0 and hope the player always ignore this and take width/height*sar in bitstream. Unfortunately QT fails to do so. I will revert this hack in next version.

So the width/height in mp4 container will remain unchanged until you specify "-aspect x" in ffmpeg cmdline, where x is the DAR of video in decimal.

[J_Darnley]

Originally Posted by roozhou

So the width/height in mp4 container will remain unchanged until you specify "-aspect x" in ffmpeg cmdline, where x is the DAR of video in decimal.

FYI, ffmpeg also supports a fraction or ratio as the argument for -aspect

[roozhou]

New build available.

@Keiyakusha
Crop values should fall between 0~15.

@b66pak
Currently ffmpeg does not support remuxing raw h264 stream.

@J_Darnley
"-aspect 16:9" is supported but "-aspect 16/9" will be recognized as "-aspect 16".

[J_Darnley]

Originally Posted by roozhou

@J_Darnley
"-aspect 16:9" is supported but "-aspect 16/9" will be recognized as "-aspect 16".

Oh yes, my bad. I didn't read the output too closely.

[b66pak]

@roozhou thanks a lot...
_

[b66pak]

ffmpeg -i sample.mkv -vcodec copy -an -t 20 sample.mp4

don't work anymore (infinite loop?)
_

[roozhou]

Originally Posted by b66pak

ffmpeg -i sample.mkv -vcodec copy -an -t 20 sample.mp4

don't work anymore (infinite loop?)
_

It's a regression in ffmpeg. -t and -ss seems not to work in recent build. See this thread.

I will release a new build as soon as it is fixed.

[popper]

hi roozhou.

is there any chance you might also make it capable of changing the SPS to lower Refs ?

care of DS way back in 24th December 2007, http://forum.doom9.org/archive/index.php/t-133070.html
posted some Annex B data
"@0 SPS: profile_idc 01001101 ( 77)
@8 SPS: constrained_set0_flag 0 ( 0)
@9 SPS: constrained_set1_flag 1 ( 1)
@10 SPS: constrained_set2_flag 0 ( 0)
@11 SPS: constrained_set3_flag 0 ( 0)
@12 SPS: reserved_zero_4bits 0000 ( 0)
@16 SPS: level_idc 00101000 ( 40)
@24 SPS: seq_parameter_set_id 1 ( 0)
@25 SPS: log2_max_frame_num_minus4 0001001 ( 8)
@32 SPS: pic_order_cnt_type 1 ( 0)
@33 SPS: log2_max_pic_order_cnt_lsb_minus4 00110 ( 5)
@38 SPS: num_ref_frames 011 ( 2)
@41 SPS: gaps_in_frame_num_value_allowed_flag 0 ( 0)
@42 SPS: pic_width_in_mbs_minus1 0000001111000 (119)
@55 SPS: pic_height_in_map_units_minus1 00000100010 ( 33)
@66 SPS: frame_mbs_only_flag 0 ( 0)
@67 SPS: mb_adaptive_frame_field_flag 0 ( 0)
@68 SPS: direct_8x8_inference_flag 1 ( 1)
@69 SPS: frame_cropping_flag 0 ( 0)
@70 SPS: vui_parameters_present_flag 1 ( 1)"

im hoping "@38 SPS: num_ref_frames 011 ( 2)" is the ref values and its possible to change them to lower values in the bitstream copy without a full re-encode (or to much loss of quality playback if any ) through your ffmpeg extensions/patches if you can find a way to add that option sometime ? please.

it would be good for quick fixing streaming to the xbox360 and its limited ref 3 abilitys to play 1080AVC.Mp4 smoothly and perhaps other non (ref16) spec playback devices too.

[roozhou]

New build available, now it should work on raw H264 streams.

@popper
Now ref frames can be modified.