x264 LAVF/FFMS input public beta test (v2.1)

[Dark Shikari]

Now that we've had a small public test (http://doom10.org/index.php?topic=133) for version 1, we need as much testing as possible done on version 2 of the LAVF/FFMS input patch. In short, this input patch lets you not only take input from nearly any file without using Avisynth, but also allows x264 to natively handle variable framerate video without timecode files.

What we need you to do: test with any input you can find, especially inputs that might be hard or otherwise suspect to breaking. Report any bugs you find, but make sure to check the known bugs list below so you don't report things that we already know about.

Download v2.1 (http://mirror05.x264.nl/Dark/x264_lavfffms2.exe)
Patch v2.1 (http://pastebin.com/m7b096160)

Download v1 (http://kemuri9.net/dev/x264/x264_lavf+ffms.zip)
Patch v1 (http://pastebin.com/m4ef9fb74)

New features (v2.1):

1. Input from anything, even without Avisynth or DirectShow codecs, even on Linux!
2. "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video. No more timecode files!
3. Use --demuxer to force a particular input method (lavf, ffms, etc).
4. Automatic handling of all kinds of weird input (changing resolution, etc), including many inputs that Avisynth wouldn't have worked correctly with.
5. A near-complete rewrite of the x264cli internals to handle the above.
6. Periodic Intra Refresh.

See the three commit messages in the patch for more details.

Gotchas (v2.1):

1. There seem to be some types of files (raw h264?) that FFMS refuses to index. We'll be looking at that, but LAVF should work fine in the meantime.
2. FFMS won't work on piped input.
3. x264 doesn't by default save the index file from FFMS, so it has to re-index on every pass unless you use --index. We may change the default behavior in the future.

Known bugs (v2.1)
<none>

Now that we're through the gotchas, feel free to test on various types of input and report any issues you have.

Disclaimer: this is a HUGE patch. It modifies dozens of files and likely has a lot of bugs. Problems are normal, and the purpose of this test is to root out as many as possible.

Mega-thanks to Mike Gurlitz (ACoolie), Steven Walters (Kemuri9), and Kieran Kunhya (Kierank) for the massive amount of high-quality work that went into this patch.

[mariush]

windows 2003, 32 bit, 4 gb of memory, avisynth installed 2.5.8 i think... source is 720p 50 fps i think

x264_lavfffms.exe --bitrate 1024 -o e:\test.mkv e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv

http://savedonthe.net/image/488/crash.png

vs 2010 debug shows this..

http://savedonthe.net/image/489/Clipboard02.png

process explorer shows this...

http://savedonthe.net/image/490/procexp.png

it wouldn't surprise me to have made something wrong, so please let me know if that's the case.. and if there's any way I can help you out in more detail I'll be glad to do it.

later edit: is it supposed to actually read mkv files? i see it doesn't crash with .avi files...

even later edit: on avi files the console window says in the taskbar indexing input but after it finishes that and begins to encode content it still says the same thing.. I think it used to show encoding progress. In the console window itself, it does show the status correctly
(and excellent news and thanks for the cool things you add to x264!)

[Dark Shikari]

Confirmed, FFMS seems to be crashing on MKV files. We'll look into it.

If FFMS crashes, try --demuxer lavf and see if that works instead.

[mariush]

ok, it worked with --demuxer lavf but noticed it wasn't able to show the percent... went counting frames:


x264_lavfffms.exe --demuxer lavf --bitrate 1024 --output e:\test.mkv "e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv"
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
Last message repeated 5 times
[matroska @ 003dbda0]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1280x720p 1:1 @ 50/1 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.2
x264 [warning]: non-monotonic pts at frame 837 (17640 < 18520)
x264 [info]: frame I:9 Avg QP:25.74 size: 32864
x264 [info]: frame P:639 Avg QP:27.54 size: 6404
x264 [info]: frame B:1306 Avg QP:32.33 size: 967
x264 [info]: consecutive B-frames: 4.8% 15.9% 3.4% 75.9%
x264 [info]: mb I I16..4: 25.7% 63.1% 11.1%
x264 [info]: mb P I16..4: 4.2% 2.9% 0.1% P16..4: 38.5% 5.4% 3.6% 0.0% 0
.0% skip:45.4%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 23.6% 0.1% 0.1% direct:
0.1% skip:75.8% L0:33.9% L1:65.6% BI: 0.5%
x264 [info]: final ratefactor: 29.93
x264 [info]: 8x8 transform intra:44.4% inter:79.0%
x264 [info]: coded y,uvDC,uvAC intra: 19.0% 32.7% 7.0% inter: 4.0% 7.1% 0.1%
x264 [info]: i16 v,h,dc,p: 44% 27% 9% 21%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 45% 3% 4% 4% 5% 3% 7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 28% 14% 4% 7% 5% 7% 4% 6%
x264 [info]: Weighted P-Frames: Y:20.7%
x264 [info]: ref P L0: 64.8% 8.6% 19.6% 3.6% 3.4%
x264 [info]: ref B L0: 95.3% 4.7%
x264 [info]: kb/s:1156.81

encoded 1954 frames, 28.38 fps, 1099.16 kb/s


File plays fine in MPC-HC with the dxva decoder though at some point it goes blocky for a couple of frames in the middle - maybe it got corrupt or it's just the way the movie was... if i remember correctly i got the sample from ffmpeg's collection of test clips

question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?

[Dark Shikari]

Bug found in FFMS that caused the problem. Fix can be found here (http://pastebin.com/d79761bd4
); build in the topic updated to v2.1 accordingly.question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?Yes, it should, and as far as I can tell it works correctly.

[mariush]

Now it works with MKV input but the aspect ratio is ignored when using --demuxer lavf command:


x264_lavfffms2.exe --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
ffms [info]: 720x432p 37:26 @ 25/1 fps (vfr)
x264 [info]: using SAR=37/26



x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
[matroska @ 003dbd70]Estimating duration from bitrate, this may be inaccurate
lavf [info]: 720x432p 0:1 @ 25/1 fps (vfr)

If you want, I can upload a small 20s clip but you probably already have some test clips for this.

I've also noticed some differences when using that short 1080p clip with the migrating white ducks or whatever birds they are:


x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
Last message repeated 29 times
[matroska @ 003dbd90]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)



x264_lavfffms2.exe --frames 100 --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
ffms [info]: 1920x1080p 1:1 @ 4000/167 fps (vfr)
x264 [info]: using SAR=1/1

[Firebird]

http://img254.imageshack.us/img254/8404/x264tests.png
Just stops on 318 frame.

Sample (http://www.mediafire.com/?jzlkzzhytmx)

[Dark Shikari]

http://img254.imageshack.us/img254/8404/x264tests.png
Just stops on 318 frame.

Sample (http://www.mediafire.com/?jzlkzzhytmx)Did you update to v2.1? That sample works fine here.

[Firebird]

Did you update to v2.1? That sample works fine here.
v2.1 fails, even with --demuxer avs
Latest stable build is working. Hm. I will test it more, maybe it's a bug of mine.

[chipzoller]

Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?

[Dark Shikari]

Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?It can open anything ffmpeg can open. Read: everything.

[prOnorama]

It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMO

[Dark Shikari]

It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMOYou can rename it, it's just what I named the executable on the server ;)

[roozhou]

DS, thanks for the awesome New Year's present!
Is vfr ratecontrol implemented in this patch?

[kemuri-_9]

Is vfr ratecontrol implemented in this patch?

no, it will come shortly later

[Episodio1]

I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?

[avivahl]

I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?Well, it probably uses this library (http://code.google.com/p/ffmpegsource/) to wrap around ffmpeg (http://ffmpeg.org/)'s decoding capabilities. I guess that's about it.

[ACoolie]

lavf is short for libavformat, which is the library ffmpeg and x264 use to open and decode videos. This demuxer is attempted last because it can open all files ffms can, as well as pipefiles, but ffms typically does it better. It doesn't actually read the entire stream before encoding has begun, so frame count is unknown (no percent progress), and fps is taken from the container header, which could be wrong for VFR video. To explicitly use this demuxer, use `x264 --demuxer lavf`, although you should only use it for testing or if ffms2 does not work. Also, --seek will be slow because frames must be read sequentially.

ffms2 is Fabulous FM Source 2 (or FFmpegSource2), a wrapper of lavf that indexes files to provide a better API than libavformat has. It will read the entire input file, which can take considerable time for large files, so you should pass `--index indexfile` to save this index across passes. ffms has fast seeking, and should theoretically give the correct average fps for vfr video (we're still working on this). We've noticed a lot of the bugged input files are caused by this library (which is good, since we can fix them), so the lavf demuxer sometimes must be used on certain files and filetypes. You can force this demuxer to be used with `--demuxer ffms`, but typically it will be the default demuxer if it can be used.

[Sharc]

.........
I guess it won't work for interlaced VC-1 though (hard to find a sample)

Confirmed. Didn't work on interlaced VC-1 here. Encoder runs but produces only a flat gray at playback.
AVC interlaced seems to be ok.

[roozhou]

@kemuri-_9

How did you calculate dts in mp4 output for vfr video when b-frames are used? It seems initdelay is still used.

[chipzoller]

What's a recommended way to use this build to open a DVD-encode project in substitute of AviSynth? Do we feed it d2v project files?

[Dark Shikari]

What's a recommended way to use this build to open a DVD-encode project in substitute of AviSynth? Do we feed it d2v project files?Correct.

[chipzoller]

What about resizing? If correcting for PAR (no upsizing) are the built-in resizing algorithms just as effective as some of those for Avisynth?

[LoRd_MuldeR]

I think if you want to apply filters, such as resize, you must still use Avisynth input...

[wOxxOm]

ffms2 has built-in resizers (lanczos & spline included, as well as much much more options) in its API, so will it be possible to specify those in x264lavf/ffms cli command line?

[Dark Shikari]

We'll be adding resizing support eventually. My thought is we'll have a few resizing modes (just an idea):

--resize WxH: resizes such that the size does not exceed W or H. Maintains aspect ratio: the input is not stretched, and by default, the SAR stays the same as the input too. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --resize 1280x720
Output: 960x720 SAR 4:3

--input-sar: Overrides the SAR of the input file. Works the way SAR does currently. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --input-sar 1:1
Output: 1440x1080 SAR 1:1

--output-sar: Modifies the resizing accordingly, in addition to setting the SAR. So, for example:

Input: 1440x1080, SAR 4:3
Settings: --output-sar 1:1
Result: 1920x1080, SAR 1:1

[kemuri-_9]

@kemuri-_9

How did you calculate dts in mp4 output for vfr video when b-frames are used? It seems initdelay is still used.

I didn't touch the output muxers, i mostly dealt with the demuxer side of things.
should ask kierank/VFR_maniac.
VFR_maniac was the one who handled most of the updated mp4 code afaik.

[wOxxOm]

We'll be adding resizing support eventually. My thought is we'll have a few resizing modes (just an idea)
cool))
just make sure that "--output-sar 1:1" alone won't resize the stream :p
And, resize quality (spline/bicubic/etc) selectors would be vital.

[Dark Shikari]

cool))
just make sure that "--output-sar 1:1" alone won't resize the stream :p Why?

--input-sar will be aliased to the old --sar option, so --output-sar will be the new option (with new behavior).

[wOxxOm]

so --output-sar will be the new option (with new behavior)as you see fit, sir. I just thought that a switch that enables and does the resizing should have corresponding mnemonics, AND - what if I want to both resize and use some specific SAR for the output? e.g. resize from 1440x1080 to 720x480 and set 4:3 SAR... I'm not sure I see an intuitive way of doing it along the proposed scheme, but i am not too smart either :p

[LoRd_MuldeR]

--resize WxH: resizes such that the size does not exceed W or H. Maintains aspect ratio: the input is not stretched, and by default, the SAR stays the same as the input too. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --resize 1280x720
Output: 960x720 SAR 4:3

Sorry, but I think that would be really confusing. If I enter "--resize WxH" then I'd expect the output at exactly WxH. That's how any application I'm aware of works.

Maybe something like "--resize-width W" and "--resize-height H" could be added for resize that automatically maintains the correct aspect ratio?

Or just a placeholder, like MEncoder's resize option. For example "--resize -1:720" would resize to Wx720 with W chosen automatically to maintain the correct AR.

[Dark Shikari]

Sorry, but I think that would be really confusing. If I enter "--resize WxH" then I'd expect the output at exactly WxH. That's how any application I'm aware of works.The number one complaint I've gotten about ffmpeg, from hundreds upon hundreds of users, is exactly the opposite of what you said: that, by default, resizing will stretch the image.

I don't know a single image editing program that, by default, allows you to stretch the image when resizing. I don't see why x264 should be any different. 99.9% of users do not want to stretch the video, so why should we design the program around the 0.1% who do?

[LoRd_MuldeR]

The number one complaint I've gotten about ffmpeg, from hundreds upon hundreds of users, is exactly the opposite of what you said: that, by default, resizing will stretch the image.

I don't know a single image editing program that, by default, allows you to stretch the image when resizing. I don't see why x264 should be any different. 99.9% of users do not want to stretch the video, so why should we design the program around the 0.1% who do?

There's an important difference:

If I use the resize filter in an image editing program, which has a GUI(!), and I enter the target size, then the "width" edit box is updated in realtime when I change the value in the "height" box - and vice versa. So the user will always see what he'll get as the final result and he can uncheck the "maintain aspect ratio" option at any time. But when I explicitly(!) pass "--resize WxH" to a CLI application, it would be really unorthodox to get something different from WxH, because there is no indication that "--resize WxH" actually means "--resize-to-the-biggest-size-that-accurately-fits-into WxH". When I explicitly(!) specify a value, I'm expecting to get that value and nothing else. And again: That's how any CLI application I'm aware of works. GUI applications are a different topic...

[Dark Shikari]

OK, then suggest a better method. I'm open ideas; the above I came up with in about 5 minutes. The method must have the following properties:

1. It must be conducive to batch encoding of many different input files, in which a user has a maximum size he wants.
2. It must minimize the number of additional parameter options.
3. It must, by default, prevent the user from inadvertently stretching his videos.
4. It must provide some way to change the SAR, including both a way to change the SAR while not changing the resolution (stretching), and a way to change the SAR while changing the resolution too (not stretching).

[LoRd_MuldeR]

Well, I can think of three methods:
1. Use "--resize WxH" for explicit resizing and use "--resize-width W" respectively "--resize-height H" for automated/managed resize that will keep the AR
2. Use "--resize WxH" for explicit resizing and use "--resize-maximum WxH" alias "--target-screen-size WxH" alias "--managed-resize WxH" for automated/managed resize that will keep the AR
3. Use "--resize WxH" for explicit resizing and use "--resize ?xW" respectively "--resize Hx?" for automated/managed resize that will keep the AR

[mariush]

I think resizing shouldn't influence the sar values, it may be too confusing for users (though most would use GUIs)

--resize widthxheight (don't care about aspect ratio, just do it as instructed, assume user knows what he's doing)

--update-sar if user wants to change the sar value from whatever is default in the source
--ignore-sar or --ignore-input-sar to ignore the sar specified in the input

or --input-sar and --output-sar, first to override the input sar, second to set the output sar, without any resizing performed

Irfanview has in Batch convert mode two additional options "Resize the longest side" and "Resize the shortest side", so maybe --resize-long and --resize-short and have the other dimension automatically adjusted to mantain proportion with the source (i'd say ignore sar in this case or automatically set it to 1:1 unless user sets --output-sar then)

I think most users would assume when they resize the image they'd have the sar to 1:1...
For example... someone gets a DVD, runs DVD decrypter on it and generates a single VOB file.. Opens it in Video Lan Client and looks in the messages window to see the width and height of the window VLC created (or uses Alt+PrintScr), so then they'd just use --resize widthxheight expecting image will be at that dimensions but if x264 preserves the sar from the vob file then result is messed up.

ps. I also think the resizing algorithm should be selectable, unless the speed difference between them is minimal. I personally use bicubic a lot when resizing from big resolution to lower resolution and when video goes to youtube type of sites. For personal backups I use Lanczos.

[buzzqw]

btw why not using the mencoder approach ?
"scale" and "crop" filters are easy to use (like crop=720:432:0:72,scale=640:272 or crop=720:432:0:72,scale=640:-2)

or totally explicit as ffmpeg

(even autocrop could be usefull too :D )

BHH

[Sharc]

Whatever we do I suggest one should stick to the formula:
New SAR = Original SAR x (resized_height x cropped_width) : (cropped_height x resized_width).
Otherwise we will end up in producing distorted pictures.
The 'original SAR' needs to be taken from a set of standard values for commercial media. (e.g. for BD, PAL DVD and NTSC DVD sources).

[ACoolie]

Although this is OT, I like the GraphicsMagick command line options for scaling. By default, --resize works how DS requested. By adding an "!" to the end of the string, resize scales the video to that exact size. Ordinary users are protected, and if you know what you are doing it's easy to override.

[easyfab]

Thanks for LAVF/FFMS input .

As I most of the time leave the defaults, could it be possible to also have a default output file (input: exemple.avi -> output: exemple.mkv or input :exemple.mkv -> output : exemple(1).mkv ) to enable the use of drag and drop.

[LoRd_MuldeR]

Although this is OT, I like the GraphicsMagick command line options for scaling. By default, --resize works how DS requested. By adding an "!" to the end of the string, resize scales the video to that exact size. Ordinary users are protected, and if you know what you are doing it's easy to override.

Sounds like a feasible solution, also for x264. However I (personally) would prefer the meaning of the "!" reversed :p

[Chengbin]

Sounds like a feasible solution, also for x264. However I (personally) would prefer the meaning of the "!" reversed :p

+1

I think this makes more sense too.

[dwrbudr]

What about --autoresize WxH or --fittobox WxH

[buzzqw]

the "!" character iirc has special pourpose on DOS shell...

BHH

[prOnorama]

What about cropping?

It's the main filter I use -most 2.35:1 and 1.85:1 movies need cropping- along with resizing (via AviSynth currently)

Some kind of preview for cropping would be nice so I can see what's it's gonna be like before encoding (I currently use GraphStudio + AvsP and save to .avs), I guess a lot of people use a preview function in a GUI they use for crop/resize preview

[Dark Shikari]

What about cropping?

It's the main filter I use -most 2.35:1 and 1.85:1 movies need cropping- along with resizing (via AviSynth currently)

Some kind of preview for cropping would be nice so I can see what's it's gonna be like before encoding (I currently use GraphStudio + AvsP and save to .avs), I guess a lot of people use a preview function in a GUI they use for crop/resize previewIIRC, swscale does cropping too, so that should be no problem.

[Selur]

2. "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video. No more timecode files!
Would be cool if x264 could also create a timecode file, so that users who prefer (e.g. because they want to add multiple audiostream and a bunch of tag infos) to manually multiplex with mkvmerge/mp4box could use it.

[bnshrdr]

I tried the beta executable in the first post on a typical NTSC DVD vob file. I don't know if this is a problem in just this build, but when I tell it to encode straight to an MP4 container, the video is all blocky. If I tell it to go straight to raw 264 stream, and mux it with YAMB, it does nicely.

YAMB output:
http://img695.imageshack.us/img695/7128/outx.png

x264 output:
http://img684.imageshack.us/img684/9812/out2.png

Edit: Nevermind about the frame rate issue, it was variable and I was looking at it incorrectly.

Edit2: Matroska exports work fine.

[fields_g]

Sounds similar to what I reported yesterday in another (http://doom10.org/index.php?topic=133.msg996#msg996) conversation.

[bnshrdr]

I think it's exactly what you posted. I used --demuxer lavf and it worked just fine. I didn't go in depth and analyze timestamps or anything, but my results seem to mirror yours.