x264 LAVF/FFMS input public beta test (v2.1)

[Dark Shikari]

Now that we've had a small public test (http://doom10.org/index.php?topic=133) for version 1, we need as much testing as possible done on version 2 of the LAVF/FFMS input patch. In short, this input patch lets you not only take input from nearly any file without using Avisynth, but also allows x264 to natively handle variable framerate video without timecode files.

What we need you to do: test with any input you can find, especially inputs that might be hard or otherwise suspect to breaking. Report any bugs you find, but make sure to check the known bugs list below so you don't report things that we already know about.

Download v2.1 (http://mirror05.x264.nl/Dark/x264_lavfffms2.exe)
Patch v2.1 (http://pastebin.com/m7b096160)

Download v1 (http://kemuri9.net/dev/x264/x264_lavf+ffms.zip)
Patch v1 (http://pastebin.com/m4ef9fb74)

New features (v2.1):

1. Input from anything, even without Avisynth or DirectShow codecs, even on Linux!
2. "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video. No more timecode files!
3. Use --demuxer to force a particular input method (lavf, ffms, etc).
4. Automatic handling of all kinds of weird input (changing resolution, etc), including many inputs that Avisynth wouldn't have worked correctly with.
5. A near-complete rewrite of the x264cli internals to handle the above.
6. Periodic Intra Refresh.

See the three commit messages in the patch for more details.

Gotchas (v2.1):

1. There seem to be some types of files (raw h264?) that FFMS refuses to index. We'll be looking at that, but LAVF should work fine in the meantime.
2. FFMS won't work on piped input.
3. x264 doesn't by default save the index file from FFMS, so it has to re-index on every pass unless you use --index. We may change the default behavior in the future.

Known bugs (v2.1)
<none>

Now that we're through the gotchas, feel free to test on various types of input and report any issues you have.

Disclaimer: this is a HUGE patch. It modifies dozens of files and likely has a lot of bugs. Problems are normal, and the purpose of this test is to root out as many as possible.

Mega-thanks to Mike Gurlitz (ACoolie), Steven Walters (Kemuri9), and Kieran Kunhya (Kierank) for the massive amount of high-quality work that went into this patch.

[mariush]

windows 2003, 32 bit, 4 gb of memory, avisynth installed 2.5.8 i think... source is 720p 50 fps i think

x264_lavfffms.exe --bitrate 1024 -o e:\test.mkv e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv

http://savedonthe.net/image/488/crash.png

vs 2010 debug shows this..

http://savedonthe.net/image/489/Clipboard02.png

process explorer shows this...

http://savedonthe.net/image/490/procexp.png

it wouldn't surprise me to have made something wrong, so please let me know if that's the case.. and if there's any way I can help you out in more detail I'll be glad to do it.

later edit: is it supposed to actually read mkv files? i see it doesn't crash with .avi files...

even later edit: on avi files the console window says in the taskbar indexing input but after it finishes that and begins to encode content it still says the same thing.. I think it used to show encoding progress. In the console window itself, it does show the status correctly
(and excellent news and thanks for the cool things you add to x264!)

[Dark Shikari]

Confirmed, FFMS seems to be crashing on MKV files. We'll look into it.

If FFMS crashes, try --demuxer lavf and see if that works instead.

[mariush]

ok, it worked with --demuxer lavf but noticed it wasn't able to show the percent... went counting frames:


x264_lavfffms.exe --demuxer lavf --bitrate 1024 --output e:\test.mkv "e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv"
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
Last message repeated 5 times
[matroska @ 003dbda0]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1280x720p 1:1 @ 50/1 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.2
x264 [warning]: non-monotonic pts at frame 837 (17640 < 18520)
x264 [info]: frame I:9 Avg QP:25.74 size: 32864
x264 [info]: frame P:639 Avg QP:27.54 size: 6404
x264 [info]: frame B:1306 Avg QP:32.33 size: 967
x264 [info]: consecutive B-frames: 4.8% 15.9% 3.4% 75.9%
x264 [info]: mb I I16..4: 25.7% 63.1% 11.1%
x264 [info]: mb P I16..4: 4.2% 2.9% 0.1% P16..4: 38.5% 5.4% 3.6% 0.0% 0
.0% skip:45.4%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 23.6% 0.1% 0.1% direct:
0.1% skip:75.8% L0:33.9% L1:65.6% BI: 0.5%
x264 [info]: final ratefactor: 29.93
x264 [info]: 8x8 transform intra:44.4% inter:79.0%
x264 [info]: coded y,uvDC,uvAC intra: 19.0% 32.7% 7.0% inter: 4.0% 7.1% 0.1%
x264 [info]: i16 v,h,dc,p: 44% 27% 9% 21%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 45% 3% 4% 4% 5% 3% 7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 28% 14% 4% 7% 5% 7% 4% 6%
x264 [info]: Weighted P-Frames: Y:20.7%
x264 [info]: ref P L0: 64.8% 8.6% 19.6% 3.6% 3.4%
x264 [info]: ref B L0: 95.3% 4.7%
x264 [info]: kb/s:1156.81

encoded 1954 frames, 28.38 fps, 1099.16 kb/s


File plays fine in MPC-HC with the dxva decoder though at some point it goes blocky for a couple of frames in the middle - maybe it got corrupt or it's just the way the movie was... if i remember correctly i got the sample from ffmpeg's collection of test clips

question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?

[Dark Shikari]

Bug found in FFMS that caused the problem. Fix can be found here (http://pastebin.com/d79761bd4
); build in the topic updated to v2.1 accordingly.question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?Yes, it should, and as far as I can tell it works correctly.

[mariush]

Now it works with MKV input but the aspect ratio is ignored when using --demuxer lavf command:


x264_lavfffms2.exe --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
ffms [info]: 720x432p 37:26 @ 25/1 fps (vfr)
x264 [info]: using SAR=37/26



x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
[matroska @ 003dbd70]Estimating duration from bitrate, this may be inaccurate
lavf [info]: 720x432p 0:1 @ 25/1 fps (vfr)

If you want, I can upload a small 20s clip but you probably already have some test clips for this.

I've also noticed some differences when using that short 1080p clip with the migrating white ducks or whatever birds they are:


x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
Last message repeated 29 times
[matroska @ 003dbd90]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)



x264_lavfffms2.exe --frames 100 --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
ffms [info]: 1920x1080p 1:1 @ 4000/167 fps (vfr)
x264 [info]: using SAR=1/1

[Firebird]

http://img254.imageshack.us/img254/8404/x264tests.png
Just stops on 318 frame.

Sample (http://www.mediafire.com/?jzlkzzhytmx)

[Dark Shikari]

http://img254.imageshack.us/img254/8404/x264tests.png
Just stops on 318 frame.

Sample (http://www.mediafire.com/?jzlkzzhytmx)Did you update to v2.1? That sample works fine here.

[Firebird]

Did you update to v2.1? That sample works fine here.
v2.1 fails, even with --demuxer avs
Latest stable build is working. Hm. I will test it more, maybe it's a bug of mine.

[chipzoller]

Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?

[Dark Shikari]

Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?It can open anything ffmpeg can open. Read: everything.

[prOnorama]

It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMO

[Dark Shikari]

It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMOYou can rename it, it's just what I named the executable on the server ;)

[roozhou]

DS, thanks for the awesome New Year's present!
Is vfr ratecontrol implemented in this patch?

[kemuri-_9]

Is vfr ratecontrol implemented in this patch?

no, it will come shortly later

[Episodio1]

I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?

[avivahl]

I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?Well, it probably uses this library (http://code.google.com/p/ffmpegsource/) to wrap around ffmpeg (http://ffmpeg.org/)'s decoding capabilities. I guess that's about it.

[ACoolie]

lavf is short for libavformat, which is the library ffmpeg and x264 use to open and decode videos. This demuxer is attempted last because it can open all files ffms can, as well as pipefiles, but ffms typically does it better. It doesn't actually read the entire stream before encoding has begun, so frame count is unknown (no percent progress), and fps is taken from the container header, which could be wrong for VFR video. To explicitly use this demuxer, use `x264 --demuxer lavf`, although you should only use it for testing or if ffms2 does not work. Also, --seek will be slow because frames must be read sequentially.

ffms2 is Fabulous FM Source 2 (or FFmpegSource2), a wrapper of lavf that indexes files to provide a better API than libavformat has. It will read the entire input file, which can take considerable time for large files, so you should pass `--index indexfile` to save this index across passes. ffms has fast seeking, and should theoretically give the correct average fps for vfr video (we're still working on this). We've noticed a lot of the bugged input files are caused by this library (which is good, since we can fix them), so the lavf demuxer sometimes must be used on certain files and filetypes. You can force this demuxer to be used with `--demuxer ffms`, but typically it will be the default demuxer if it can be used.

[Sharc]

.........
I guess it won't work for interlaced VC-1 though (hard to find a sample)

Confirmed. Didn't work on interlaced VC-1 here. Encoder runs but produces only a flat gray at playback.
AVC interlaced seems to be ok.

[roozhou]

@kemuri-_9

How did you calculate dts in mp4 output for vfr video when b-frames are used? It seems initdelay is still used.

[chipzoller]

What's a recommended way to use this build to open a DVD-encode project in substitute of AviSynth? Do we feed it d2v project files?

[Dark Shikari]

What's a recommended way to use this build to open a DVD-encode project in substitute of AviSynth? Do we feed it d2v project files?Correct.

[chipzoller]

What about resizing? If correcting for PAR (no upsizing) are the built-in resizing algorithms just as effective as some of those for Avisynth?

[LoRd_MuldeR]

I think if you want to apply filters, such as resize, you must still use Avisynth input...

[wOxxOm]

ffms2 has built-in resizers (lanczos & spline included, as well as much much more options) in its API, so will it be possible to specify those in x264lavf/ffms cli command line?

[Dark Shikari]

We'll be adding resizing support eventually. My thought is we'll have a few resizing modes (just an idea):

--resize WxH: resizes such that the size does not exceed W or H. Maintains aspect ratio: the input is not stretched, and by default, the SAR stays the same as the input too. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --resize 1280x720
Output: 960x720 SAR 4:3

--input-sar: Overrides the SAR of the input file. Works the way SAR does currently. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --input-sar 1:1
Output: 1440x1080 SAR 1:1

--output-sar: Modifies the resizing accordingly, in addition to setting the SAR. So, for example:

Input: 1440x1080, SAR 4:3
Settings: --output-sar 1:1
Result: 1920x1080, SAR 1:1

[kemuri-_9]

@kemuri-_9

How did you calculate dts in mp4 output for vfr video when b-frames are used? It seems initdelay is still used.

I didn't touch the output muxers, i mostly dealt with the demuxer side of things.
should ask kierank/VFR_maniac.
VFR_maniac was the one who handled most of the updated mp4 code afaik.

[wOxxOm]

We'll be adding resizing support eventually. My thought is we'll have a few resizing modes (just an idea)
cool))
just make sure that "--output-sar 1:1" alone won't resize the stream :p
And, resize quality (spline/bicubic/etc) selectors would be vital.

[Dark Shikari]

cool))
just make sure that "--output-sar 1:1" alone won't resize the stream :p Why?

--input-sar will be aliased to the old --sar option, so --output-sar will be the new option (with new behavior).

[wOxxOm]

so --output-sar will be the new option (with new behavior)as you see fit, sir. I just thought that a switch that enables and does the resizing should have corresponding mnemonics, AND - what if I want to both resize and use some specific SAR for the output? e.g. resize from 1440x1080 to 720x480 and set 4:3 SAR... I'm not sure I see an intuitive way of doing it along the proposed scheme, but i am not too smart either :p

[LoRd_MuldeR]

--resize WxH: resizes such that the size does not exceed W or H. Maintains aspect ratio: the input is not stretched, and by default, the SAR stays the same as the input too. So, for example:
Input: 1440x1080, SAR 4:3
Settings: --resize 1280x720
Output: 960x720 SAR 4:3

Sorry, but I think that would be really confusing. If I enter "--resize WxH" then I'd expect the output at exactly WxH. That's how any application I'm aware of works.

Maybe something like "--resize-width W" and "--resize-height H" could be added for resize that automatically maintains the correct aspect ratio?

Or just a placeholder, like MEncoder's resize option. For example "--resize -1:720" would resize to Wx720 with W chosen automatically to maintain the correct AR.

[Dark Shikari]

Sorry, but I think that would be really confusing. If I enter "--resize WxH" then I'd expect the output at exactly WxH. That's how any application I'm aware of works.The number one complaint I've gotten about ffmpeg, from hundreds upon hundreds of users, is exactly the opposite of what you said: that, by default, resizing will stretch the image.

I don't know a single image editing program that, by default, allows you to stretch the image when resizing. I don't see why x264 should be any different. 99.9% of users do not want to stretch the video, so why should we design the program around the 0.1% who do?

[LoRd_MuldeR]

The number one complaint I've gotten about ffmpeg, from hundreds upon hundreds of users, is exactly the opposite of what you said: that, by default, resizing will stretch the image.

I don't know a single image editing program that, by default, allows you to stretch the image when resizing. I don't see why x264 should be any different. 99.9% of users do not want to stretch the video, so why should we design the program around the 0.1% who do?

There's an important difference:

If I use the resize filter in an image editing program, which has a GUI(!), and I enter the target size, then the "width" edit box is updated in realtime when I change the value in the "height" box - and vice versa. So the user will always see what he'll get as the final result and he can uncheck the "maintain aspect ratio" option at any time. But when I explicitly(!) pass "--resize WxH" to a CLI application, it would be really unorthodox to get something different from WxH, because there is no indication that "--resize WxH" actually means "--resize-to-the-biggest-size-that-accurately-fits-into WxH". When I explicitly(!) specify a value, I'm expecting to get that value and nothing else. And again: That's how any CLI application I'm aware of works. GUI applications are a different topic...

[Dark Shikari]

OK, then suggest a better method. I'm open ideas; the above I came up with in about 5 minutes. The method must have the following properties:

1. It must be conducive to batch encoding of many different input files, in which a user has a maximum size he wants.
2. It must minimize the number of additional parameter options.
3. It must, by default, prevent the user from inadvertently stretching his videos.
4. It must provide some way to change the SAR, including both a way to change the SAR while not changing the resolution (stretching), and a way to change the SAR while changing the resolution too (not stretching).

[LoRd_MuldeR]

Well, I can think of three methods:
1. Use "--resize WxH" for explicit resizing and use "--resize-width W" respectively "--resize-height H" for automated/managed resize that will keep the AR
2. Use "--resize WxH" for explicit resizing and use "--resize-maximum WxH" alias "--target-screen-size WxH" alias "--managed-resize WxH" for automated/managed resize that will keep the AR
3. Use "--resize WxH" for explicit resizing and use "--resize ?xW" respectively "--resize Hx?" for automated/managed resize that will keep the AR

[mariush]

I think resizing shouldn't influence the sar values, it may be too confusing for users (though most would use GUIs)

--resize widthxheight (don't care about aspect ratio, just do it as instructed, assume user knows what he's doing)

--update-sar if user wants to change the sar value from whatever is default in the source
--ignore-sar or --ignore-input-sar to ignore the sar specified in the input

or --input-sar and --output-sar, first to override the input sar, second to set the output sar, without any resizing performed

Irfanview has in Batch convert mode two additional options "Resize the longest side" and "Resize the shortest side", so maybe --resize-long and --resize-short and have the other dimension automatically adjusted to mantain proportion with the source (i'd say ignore sar in this case or automatically set it to 1:1 unless user sets --output-sar then)

I think most users would assume when they resize the image they'd have the sar to 1:1...
For example... someone gets a DVD, runs DVD decrypter on it and generates a single VOB file.. Opens it in Video Lan Client and looks in the messages window to see the width and height of the window VLC created (or uses Alt+PrintScr), so then they'd just use --resize widthxheight expecting image will be at that dimensions but if x264 preserves the sar from the vob file then result is messed up.

ps. I also think the resizing algorithm should be selectable, unless the speed difference between them is minimal. I personally use bicubic a lot when resizing from big resolution to lower resolution and when video goes to youtube type of sites. For personal backups I use Lanczos.

[buzzqw]

btw why not using the mencoder approach ?
"scale" and "crop" filters are easy to use (like crop=720:432:0:72,scale=640:272 or crop=720:432:0:72,scale=640:-2)

or totally explicit as ffmpeg

(even autocrop could be usefull too :D )

BHH

[Sharc]

Whatever we do I suggest one should stick to the formula:
New SAR = Original SAR x (resized_height x cropped_width) : (cropped_height x resized_width).
Otherwise we will end up in producing distorted pictures.
The 'original SAR' needs to be taken from a set of standard values for commercial media. (e.g. for BD, PAL DVD and NTSC DVD sources).

[ACoolie]

Although this is OT, I like the GraphicsMagick command line options for scaling. By default, --resize works how DS requested. By adding an "!" to the end of the string, resize scales the video to that exact size. Ordinary users are protected, and if you know what you are doing it's easy to override.

[easyfab]

Thanks for LAVF/FFMS input .

As I most of the time leave the defaults, could it be possible to also have a default output file (input: exemple.avi -> output: exemple.mkv or input :exemple.mkv -> output : exemple(1).mkv ) to enable the use of drag and drop.

[LoRd_MuldeR]

Although this is OT, I like the GraphicsMagick command line options for scaling. By default, --resize works how DS requested. By adding an "!" to the end of the string, resize scales the video to that exact size. Ordinary users are protected, and if you know what you are doing it's easy to override.

Sounds like a feasible solution, also for x264. However I (personally) would prefer the meaning of the "!" reversed :p

[Chengbin]

Sounds like a feasible solution, also for x264. However I (personally) would prefer the meaning of the "!" reversed :p

+1

I think this makes more sense too.

[dwrbudr]

What about --autoresize WxH or --fittobox WxH

[buzzqw]

the "!" character iirc has special pourpose on DOS shell...

BHH

[prOnorama]

What about cropping?

It's the main filter I use -most 2.35:1 and 1.85:1 movies need cropping- along with resizing (via AviSynth currently)

Some kind of preview for cropping would be nice so I can see what's it's gonna be like before encoding (I currently use GraphStudio + AvsP and save to .avs), I guess a lot of people use a preview function in a GUI they use for crop/resize preview

[Dark Shikari]

What about cropping?

It's the main filter I use -most 2.35:1 and 1.85:1 movies need cropping- along with resizing (via AviSynth currently)

Some kind of preview for cropping would be nice so I can see what's it's gonna be like before encoding (I currently use GraphStudio + AvsP and save to .avs), I guess a lot of people use a preview function in a GUI they use for crop/resize previewIIRC, swscale does cropping too, so that should be no problem.

[Selur]

2. "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video. No more timecode files!
Would be cool if x264 could also create a timecode file, so that users who prefer (e.g. because they want to add multiple audiostream and a bunch of tag infos) to manually multiplex with mkvmerge/mp4box could use it.

[bnshrdr]

I tried the beta executable in the first post on a typical NTSC DVD vob file. I don't know if this is a problem in just this build, but when I tell it to encode straight to an MP4 container, the video is all blocky. If I tell it to go straight to raw 264 stream, and mux it with YAMB, it does nicely.

YAMB output:
http://img695.imageshack.us/img695/7128/outx.png

x264 output:
http://img684.imageshack.us/img684/9812/out2.png

Edit: Nevermind about the frame rate issue, it was variable and I was looking at it incorrectly.

Edit2: Matroska exports work fine.

[fields_g]

Sounds similar to what I reported yesterday in another (http://doom10.org/index.php?topic=133.msg996#msg996) conversation.

[bnshrdr]

I think it's exactly what you posted. I used --demuxer lavf and it worked just fine. I didn't go in depth and analyze timestamps or anything, but my results seem to mirror yours.

[VFR maniac]

Would be cool if x264 could also create a timecode file, so that users who prefer (e.g. because they want to add multiple audiostream and a bunch of tag infos) to manually multiplex with mkvmerge/mp4box could use it.

How about this Win32 binary?
http://www.esnips.com/doc/1a661a92-01eb-417e-b68f-c8f34235cf7f/x264_vfr_rev1379_release3
Mirror (megaupload)
http://www.megaupload.com/?d=LY5OXCR3

The patch is here.
http://pastebin.com/m6d750da9
I am managing timecode format file I/O.

[Brazil2]

How about this Win32 binary?
http://www.esnips.com/doc/1a661a92-01eb-417e-b68f-c8f34235cf7f/x264_vfr_rev1379_release3

How do you download this file ? Looks like the site wants me to sign in and to enter a password.
Please could you upload it elsewhere (Megaupload) ?

[Dark Shikari]

Would be cool if x264 could also create a timecode file, so that users who prefer (e.g. because they want to add multiple audiostream and a bunch of tag infos) to manually multiplex with mkvmerge/mp4box could use it.We will merge VFRManiac's timecode patch after this is done.

[roozhou]

Would be cool if x264 could also create a timecode file, so that users who prefer (e.g. because they want to add multiple audiostream and a bunch of tag infos) to manually multiplex with mkvmerge/mp4box could use it.

This is useless because:
1) mkvmerge does not need timecodes file because it correctly handles pts from input files.
2) MP4Box does not support matroska timecode files

[Stephen R. Savage]

This is only tangentially related to the thread, but could a "--timecodes" parameter be added to x264, so it can load external timecodes (e.g. v1/v2 Matroska) for use with input methods other than LAVF/FFMS? This would allow for bitrate-based control over input formats that are CFR in nature (e.g. Avisynth, YUV, Y4M). This would eliminate the need to calculate the average framerate of these sources, and would allow an accurate bitrate report in the output.

[Dark Shikari]

This is only tangentially related to the thread, but could a "--timecodes" parameter be added to x264, so it can load external timecodes (e.g. v1/v2 Matroska) for use with input methods other than LAVF/FFMS? This would allow for bitrate-based control over input formats that are CFR in nature (e.g. Avisynth, YUV, Y4M). This would eliminate the need to calculate the average framerate of these sources, and would allow an accurate bitrate report in the output.Yes, that's what VFRmaniac's patch does. We'll be adding support for that.

[buzzqw]

fyi i added initial support for this x264 build to my MicroX264.

now is possible to select demuxer (no audio or resize is done)

BHH

[buzzqw]

--demuxer avs and --demuxer ffms detect different framerate even if both using ffmpegsource

C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer ffms vc1.fps.problem.m2ts
ffms [info]: 1920x1080p 1:1 @ 30000/1001 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile High, level 4.0
^C
C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer avs vc1.fps.problem.m2ts
avs [info]: trying FFmpegSource2... indexing... succeeded
avs [info]: 1920x1080p 0:0 @ 250000/8211 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64

C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer lavf vc1.fps.problem.m2ts
[mpegts @ 0030a6b0]MAX_READ_SIZE:5000000 reached
lavf [info]: 1920x1080p 1:1 @ 30000/1001 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64

sample http://www.64k.it/andres/data/Varie/vc1.fps.problem.m2ts

BHH

[Selur]

1) mkvmerge does not need timecodes file because it correctly handles pts from input files.
2) MP4Box does not support matroska timecode files
Sad. Hoped for an easy way to create vfr files from cfr input in linux (without needing wine and avisynth) by:
1. reencode input with x264 (creating a .264 and a timecode file)
2. convert audio (creating a bunch of .mp3, .aac, .ac3 files)
3. multiplex subtitles, audio, video with the timecode file and a bunch of tags

Would it be hard to output mp4 time code files?

[roozhou]

--demuxer avs and --demuxer ffms detect different framerate even if both using ffmpegsource

C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer ffms vc1.fps.problem.m2ts
ffms [info]: 1920x1080p 1:1 @ 30000/1001 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile High, level 4.0
^C
C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer avs vc1.fps.problem.m2ts
avs [info]: trying FFmpegSource2... indexing... succeeded
avs [info]: 1920x1080p 0:0 @ 250000/8211 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64

C:\PROGRA~1\PUREBA~2\_TESTF~1\NUOVAC~1>x264.exe --output zzz.mkv --demuxer lavf vc1.fps.problem.m2ts
[mpegts @ 0030a6b0]MAX_READ_SIZE:5000000 reached
lavf [info]: 1920x1080p 1:1 @ 30000/1001 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64

sample http://www.64k.it/andres/data/Varie/vc1.fps.problem.m2ts

BHH

That's normal because avs does not support vfr and requires an average fps.

With true vfr support the fps value does not really matter, you can use any value without losing a/v sync.

[roozhou]

Sad. Hoped for an easy way to create vfr files from cfr input in linux (without needing wine and avisynth) by:
1. reencode input with x264 (creating a .264 and a timecode file)
2. convert audio (creating a bunch of .mp3, .aac, .ac3 files)
3. multiplex subtitles, audio, video with the timecode file and a bunch of tags

Would it be hard to output mp4 time code files?
You don't need it because we have ffmpeg. Timecodes are only needed when you are using outdated frameworks that does not correctly handle timestamps.(e.g. avisynth, vfw)

1. encode input with x264 and create a vfr mp4
2. convert audio
3. multiplex subtitle, audio and video using ffmpeg

I know there are several bugs in ffmpeg's muxers, but one day ffmpeg will handle all these things.

[bob0r]

Patch used: "x264.LAVF.FFMS.input.public.beta.test.v2.2.diff", it's not public and probably not v2.2 eighter, i just saved it as 2.2 instead of 2.1 (laziness).
This patch was given to me by Dark to fix a x264 configure issue.

- pthreads, gpac, ffmpeg and ffmpegsource all compiled with gcc 4.4.2 aswell.
- upx --lzma --best x264.exe

x264.1376.LAVF.FFMS.gcc.4.4.2.make.exe (http://x264.nl/x264.1376.LAVF.FFMS.gcc.4.4.2.make.exe)
x264.1376.LAVF.FFMS.gcc.4.4.2.make.upx.exe (http://x264.nl/x264.1376.LAVF.FFMS.gcc.4.4.2.make.upx.exe)
x264.1376.LAVF.FFMS.gcc.4.4.2.make.fprofiled.exe (http://x264.nl/x264.1376.LAVF.FFMS.gcc.4.4.2.make.fprofiled.exe)
x264.1376.LAVF.FFMS.gcc.4.4.2.make.fprofiled.upx.exe (http://x264.nl/x264.1376.LAVF.FFMS.gcc.4.4.2.make.fprofiled.upx.exe)

What do we want to know? Are there any (encoding) speed differences?
Do they work at all? Compare them to Dark's .exe.

[roozhou]

What do we want to know? Are there any (encoding) speed differences?
Do they work at all? Compare them to Dark's .exe.

I don't think we should care about speed atm but focus on fixing various bugs concerning ffmpeg input. Is there any improvement in v2.2?

[kemuri-_9]

I don't think we should care about speed atm but focus on fixing various bugs concerning ffmpeg input. Is there any improvement in v2.2?

"2.2" is outdated, should wait on the current revision.


- pthreads, gpac, ffmpeg and ffmpegsource all compiled with gcc 4.4.2 aswell.

and supply what options you gave to configure for your ffmpeg compile....

[bob0r]

I don't think we should care about speed atm but focus on fixing various bugs concerning ffmpeg input. Is there any improvement in v2.2?

Speed between the versions, not speed input wise. :)

And yeah the patch is outdated, just need to know if it runs at all. (If there are no GCC 4.4.2 issues.)

[roozhou]

"2.2" is outdated, should wait on the current revision.


Where is the current version?

[Dark Shikari]

A lot of the "bugs" at this point are bugs in FFMS and in LAVF. Most programs don't run into these bugs because they use constant framerate, or they hack around them in ugly fashion.

For example, we have issues with:

1) LAVF returning invalid PTS (duplicate PTS, and sometimes AV_NOPTS_VALUE)
2) FFMS returning wrong FPS and broken PTS on telecined files
3) FFMS breaking on raw video AVIs
4) FFMS breaking on .h264 files

[LoRd_MuldeR]

I hope all the FFMS problems can be fixed and backported to the standalone FFMS2 plugin for Avisynth. Better FFMS2 source would be great :)

[popper]

You don't need it because we have ffmpeg. Timecodes are only needed when you are using outdated frameworks that does not correctly handle timestamps.(e.g. avisynth, vfw)

1. encode input with x264 and create a vfr mp4
2. convert audio
3. multiplex subtitle, audio and video using ffmpeg

I know there are several bugs in ffmpeg's muxers, but one day ffmpeg will handle all these things.

as i understand it <kierank> intends at some point to try and make a generic DVB-* style 'Stat Mux' capable TS (Transport Stream) framework for x264, so industrial Timecode capability and even New stat mux and related timecode insertion/editing apps if the need arises will be useful in the future for many AVC x264 things it seems, although its not clear exactly how he will impliment it yet!

BTW why dont you spend time on the #x264dev IRC roozhou ?,
http://gogloom.com/client2/index2?mainCHAT=1&network=FreeNode&channel=%23x264dev
your coding skills will help out there for review and patchs etc it seems.

[SledgeHammer_999]

I hope all the FFMS problems can be fixed and backported to the standalone FFMS2 plugin for Avisynth. Better FFMS2 source would be great :)

I second that!

Also, (I haven't tested it) but does it handle subs too?(hardsubbing them mainly)

[kemuri-_9]

as the avs plugin is a wrapper around the FFMS2 API, the same API that x264 uses, any issues found and fixed should also be fixed for the avisynth plugin.
Also, (I haven't tested it) but does it handle subs too?(hardsubbing them mainly)
no.

[roozhou]

I second that!

Also, (I haven't tested it) but does it handle subs too?(hardsubbing them mainly)

I have improved vsfilter.dll's csri interface and hardsub works perfectly with my dshow x264 build under windows. As soon as ffmpeg input in x264 is finished, this can be easily port to lavf input. However hardsub requires true timestamp for each frame not fake cfr or average fps.

[buzzqw]

a request: could be added a --size XXX parameter ?

using this option x264 will compute the bitrate (on 2 pass encoding) to achive the size requested

thanks

BHH

[Sharc]

a request: could be added a --size XXX parameter ?

using this option x264 will compute the bitrate (on 2 pass encoding) to achive the size requested

I one sets --bitrate yyyyy --size xxxx: who will be the winner? The last parameter superseding the first one?

[sneaker_ger]

I have improved vsfilter.dll's csri interface and hardsub works perfectly with my dshow x264 build under windows. As soon as ffmpeg input in x264 is finished, this can be easily port to lavf input. However hardsub requires true timestamp for each frame not fake cfr or average fps.

Can we expect this to be ported into FFmpegSource (https://forum.doom9.org/showthread.php?t=127037)?

[LoRd_MuldeR]

a request: could be added a --size XXX parameter ?

using this option x264 will compute the bitrate (on 2 pass encoding) to achive the size requested

thanks

BHH

IMO a "--size" parameter would be kind of superfluous. All it would do is set "--bitrate" to "size/duration", which can be calculated easily before calling x264.

Also x264 doesn't always know the duration in advance! As far as I understood, it doesn't know the framecount in advance with the "lavc" demuxer, only with the "ffms2" demuxer. And of course with Avisynth.

It doesn't know the number of frames for sure when reading source from STDIN, unless the user passes "--frames" explicitly. So how should "--size" behave in those situations ???

I one sets --bitrate yyyyy --size xxxx: who will be the winner? The last parameter superseding the first one?

In that case, if "--size" really is added, x264 should abort with error, because the user has set contradictory options...

[J_Darnley]

In that case, if "--size" really is added, x264 should abort with error, because the user has set contradictory options...

x264 doesn't abort at present if you do --bitrate x --crf y. The latter option would be used. Whether it should, I don't know.

But I agree, with your statement that it is pointless.

[bnshrdr]

In my opinion, the whole --size option shouldn't be part of the program that does the encoding, but part of a front-end, as it has been done in the past. (MeGUI, Staxrip, and others....)

But then again, I have the whole linux way of looking at things, one program should have one task. I don't want to see my media players turn into web browsers :).

[roozhou]

It doesn't know the number of frames for sure when reading source from STDIN, unless the user passes "--frames" explicitly. So how should "--size" behave in those situations ???


--size should work in 2nd pass.

[kypec]

In my opinion, the whole --size option shouldn't be part of the program that does the encoding, but part of a front-end, as it has been done in the past. (MeGUI, Staxrip, and others....)
My words exactly. If someone is not able to calculate the most simple algebra of elementary school (Bitrate = Size / Duration hence Size = Bitrate * Duration) he shouldn't be using x264 at all.:p
It really goes beyond my comprehension why an author of multiple x264 GUI's [buzzqw] would have requested such useless option...:confused:

[buzzqw]

It really goes beyond my comprehension why an author of multiple x264 GUI's [buzzqw] would have requested such useless option...
because personally i don't use gui, only command line and notepad

BHH

[roozhou]

Can someone provide a compilation guide on lavf and ffms for x264?

[kypec]

because personally i don't use gui, only command line and notepad
BHH
<IRONICMOOD>Well, in that case I encourage you to try another amazing tool which could help you a lot -> calculator.</IRONICMOOD>

[ACoolie]

I'm sure size/duration works for the general case, but if you've ever looked at the code for the MeGUI bitrate calculator (which, in my experience has been somewhat more accurate than my own calculations), its not that simple. I'm not sure if it got into this patch, but someone suggested output.write_frame return the amount of new data written (or the written size of the frame?), which would make a --size option most accurate to implement, although I probably wouldn't want it committed.
Anyways, congrats on getting this done :)

[elguaxo]

can someone provide a compilation guide on lavf and ffms for x264?

+1 :)

[VFR maniac]

MSYS/MinGW32 (for static build)

pthreads-win32 (2.9.0.0 autostatic)
Patch this (http://pastebin.com/m476bc29).
make clean GC-autostatic
cp libpthreadGC2.a /usr/local/lib
cp pthread.h sched.h semaphore.h /usr/local/include


ffmpeg (rev21392) and libswscale (rev30392)
Note: If you didn't build pthreads-win32 as autostatic, remove --enable-pthreads. If you built it as autostatic, patch this (http://pastebin.com/maa04d5f).
./configure --enable-pthreads --enable-gpl --enable-postproc --enable-memalign-hack --enable-runtime-cpudetect --disable-devices --disable-filters --disable-encoders --disable-muxers --disable-network --disable-decoder=aac,ac3,adpcm_*,alac,als,ape,atrac?,cook,dca,dsicinaudio,dxa,eac3,flac,interplay_dpcm,mlp,mp1,mp2,mp3,mp3*,mpc?,pcm_*,qcelp,ra_*,sipr,truehd,truespeech,tta,vorbis,wavpack,wma*,twinvq --disable-demuxer=aac,ac3,pcm_*,ape,amr,ass,au,avs,dts,eac3,flac,mp3,mpc,mpc8,truehd,tta,w64,wav,wv --disable-parser=aac,ac3,dca,mlp,mpegaudio --prefix=/usr/local --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib"
make
make install



ffms2 (rev262)
Requirments: zlib (http://sourceforge.net/projects/mingw/files/MinGW%20zlib/zlib-1.2.3-1-mingw32/zlib-1.2.3-1-mingw32-src.tar.gz/download), bzip2 (http://sourceforge.net/projects/mingw/files/MinGW%20bzip2/release%201.0.5-2/bzip2-1.0.5-2-mingw32-src.tar.gz/download), cmake (http://www.cmake.org/files/v2.8/cmake-2.8.0.zip) <--- install them in /usr/local
Patch this (http://pastebin.com/m66c52ea7).
cmake -G "MSYS Makefiles"
make
cp include/ffms.h /usr/local/include
cp libFFMS2.a /usr/local/lib


x264
./configure --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib"
make

[LoRd_MuldeR]

--size should work in 2nd pass.

You need to specify a target bitrate also for the first pass if a 2-Pass encode, so you'd need to know the total number of frames before the first pass.

[Selur]

Since 'LAVF/FFMS input support, native VFR timestamp handling' made it into the git repository I wanted to ask:
Does this feature provide any benefit over piping to x264 with ffmpeg/mencoder? (aside from the point of not needing an external decoder to pipe to x264)

[Dark Shikari]

Since 'LAVF/FFMS input support, native VFR timestamp handling' made it into the git repository I wanted to ask:
Does this feature provide any benefit over piping to x264 with ffmpeg/mencoder? (aside from the point of not needing an external decoder to pipe to x264)Piping loses timestamps.

[Selur]

yes, but does x264 profit from the timestamps? iirc (not too sure) decoding a vfr source with ffmpeg/mencoder should result in a cfr output with duplicated frames and since the frames should be exact duplicates it shouldn't harm compression so it seems like it might be the question is x264 can detect duplicate frames and 'create vfr output' (not code duplicates and adjust the timestamps accordingly).

Cu Selur

[Dark Shikari]

yes, but does x264 profit from the timestamps? iirc (not too sure) decoding a vfr source with ffmpeg/mencoder should result in a cfr output with duplicated frames and since the frames should be exact duplicates it shouldn't harm compression so it seems like it might be the question is x264 can detect duplicate frames and 'create vfr output' (not code duplicates and adjust the timestamps accordingly).

Cu Selurffmpeg does not duplicate frames unless you use -r to tell it to.

[Selur]

Assuming I use -r FRAMERATE for the output (which I normally do when piping to x264), will it make a difference for the output if x264 is fed via ffmpeg&pipe or directly with the file?

[Dark Shikari]

Assuming I use -r FRAMERATE for the output (which I normally do when piping to x264), will it make a difference for the output if x264 is fed via ffmpeg&pipe or directly with the file?No. Using -r forces CFR by duplicating or dropping frames.

Do note that using --fps in x264 does not force CFR at the moment when dealing with VFR input: we will change that in a future patch.

Also note that x264 WILL NEVER DROP FRAMES currently: it has no internal sync system, so you can't convert VFR->CFR. "Forcing CFR" just changes the timestamps on existing frames, potentially losing sync.

[Selur]

Thanks for clearing that up. :)

[roozhou]

Hi DS,

- pic.i_pts = (int64_t)i_frame * param->i_fps_den;
+ if( !param->b_vfr_input )
+ pic.i_pts = i_frame;


Why? MP4, flv and mkv muxer can assume all streams vfr (timestamp based) and easily handle them. But now they have to treat cfr and vfr differently.

[Dark Shikari]

Hi DS,

- pic.i_pts = (int64_t)i_frame * param->i_fps_den;
+ if( !param->b_vfr_input )
+ pic.i_pts = i_frame;


Why? MP4, flv and mkv muxer can assume all streams vfr (timestamp based) and easily handle them. But now they have to treat cfr and vfr differently.Huh? That code you quoted is synthesizing timestamps for the muxers, AFAIK.

[kemuri-_9]

Huh? That code you quoted is synthesizing timestamps for the muxers, AFAIK.

yes, that synthesizes timestamps for CFR input (which is exactly how it reads :P)

[roozhou]

Really confused on this. What is the real PTS for each frame (in seconds)?

A) i_pts * i_fps_den / i_fps_num
B) i_pts * i_timebase_num / i_timebase_den

[Dark Shikari]

Really confused on this. What is the real PTS for each frame (in seconds)?

A) i_pts * i_fps_den / i_fps_num
B) i_pts * i_timebase_num / i_timebase_denThe timebase is fps_den / fps_num.

This means that if the pts of frames are:

0, 1, 2, 3, 4, 5...

And the fps is:

24000/1001

The frames' times, in seconds, are:

0/24000, 1001/24000, 2002/24000, 3003/24000, 4004/24000, 5005/24000...

[roozhou]

The timebase is fps_den / fps_num.

This means that if the pts of frames are:

0, 1, 2, 3, 4, 5...

And the fps is:

24000/1001

The frames' times, in seconds, are:

0/24000, 1001/24000, 2002/24000, 3003/24000, 4004/24000, 5005/24000...
CFR input module should let i_timebase_num = i_fps_num and i_timebase_den = i_fps_den, and leave i_pts unchanged

VFR input module should let i_timebase_num = 1 and i_timebase_den = i_fps_den, and let each i_pts = real_timestamp * i_timebase_den

Right?

[Dark Shikari]

CFR input module should let i_timebase_num = i_fps_num and i_timebase_den = i_fps_den, and leave i_pts unchanged

VFR input module should let i_timebase_num = 1 and i_timebase_den = i_fps_den, and let each i_pts = real_timestamp * i_timebase_den

Right?Wrong.

CFR, by definition, must synthesize timestamps. There are no PTS to "remain unchanged". It makes up the PTS in x264.c.

VFR input module does not change the PTS or the timebase (unless the PTS are outright invalid, of course).

[roozhou]

Wrong.

CFR, by definition, must synthesize timestamps. There are no PTS to "remain unchanged". It makes up the PTS in x264.c.

VFR input module does not change the PTS or the timebase (unless the PTS are outright invalid, of course).

I wonder how an input module should set i_pts in x264_picture_t. Of course I know VFR input should leave PTS as is because my x264 build started to support VFR since Oct 2008.

[JoeMiller]

currently, I compile x264 into mencoder. mencoder eats almost any input, no matter which codec or container. Does the x264 LAVF/FFMS version offer any advantage over my mencoder+x264 combination?

[Dark Shikari]

currently, I compile x264 into mencoder. mencoder eats almost any input, no matter which codec or container. Does the x264 LAVF/FFMS version offer any advantage over my mencoder+x264 combination?Mencoder doesn't support VFR.

[roozhou]

Mencoder doesn't support VFR.

Mencoder does support VFR input and output, but its lavf muxer is crappy. Remember to set -ofps to maximum possible fps of the input video, otherwise it may unnecessarily drop frames.

[Snowknight26]

ffms [info]: 640x480p 0:1 @ 30/1 fps (vfr)
Kind of weird that FFMS thinks Fraps videos are VFR. Oh well, glad you can force CFR.

[kemuri-_9]

Kind of weird that FFMS thinks Fraps videos are VFR. Oh well, glad you can force CFR.

http://doom10.org/index.php?topic=133.msg1151#msg1151

[Snowknight26]

Having ffmpeg pipe to x264:
http://i47.tinypic.com/25ewvb8.jpg
ffmpeg -i "%%A" -vcodec rawvideo -f yuv4mpegpipe -an -pix_fmt yuv420p - | x264.x64 - --stdin y4m --crf 19 --bframes 5 --b-adapt 2 --ref 4 --mixed-refs --no-fast-pskip --direct auto --deblock -3:-3 --subme 10 --trellis 2 --analyse all --8x8dct --me umh --output "%%~nA.noaudio.mkv"

Having x264 open a file directly:
http://i45.tinypic.com/15p1h1e.jpg
x264.x64 --crf 19 --bframes 5 --b-adapt 2 --ref 4 --mixed-refs --no-fast-pskip --direct auto --deblock -3:-3 --subme 10 --trellis 2 --analyse all --8x8dct --me umh --output "%%~nA.noaudio.mkv" "%%A"

Standard Fraps video (force lossless RGB unchecked). Haven't checked further but it might be an issue with the colorspace conversion (BGR24 -> YV12) that FFMS is doing. Check the window edges/reds if you can't see it.

[CruNcher]

yeah the chroma looks totally dead :(

[Dark Shikari]

Standard Fraps video (force lossless RGB unchecked). Haven't checked further but it might be an issue with the colorspace conversion (BGR24 -> YV12) that FFMS is doing. Check the window edges/reds if you can't see it.I can't see any difference at all. If there is one, it's because swscale uses different algorithms from Avisynth. I trust swscale more.

[Mr VacBob]

I can see it. But I'd recommend turning off subpixel rendering for a screen recording anyway…

[Snowknight26]

I can't see any difference at all.

http://i46.tinypic.com/1zb71cl.jpg

If there is one, it's because swscale uses different algorithms from Avisynth. I trust swscale more.

The first screenshot is identical to how the Fraps decoder decodes the video.. unlike the 2nd screenshot. Granted, the colors are more accurate in the 2nd, but the chroma is messed up as CruNcher noted.

I can see it. But I'd recommend turning off subpixel rendering for a screen recording anyway…

How would I go about achieving that?

[me7]

I just started encoding a 24/1.001 fps BluRay rip where Avisynth apparently reports a wrong framerate:
avs [info]: 1280x692p 0:0 @ 250000/10427 fps (cfr)
Input file is AVC in mkv container, read via FFVideoSource.

Should I be worried? Will the encoded file have a wrong framerate in it's metadata? Will x264 alter the quality-per-frame in order to get a desired quality-per-second?

[kemuri-_9]

Should I be worried? Will the encoded file have a wrong framerate in it's metadata? Will x264 alter the quality-per-frame in order to get a desired quality-per-second?

A) yes, the encoded file will have the wrong framerate.
B) if using bitrate mode, this will have an extremely slight difference as 250000/10427 is extremely slightly different from 24000/1001.

and this issue has actually already been addressed and fixed in ffmpegsource so update to a newer version.

[me7]

and this issue has actually already been addressed and fixed in ffmpegsource so update to a newer version.

I tried the latest version available here (http://code.google.com/p/ffmpegsource/downloads/list), but it doesn't fix the problem.

[rack04]

I tried the latest version available here (http://code.google.com/p/ffmpegsource/downloads/list), but it doesn't fix the problem.

The latest version is r263.

[me7]

The latest version is r263.

Where can I get it?

[LoRd_MuldeR]

Where can I get it?

http://code.google.com/p/ffmpegsource/source/checkout

[me7]

http://code.google.com/p/ffmpegsource/source/checkout

Now I really regret not knowing anything about C++ :mad:
Still, thanks for the info.

[LoRd_MuldeR]

Now I really regret not knowing anything about C++ :mad:
Still, thanks for the info.

Actually most of the time you can compile stuff without having to understand the code. You don't even need to look at the code.

Once you have set up the build environment, you only need to know how to use SVN and Make :)

[Darkwhisperer]

If your'll don't mind my asking, how are the tests going? Is there a planned release date perhaps?

:thanks:

[J_Darnley]

Why does something that was committed need updating?

[burfadel]

The all-in-one x264 is a great idea, however it does rely on a third party decoder that is under continual development. Since the third party tool is under continual development x264 will need to have that component continually updated to include the features added/fixed in the third party provision. The solution would be to simply update those components each time x264 components are updated, in order to stay current.

[smok3]

do we have osx and/or ubuntu-karmic builds?

[Audionut]

do we have osx and/or ubuntu-karmic builds?
http://doom10.org/index.php?topic=3.0

[hajj_3]

Does this mean we could have a 64bit of MeGUI and use a 64bit of x264.exe with LAVF/FFMS integrated into it? Do we know what kind of speed improvements we would get using this x264 beta compared to using regular x264.exe and avisynth 2.5.8 or 2.6.0 MT alpha?

[Blue_MiSfit]

I'm having great results with these new features! I've been doing lots of high speed transcodes of my movie / TV backups to baseline profile H.264 (for the iPhone).

Being able to directly open files in x264 without help from AviSynth, all in "pure" 64 bit is extremely convenient, and is pretty durn cool too :)

~MiSfit

[rack04]

I'm trying to mux a 264 stream generated by x264 using LAVF as the input method. The source file is mov. When I try to mux to mkv using eac3to it complains that the stream does not contain fps information. Is this intentional or a bug? Using avs input the file muxes without error. I am using this (http://forum.doom9.org/showthread.php?p=1374688#post1374688) build.

Here is a sample (http://www.multiupload.com/ME5ZMJJ7N8), x264 stats, and vui_parameters:

ffms [info]: 848x480p 0:1 @ 24000/1001 fps (vfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.1
x264 [info]: frame I:51 Avg QP:18.35 size: 41968
x264 [info]: frame P:1235 Avg QP:23.91 size: 8045
x264 [info]: frame B:2309 Avg QP:28.10 size: 2366
x264 [info]: consecutive B-frames: 6.5% 18.9% 6.3% 63.8% 2.0% 2.5%
x264 [info]: mb I I16..4: 31.8% 18.9% 49.2%
x264 [info]: mb P I16..4: 8.4% 8.3% 3.9% P16..4: 22.0% 8.9% 4.6% 0.0% 0.0% skip:43.9%
x264 [info]: mb B I16..4: 0.8% 1.5% 1.0% B16..8: 20.0% 1.4% 1.6% direct: 1.6% skip:72.1% L0:39.1% L1:47.7% BI:13.2%
x264 [info]: 8x8 transform intra:38.7% inter:59.2%
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: coded y,uvDC,uvAC intra: 48.0% 59.4% 29.1% inter: 7.3% 10.9% 3.6%
x264 [info]: i16 v,h,dc,p: 44% 27% 11% 18%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 19% 18% 6% 7% 8% 8% 7% 10%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 20% 11% 6% 9% 8% 9% 7% 10%
x264 [info]: Weighted P-Frames: Y:7.9%
x264 [info]: ref P L0: 72.3% 9.1% 9.2% 2.9% 2.6% 1.5% 1.0% 0.5% 0.5% 0.4% 0.0% 0.0%
x264 [info]: ref B L0: 90.8% 4.4% 2.1% 0.9% 0.6% 0.5% 0.4% 0.2% 0.1%
x264 [info]: ref B L1: 98.5% 1.5%
x264 [info]: kb/s:935.79

encoded 3595 frames, 10.60 fps, 935.79 kb/s

vui_parameters_present_flag: 1
aspect_ratio_info_present_flag: 0
overscan_info_present_flag: 0
video_signal_info_present_flag: 0
chroma_loc_info_present_flag: 0
timing_info_present_flag: 1
num_units_in_tick: 1
time_scale: 5994
fixed_frame_scale: 0
nal_hrd_parameters_present_flag: 0
vcl_hrd_parameters_present_flag: 0
pic_struct_present_flag: 0
motion_vectors_over_pic_boundaries_flag: 1
max_bytes_per_pic_denom: 0
max_bits_per_mb_denom: 0
log2_max_mv_length_horizontal: 11
log2_max_mv_length_vertical: 11
num_reorder_frames: 2
max_dec_frame_buffering: 10

[Dark Shikari]

In VFR mode x264 does not write fixed_frame_rate_flag for obvious reasons.

You can use --force-cfr to get Avisynth's behavior in LAVF/FFMS.

[lucassp]

Why does it always default to VFR?

[Snowknight26]

http://doom10.org/index.php?topic=133.msg1151#msg1151

5char.

[PowerGamer]

Currently (x264 rev. 1442) FFMS can't handle raw H.264 streams. x264 opens them with LAVF but is not able to determine total amount of frames in the video being encoded. I wonder if that (inability to determine total amount of frames) has any negative effect on quality/speed of the encoding process (because if it has I can always specify total amount of frames with --frames switch)?

[Dark Shikari]

Currently (x264 rev. 1442) FFMS can't handle raw H.264 streams. x264 opens them with LAVF but is not able to determine total amount of frames in the video being encoded. I wonder if that (inability to determine total amount of frames) has any negative effect on quality/speed of the encoding process (because if it has I can always specify total amount of frames with --frames switch)?No negative effect currently. In the future this may in theory change when using 1-pass ABR mode.

[bnshrdr]

Hey where can I get a completely up-to-date version of ffmpeg so that I can compile x264 with lavf support?
I would have thought git clone git://git.ffmpeg.org/ffmpeg/ would be as up-to-date as it is needed to be but keep getting this warning from x264's ./configure:
Warning: libavformat is too old, update to ffmpeg r18351+

[Dark Shikari]

Hey where can I get a completely up-to-date version of ffmpeg so that I can compile x264 with lavf support?
I would have thought git clone git://git.ffmpeg.org/ffmpeg/ would be as up-to-date as it is needed to be but keep getting this warning from x264's ./configure:
Warning: libavformat is too old, update to ffmpeg r18351+You likely have an older ffmpeg installed that x264 is trying to link to.

[bnshrdr]

Well I actually opened up the x264 ./configure file and saw that it was checking for the function avcodec_decode_video2 in one of the libavformat header files. I went and checked, and what do you know, x264 is right. Is there a chance that the master ffmpeg revision isn't truly the master revision.

[Mr VacBob]

ffmpeg git is up to date (r21977). You probably have an older version installed somewhere else.

[bnshrdr]

As much as I understand your logic, I am using a clean system and have not installed anything.

On line 529 of x264's configure script, it checks for most likely a critical function:
if cc_check libavformat/avformat.h "$LAVF_CFLAGS $LAVF_LIBS" "avcodec_decode_video2( NULL, NULL, NULL, NULL );" ; then
lavf_input="yes"
define LAVF_INPUT
else
echo "Warning: libavformat is too old, update to ffmpeg r18351+"

I took the liberty of opening libavformat/avformat.h myself to search for this function prototype and it cannot be found. In fact I scanned the whole source code tree to find out where it actually appears, and found it to be in the header file libavcodec/avcodec.h.

I'll try editing the configure script to check that file instead of the other header file and see what happens.

Edit: I seemed to have tricked it successfully. I compiled x264 with lavf support and tested it and it works. Might wanna take a look at that configure script.

Needs a diff that basically does this:
-if cc_check libavformat/avformat.h "$LAVF_CFLAGS $LAVF_LIBS" "avcodec_decode_video2( NULL, NULL, NULL, NULL );" ; then
+if cc_check libavcodec/avcodec.h "$LAVF_CFLAGS $LAVF_LIBS" "avcodec_decode_video2( NULL, NULL, NULL, NULL );" ; then

[kemuri-_9]

Needs a diff that basically does this:
-if cc_check libavformat/avformat.h "$LAVF_CFLAGS $LAVF_LIBS" "avcodec_decode_video2( NULL, NULL, NULL, NULL );" ; then
+if cc_check libavcodec/avcodec.h "$LAVF_CFLAGS $LAVF_LIBS" "avcodec_decode_video2( NULL, NULL, NULL, NULL );" ; then

A) libavformat/avformat.h includes "libavcodec/avcodec.h"
B) we're specifically using both libavformat and libavcodec
C) you removed the check to see if the critically required libavformat/avformat.h header is actually available/correct

so your change is incorrect and can cause issues. fix your installation rather than hack the configure.

[smok3]

http://doom10.org/index.php?topic=3.0

http://doom10.org/index.php?topic=220.0

i have to do everything myself.:devil:

[turbojet]

From about 30 different tries so far 24000/1001 m2ts input is detected at 23998/10001 and 24005/1001 both vfr but never has one been detected correctly at 24000/1001 cfr.

30000/1001 interlaced is detected as 60020/1001 vfr.

[Atak_Snajpera]

30000/1001 interlaced is detected as 60020/1001 vfr.
I had similar problem with FFmpegSource.
http://forum.doom9.org/showthread.php?p=1363469#post1363469
http://forum.doom9.org/showthread.php?p=1364213#post1364213

Nobody cares :/

[bnshrdr]

Hey I am not sure if this has been brought up yet, but recently I had an input file set to be demuxed with ffms2, and I threw the --frames switch to x264, but it still seemed to index the entire file, which depending on how long the video is it can be pretty lengthy if I only wanted 250 frames. Would there be a way that x264 could tell ffms2 to stop indexing at X frame?

I mean, yes, I could change the demuxer to lavf, but ffms2 has to stay competitive.

[kemuri-_9]

Hey I am not sure if this has been brought up yet, but recently I had an input file set to be demuxed with ffms2, and I threw the --frames switch to x264, but it still seemed to index the entire file, which depending on how long the video is it can be pretty lengthy if I only wanted 250 frames. Would there be a way that x264 could tell ffms2 to stop indexing at X frame?

I mean, yes, I could change the demuxer to lavf, but ffms2 has to stay competitive.

this is not currently possible on x264's side of things as all the API calls that FFMS2 has are to index the entire video file.
there are no 'stop indexing at frame X' options.
the only thing to alleviate this is the indexing file, which allows reading from the index on subsequent executions rather than re-indexing every time.
this is an option x264 has made available but is not on by default.

the ffms2 codebase is likely to be designed on the fact the entire video is indexed, so changing this would be (again, likely) be time consuming with a seemingly small return.

[roozhou]

Hey I am not sure if this has been brought up yet, but recently I had an input file set to be demuxed with ffms2, and I threw the --frames switch to x264, but it still seemed to index the entire file, which depending on how long the video is it can be pretty lengthy if I only wanted 250 frames. Would there be a way that x264 could tell ffms2 to stop indexing at X frame?

I mean, yes, I could change the demuxer to lavf, but ffms2 has to stay competitive.

I hate ffms's indexing too. It takes a long time for large files. A couple of questions about indexing:
1) Why on earth do we have to index a file that already has an index in the container(e.g. avi, mkv, mp4)?
2) If we only need linear access(encode from beginning to end), why can't we skip the indexing phase? Index can be generated during decoding.

I suggest the inclusion of on-demand indexing. That is only the needed part should be indexed. e.g. I need to encode from 1'0" to 1'30", so indexing should be done from beginning to the GOP containing the frame at 1'30".

FFMS2 was firstly designed to use with avisynth only and perhaps these features are not compatible with avisynth. I know that developers would say "we don't want to break API", but the current implementation really sucks.

[Myrsloik]

I hate ffms's indexing too. It takes a long time for large files. A couple of questions about indexing:
1) Why on earth do we have to index a file that already has an index in the container(e.g. avi, mkv, mp4)?
2) If we only need linear access(encode from beginning to end), why can't we skip the indexing phase? Index can be generated during decoding.


1. Because no one fixed ffmpeg so you can just seek to the nth real frame. That's why.

2. FFMS2 is mostly pointless to use in that scenario, but it's been on the todo list for ages and will probably happen some day.

[roozhou]

1. Because no one fixed ffmpeg so you can just seek to the nth real frame. That's why.
This is unrelated. My question is why not use the index in the container. It contains all the information you need and is much faster to read.

And when you need to seek to the nth real frame, just seek to the nearest key frame and start decoding. When the pts or dts matches your desired timestamp, you get it.

[Myrsloik]

I (and the rest of the world too, apparently) don't have the time or motivation to rewrite that many demuxers, especially not to a compatibility level near that of lavf. That's why.

[Dark Shikari]

I (and the rest of the world too, apparently) don't have the time or motivation to rewrite that many demuxers, especially not to a compatibility level near that of lavf. That's why.AFAIK lavf has an interface that allows you to access the index of files that have it (though I don't know how good it is).

Also, get on IRC, we have a crash report for you from a GSOC student.

[Mr VacBob]

I think lavf's index_entries is private. I know it contains confusingly incorrect data for some AVI files that nevertheless work with av_read_frame(). There is a SOC project for better seeking API, and I need an index reading API myself but haven't brought it up yet.

[TheFluff]

I had similar problem with FFmpegSource.
http://forum.doom9.org/showthread.php?p=1363469#post1363469
http://forum.doom9.org/showthread.php?p=1364213#post1364213

Nobody cares :/

Both myrsloik and I are absurdly busy at the moment (him even more than me though), submit a patch or harass someone else until they do.
M2TS support is still generally lolbroken though and I think the real solution is to either fix lavf's parser or write a new ts parsing library from scratch and get ffms2 to use it.

[LoRd_MuldeR]

Both myrsloik and I are absurdly busy at the moment (him even more than me though), submit a patch or harass someone else until they do.
M2TS support is still generally lolbroken though and I think the real solution is to either fix lavf's parser or write a new ts parsing library from scratch and get ffms2 to use it.

...I though FFMS2 uses Haali's TS parser, if available? :confused:

[Atak_Snajpera]

me too

[TheFluff]

...I though FFMS2 uses Haali's TS parser, if available? :confused:

It does, but even that doesn't always work, especially not for m2ts (it also doesn't work on non-Windows platforms because it's called as a COM object). There are tons of funny issues with m2ts, judging by how many people are complaining about it.

[Atak_Snajpera]

Can you just force FFmpegSource to divide frame number by 2 for m2ts flaged as interlaced? ffmpegsource should report number of real frames not number of fields. for examples 24p NTSC AVCHD footage is always stored as 29.97i (telecine 3:2) and number of frames is absolutely wrong.

[TheFluff]

What does "flagged as interlaced" mean? PAFF, I assume (what about MBAFF)? How do you tell the difference with lavc anyway, scan the entire stream and check for avframe->interlaced_frame? I guess I could do that during indexing, but I'd rather find out why it's doing it at all.

[Mr VacBob]

You can't tell PAFF from decoded AVFrames, since ffmpeg always returns whole frames (= two fields). The only way they're exposed in the API is that avcodec_decode_video doesn't return a frame when you pass it the first field's packet.

AVParser could be extended to do this, or you could do handwritten parsing since only MPEG2 and H.264 are weird enough to have separate field pictures.

[kemuri-_9]

post a sample.

[Blue_MiSfit]

@Dogway:

Please do not bring old threads back to life. Start a new thread for issues like this.

Thanks,
Derek

[Dogway]

Sorry, I was unsure what to do really.

[dansus]

Im struggling to work out the parms required to force SAR 1:1 in --Video-Filter;

--video-filter crop:2,2,2,2/resize:1280,720,method=lanczos/? (source 4:3 anamorphic, want 16:9 1:1 ouput)

Edit: Coffee's kicked in, worked it out;

--video-filter crop:2,2,2,2/resize:1280,720,1:1,method=lanczos


Still struggling with correct fps output, source is 29.970 after deinterlace and its spitting out 59.940.

ffms [info]: 1440x1080i 0:1 @ 60030/1001 fps (vfr)
.

[sneaker_ger]

--no-interlaced --fps 30000/1001

[sneaker_ger]

btw.:
Why isn't "--no-interlaced" documented in fullhelp?
And why is everything from "--no-i" to "--no-interlaced" accepted?

[LoRd_MuldeR]

And why is everything from "--no-i" to "--no-interlaced" accepted?

getopt (http://www.gnu.org/s/libc/manual/html_node/Getopt.html) ;)

[sneaker_ger]

I see.

[dansus]

--no-interlaced --fps 30000/1001

That seems to have worked, except playback is in slow motion now.. :p

[sneaker_ger]

Then your source is not 29.970.

[dansus]

Then your source is not 29.970.

Media Info says;

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Format settings, GOP : M=1, N=56
Codec ID : 27
Duration : 1mn 11s
Bit rate mode : Variable
Bit rate : 8 593 Kbps
Maximum bit rate : 20.0 Mbps
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First

[sneaker_ger]

Did you perhaps use bobbing (e.g. yadif mode 1) or mistyped "30000/1001"?
Upload a sample otherwise.

[dansus]

Did you perhaps use bobbing (e.g. yadif mode 1) or mistyped "30000/1001"?
Upload a sample otherwise.

Seems ok, using ffms in x264.exe

x264.exe --preset slow --crf 21.5 --level 4.1 --video-filter crop:2,2,2,2/resize:1280,720,1:1,method=lanczos --no-interlaced --fps 30000/1001 --output "C:\Output.mkv" --demuxer ffms --index "C:\x264-index" "C:\testhd.ts"

Sample - http://www.mediafire.com/?5j9x0az19sx00eq

[dansus]

Doesnt like PAL MBAFF either (BBCHD)


ffms [info]: 1440x1080i 4:3 @ 27679/1001 fps (vfr) ??

Theres some weird effect playback too.

http://www.mediafire.com/imgbnc.php/e2879fbe2596144c80f54cec75eb04b8929a7f61334b2b312e76dddd0fb0eedf6g.jpg

[sneaker_ger]

Yes, I also had some troubles on your sample.
This command line seems to result in acceptable output:
x264 testhd.ts -o output.mkv --vf yadif:0,bff/select_every:2,0 --fps 60000/1001

I didn't manage to bob and I don't know what I'm doing wrong.

[dansus]

Yes, I also had some troubles on your sample.
This command line seems to result in acceptable output:
x264 testhd.ts -o output.mkv --vf yadif:0,bff/select_every:2,0 --fps 60000/1001

I didn't manage to bob and I don't know what I'm doing wrong.

So your saying you couldnt get the right output with ffms?
.

[sneaker_ger]

Test my command line, it looks fine, but I did not manage to bob.

[dansus]

Test my command line, it looks fine, but I did not manage to bob.

Cant seem to get that code to run, crashes after index. Im sure its just me being a noob.. :)

[sneaker_ger]

I'm using this build (http://www.cs.helsinki.fi/u/jeekstro/x264/1913/x264_1913_64.7z).

[dansus]

I'm using this build (http://www.cs.helsinki.fi/u/jeekstro/x264/1913/x264_1913_64.7z).

Is that a x64 build? Yep, rename link to 1913_32 if you need x86 version.

[dansus]

This command line seems to result in acceptable output:
x264 testhd.ts -o output.mkv --vf yadif:0,bff/select_every:2,0 --fps 60000/1001


That did the trick. Thanks.

What would be the best parms for PAL i50 content, --vf yadif:0,tff --fps 25/1 ?

Wasnt aware Yadif had been built in to x264, need to look into this more, see what other goodies lie within.

http://www.mediafire.com/?kehnkpejkcmluk9

[sneaker_ger]

Good question, theoretically "--vf yadif --fps 25" should work, but since it screws up on your sample it either means the capture is corrupted or ffms is too buggy to handle it. Search around the forum for BBC and you might find some tips as you are not the only one that has problem with it.