Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd January 2010, 00:25   #1  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
x264 LAVF/FFMS input public beta test (v2.1)

Now that we've had a small public test for version 1, we need as much testing as possible done on version 2 of the LAVF/FFMS input patch. In short, this input patch lets you not only take input from nearly any file without using Avisynth, but also allows x264 to natively handle variable framerate video without timecode files.

What we need you to do: test with any input you can find, especially inputs that might be hard or otherwise suspect to breaking. Report any bugs you find, but make sure to check the known bugs list below so you don't report things that we already know about.

Download v2.1
Patch v2.1

Download v1
Patch v1

New features (v2.1):

1. Input from anything, even without Avisynth or DirectShow codecs, even on Linux!
2. "True VFR": x264 maintains timestamps from the input, allowing native processing of VFR video. No more timecode files!
3. Use --demuxer to force a particular input method (lavf, ffms, etc).
4. Automatic handling of all kinds of weird input (changing resolution, etc), including many inputs that Avisynth wouldn't have worked correctly with.
5. A near-complete rewrite of the x264cli internals to handle the above.
6. Periodic Intra Refresh.

See the three commit messages in the patch for more details.

Gotchas (v2.1):

1. There seem to be some types of files (raw h264?) that FFMS refuses to index. We'll be looking at that, but LAVF should work fine in the meantime.
2. FFMS won't work on piped input.
3. x264 doesn't by default save the index file from FFMS, so it has to re-index on every pass unless you use --index. We may change the default behavior in the future.

Known bugs (v2.1)
<none>

Now that we're through the gotchas, feel free to test on various types of input and report any issues you have.

Disclaimer: this is a HUGE patch. It modifies dozens of files and likely has a lot of bugs. Problems are normal, and the purpose of this test is to root out as many as possible.

Mega-thanks to Mike Gurlitz (ACoolie), Steven Walters (Kemuri9), and Kieran Kunhya (Kierank) for the massive amount of high-quality work that went into this patch.

Last edited by Dark Shikari; 2nd January 2010 at 01:29.
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 00:40   #2  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 564
windows 2003, 32 bit, 4 gb of memory, avisynth installed 2.5.8 i think... source is 720p 50 fps i think

x264_lavfffms.exe --bitrate 1024 -o e:\test.mkv e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv



vs 2010 debug shows this..



process explorer shows this...



it wouldn't surprise me to have made something wrong, so please let me know if that's the case.. and if there's any way I can help you out in more detail I'll be glad to do it.

later edit: is it supposed to actually read mkv files? i see it doesn't crash with .avi files...

even later edit: on avi files the console window says in the taskbar indexing input but after it finishes that and begins to encode content it still says the same thing.. I think it used to show encoding progress. In the console window itself, it does show the status correctly
(and excellent news and thanks for the cool things you add to x264!)

Last edited by mariush; 2nd January 2010 at 00:46.
mariush is offline   Reply With Quote
Old 2nd January 2010, 00:44   #3  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Confirmed, FFMS seems to be crashing on MKV files. We'll look into it.

If FFMS crashes, try --demuxer lavf and see if that works instead.
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 00:55   #4  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 564
ok, it worked with --demuxer lavf but noticed it wasn't able to show the percent... went counting frames:

Code:
x264_lavfffms.exe --demuxer lavf --bitrate 1024 --output e:\test.mkv "e:\ORF1HD.Demo-Loop.720p.DD5.1.mkv"
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
    Last message repeated 5 times
[matroska @ 003dbda0]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014b65c0]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1280x720p 1:1 @ 50/1 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.2
x264 [warning]: non-monotonic pts at frame 837 (17640 < 18520)
x264 [info]: frame I:9     Avg QP:25.74  size: 32864
x264 [info]: frame P:639   Avg QP:27.54  size:  6404
x264 [info]: frame B:1306  Avg QP:32.33  size:   967
x264 [info]: consecutive B-frames:  4.8% 15.9%  3.4% 75.9%
x264 [info]: mb I  I16..4: 25.7% 63.1% 11.1%
x264 [info]: mb P  I16..4:  4.2%  2.9%  0.1%  P16..4: 38.5%  5.4%  3.6%  0.0%  0
.0%    skip:45.4%
x264 [info]: mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 23.6%  0.1%  0.1%  direct:
 0.1%  skip:75.8%  L0:33.9% L1:65.6% BI: 0.5%
x264 [info]: final ratefactor: 29.93
x264 [info]: 8x8 transform intra:44.4% inter:79.0%
x264 [info]: coded y,uvDC,uvAC intra: 19.0% 32.7% 7.0% inter: 4.0% 7.1% 0.1%
x264 [info]: i16 v,h,dc,p: 44% 27%  9% 21%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 45%  3%  4%  4%  5%  3%  7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 28% 14%  4%  7%  5%  7%  4%  6%
x264 [info]: Weighted P-Frames: Y:20.7%
x264 [info]: ref P L0: 64.8%  8.6% 19.6%  3.6%  3.4%
x264 [info]: ref B L0: 95.3%  4.7%
x264 [info]: kb/s:1156.81

encoded 1954 frames, 28.38 fps, 1099.16 kb/s
File plays fine in MPC-HC with the dxva decoder though at some point it goes blocky for a couple of frames in the middle - maybe it got corrupt or it's just the way the movie was... if i remember correctly i got the sample from ffmpeg's collection of test clips

question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?

Last edited by mariush; 2nd January 2010 at 01:08.
mariush is offline   Reply With Quote
Old 2nd January 2010, 01:30   #5  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Bug found in FFMS that caused the problem. Fix can be found here; build in the topic updated to v2.1 accordingly.
Quote:
Originally Posted by mariush View Post
question: if the video source has a SAR specified (example mkv file with h264 720x432 (185:78) 25.00fps so shows the image at about 1024x432), shouldn't x264 export the sar automatically to the output format, mkv for example, or at least issue a warning or debug message?
Yes, it should, and as far as I can tell it works correctly.
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 02:06   #6  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 564
Now it works with MKV input but the aspect ratio is ignored when using --demuxer lavf command:

Code:
x264_lavfffms2.exe --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
ffms [info]: 720x432p 37:26 @ 25/1 fps (vfr)
x264 [info]: using SAR=37/26
Code:
x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv e:\test_in-001.mkv
[matroska @ 003dbd70]Estimating duration from bitrate, this may be inaccurate
lavf [info]: 720x432p 0:1 @ 25/1 fps (vfr)
If you want, I can upload a small 20s clip but you probably already have some test clips for this.

I've also noticed some differences when using that short 1080p clip with the migrating white ducks or whatever birds they are:

Code:
x264_lavfffms2.exe --demuxer lavf --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
    Last message repeated 29 times
[matroska @ 003dbd90]Estimating duration from bitrate, this may be inaccurate
[h264 @ 014c2360]number of reference frames exceeds max (probably corrupt input)
, discarding one
lavf [info]: 1920x1080p 0:1 @ 24000/1001 fps (vfr)
Code:
x264_lavfffms2.exe --frames 100 --bitrate 1024 -o e:\test.mkv w:\Downloads\killer_sample.mkv
ffms [info]: 1920x1080p 1:1 @ 4000/167 fps (vfr)
x264 [info]: using SAR=1/1
mariush is offline   Reply With Quote
Old 2nd January 2010, 02:24   #7  |  Link
Firebird
Registered User
 
Join Date: Mar 2008
Posts: 61
http://img254.imageshack.us/img254/8404/x264tests.png
Just stops on 318 frame.

Sample
Firebird is offline   Reply With Quote
Old 2nd January 2010, 02:33   #8  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Quote:
Originally Posted by Firebird View Post
Did you update to v2.1? That sample works fine here.
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 03:01   #9  |  Link
Firebird
Registered User
 
Join Date: Mar 2008
Posts: 61
Quote:
Did you update to v2.1? That sample works fine here.
v2.1 fails, even with --demuxer avs
Latest stable build is working. Hm. I will test it more, maybe it's a bug of mine.
Firebird is offline   Reply With Quote
Old 2nd January 2010, 03:35   #10  |  Link
chipzoller
Mr. Woof
 
chipzoller's Avatar
 
Join Date: Jan 2002
Location: USA
Posts: 784
Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?
chipzoller is offline   Reply With Quote
Old 2nd January 2010, 04:52   #11  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Quote:
Originally Posted by chipzoller View Post
Dark Shikari, could you, at some point, update the first post for those of us not familiar with work on this patch so we know exactly what its capabilities are and what file types it opens?
It can open anything ffmpeg can open. Read: everything.
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 05:25   #12  |  Link
prOnorama
Registered User
 
Join Date: Mar 2005
Posts: 249
It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMO
prOnorama is offline   Reply With Quote
Old 2nd January 2010, 05:32   #13  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Quote:
Originally Posted by prOnorama View Post
It worked fine here on a 2 minute progressive VC-1 1080p .m2ts sample

I guess it won't work for interlaced VC-1 though (hard to find a sample)

The name could be a bit more simple, I typed it wrong 3 times before I got it working lol, though logical "x264_lavfffms2" doesn't sound very simple or "sexy" IMO
You can rename it, it's just what I named the executable on the server
Dark Shikari is offline   Reply With Quote
Old 2nd January 2010, 11:04   #14  |  Link
roozhou
Registered User
 
Join Date: Apr 2008
Posts: 1,181
DS, thanks for the awesome New Year's present!
Is vfr ratecontrol implemented in this patch?
roozhou is offline   Reply With Quote
Old 2nd January 2010, 13:06   #15  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
Quote:
Originally Posted by roozhou View Post
Is vfr ratecontrol implemented in this patch?
no, it will come shortly later
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 2nd January 2010, 15:14   #16  |  Link
Episodio1
Registered User
 
Join Date: Feb 2002
Location: Spain
Posts: 197
I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?
Episodio1 is offline   Reply With Quote
Old 2nd January 2010, 15:19   #17  |  Link
avivahl
Registered User
 
Join Date: Dec 2007
Posts: 215
Quote:
Originally Posted by Episodio1 View Post
I haven't used ffdshow of ffmpeg and I've googled "lavs" and "ffms" and it showed results in doom9 and ffmpeg.. but no explanation of what that options mean.

Could anyone explain?
Well, it probably uses this library to wrap around ffmpeg's decoding capabilities. I guess that's about it.
avivahl is offline   Reply With Quote
Old 2nd January 2010, 17:20   #18  |  Link
ACoolie
Registered User
 
Join Date: Mar 2008
Posts: 30
lavf is short for libavformat, which is the library ffmpeg and x264 use to open and decode videos. This demuxer is attempted last because it can open all files ffms can, as well as pipefiles, but ffms typically does it better. It doesn't actually read the entire stream before encoding has begun, so frame count is unknown (no percent progress), and fps is taken from the container header, which could be wrong for VFR video. To explicitly use this demuxer, use `x264 --demuxer lavf`, although you should only use it for testing or if ffms2 does not work. Also, --seek will be slow because frames must be read sequentially.

ffms2 is Fabulous FM Source 2 (or FFmpegSource2), a wrapper of lavf that indexes files to provide a better API than libavformat has. It will read the entire input file, which can take considerable time for large files, so you should pass `--index indexfile` to save this index across passes. ffms has fast seeking, and should theoretically give the correct average fps for vfr video (we're still working on this). We've noticed a lot of the bugged input files are caused by this library (which is good, since we can fix them), so the lavf demuxer sometimes must be used on certain files and filetypes. You can force this demuxer to be used with `--demuxer ffms`, but typically it will be the default demuxer if it can be used.
ACoolie is offline   Reply With Quote
Old 2nd January 2010, 17:33   #19  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,367
Quote:
Originally Posted by prOnorama View Post
.........
I guess it won't work for interlaced VC-1 though (hard to find a sample)
Confirmed. Didn't work on interlaced VC-1 here. Encoder runs but produces only a flat gray at playback.
AVC interlaced seems to be ok.
Sharc is offline   Reply With Quote
Old 2nd January 2010, 18:05   #20  |  Link
roozhou
Registered User
 
Join Date: Apr 2008
Posts: 1,181
@kemuri-_9

How did you calculate dts in mp4 output for vfr video when b-frames are used? It seems initdelay is still used.
roozhou is offline   Reply With Quote
Reply

Tags
beta, ffmpegsource, libavformat, test, x264

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.