FFmpegSource - Page 64

TheFluff · 25th August 2011, 23:00

New test build: ffms2-r532.7z
Edit: now with 100% more pthreads thanks to Myrsloik.

We're slowly working on trying to fix enough bugs to motivate a release of 2.16, so test everything, submit all your weirdest sample files and win a prize if it causes a reproducible crash. Test audio in particular since we've been messing around with that a lot lately.

TheRyuu · 26th August 2011, 09:25

Yet another new test build: ffms2-r533.7z

All of the above plus this should fix the invalid postprocess bug (for good). Go forth and test.

TheFluff · 26th August 2011, 21:48

FFMS 2.16 has been released. It is mostly a bugfix release, but it adds a few features as well; most importantly full support for YUV 4:4:4 as well as high bitdepth YUV.

The following downloads are now available:

ffms-2.16.7z - the usual binary. Includes multithreaded decoding functionality; you do no longer need a special -mt version.
ffms-2.16-x64.7z - 64-bit version, for use with avs64. Note that it is a native Avisynth plugin now; no need for LoadCPlugin or LoadStdcallPlugin.
ffms-2.16-src.tar.bz2 - source code.
ffms-2.16_SDK.7z - package for Windows developers who want to develop programs that use FFMS2 but don't want to deal with MinGW.

Full changelog since 2.15:

Reimplemented output colorspace selection, this should fix all issues with the avisynth plugin when opening yuv420p10 or yuv444 material plus several other less common cases. (Myrsloik)
Added FFMS_SetOutputFormatV2 to the API. This function allows you to specify PixelFormats >= 64 for use as output. (Myrsloik)
Fixed a serious bug that could cause crashes and completely useless index files with h264 in matroska. (Myrsloik)
Automatically detect number of decoding threads. The avisynth video source funtion already did this, now moved so the api can use it as well. (TheRyuu)
Re-add the ability to target x64 with msvc since it's a bit more sane now. (TheRyuu)
Fixed a bug that could cause crashes when reading audio if FFMS2 was compiled with GCC. (Myrsloik)
ffmsindex will no longer crash if it cannot open the file given to it for indexing. (TheFluff)
FFMS2 will no longer crash if the video decoder feeds it an empty frame (can sometimes happen when using lots of decoder threads); you'll get a nice error message instead. (TheFluff)
The C-plugin can now act as both an Avisynth 2.6 plugin (including support for new colorspaces) as well as an Avisynth 2.5 one, in the same binary. (kemuri_-9)
Fixed an issue that could cause opening Vorbis audio to fail because FFMS2 couldn't find an initial valid PTS. (TheFluff)
FFMS2 will no longer crash if forced or tricked into using an index file generated by a FFMS2 version compiled for a different architecture. (TheRyuu)
Fixed a crash when the last frame was requested using the Avisynth plugin's forced CFR mode. (Plorkyeran)
Fixed various issues with decoding audio from the Ogg container without Haali's splitter. (Myrsloik, TheFluff)
Fixed the "invalid postprocessing settings"; they were caused by a parsing bug in libpostproc, and a workaround has been added. (Myrsloik)
Tinkered a bit with the non-MSVS build system. (Daemon404, Kovensky)

Important notice for postprocessing users:
Support for postprocessing in FFMS2 will be dropped in the next release. The reason is that both libav is dropping the libpostproc library from their own releases, and so we cannot continue supporting it.

TheRyuu · 30th August 2011, 16:07

Quote:

Originally Posted by rean

Could you also upload an x64 version? I use it to mix video clips, but a 32-bit version of Avisynth cannot be used - no memory...

I'll probably get to it later today or tomorrow.

chipzoller · 31st August 2011, 17:53

I'm not sure if this thread also doubles as the "official" support thread and so hate to cross-post, but I'm having an issue with 2.16 throwing "decoder returned an empty frame". Original thread is here and will gladly merge/delete if desired: http://forum.doom9.org/showthread.php?t=162391

TheFluff · 31st August 2011, 23:39

Quote:

Originally Posted by chipzoller

I'm not sure if this thread also doubles as the "official" support thread and so hate to cross-post, but I'm having an issue with 2.16 throwing "decoder returned an empty frame". Original thread is here and will gladly merge/delete if desired: http://forum.doom9.org/showthread.php?t=162391

This thread isn't the officially designated anything, really, but if you want to be sure your questions are seen by one of the devs this is the place to post (most of us do prowl the rest of the forums from time to time as well, though). You can chill here as much as you want and ask about whatever you like, as long as you take off your shoes before entering and don't act like a dick.

As for your problem: as I posted in the other thread, it turns out libavcodec doesn't support interlaced VC-1 at all (yet), so unfortunately we cannot help you. You'll have to turn to some other source plugin.

qyot27 · 1st September 2011, 00:32

When cross-compiling the C plugin, the recent version.sh-related changes result in the .dll being built as ffms.dll instead of ffms2.dll. On Windows this isn't an issue, because of MSys' bash quirks or file permissions for NTFS always being executable or something.

This is the problematic section:

Code:

version=`version.sh`
API=$(echo $version | cut -f 1 -d '.')

On Linux, this results in version.sh not being found when configure gets called (so the version info is left blank and the '2' isn't inserted into the filename of the .dll), and even when having used chmod +x on version.sh, it seems to still not want to add it as an environment variable.

I managed to fix it (and have it work on both Windows and Linux), by commenting out the version ENV line and adjusting the API variable to use version.sh's filename directly:

Code:

#version=`version.sh`
API=$(echo $(./version.sh) | cut -f 1 -d '.')

kemuri-_9 · 1st September 2011, 10:40

oops -> fixed

henryho_hk · 1st September 2011, 12:15

why not simply

Code:

API=$(./version.sh | /bin/cut -f 1 -d '.')

Chumbo · 5th September 2011, 20:33

Quote:

Originally Posted by TheRyuu

I'll probably get to it later today or tomorrow.

Thank you.

TheRyuu · 5th September 2011, 21:22

ffms-2.16-x64.7z - 64-bit version, for use with avs64. Note that it is a native Avisynth plugin now; no need for LoadCPlugin or LoadStdcallPlugin
ffms-2.16_SDK.7z - updated to now include the 64-bit stuff.

Little late.
Have at it.

Chumbo · 5th September 2011, 23:55

Quote:

Originally Posted by TheRyuu

ffms-2.16-x64.7z - 64-bit version, for use with avs64. Note that it is a native Avisynth plugin now; no need for LoadCPlugin or LoadStdcallPlugin
ffms-2.16_SDK.7z - updated to now include the 64-bit stuff.

Have at it.

Very much appreciated.

Chumbo · 9th September 2011, 21:17

Quote:

Originally Posted by rean

It seams, a 64-bit version creates incorrect indexes. I get audio position error messages or program response lost (most cases) using VirtualDub. My videos are x264-10bit encoded .mkv with a pcm audio ~ 15-30 minutes and joined using v1++v2++v3++v4... I cannot change a view position in all cases. Deleting of index files do not resolve the situation - seems indexes are not valid.

A x86 version works as charm. Also in some cases a 64 bit version uses a 32-bit-generated index files. Also switching an edition produces every reindex. It is very bad, because eats a time. Please do not overwrite existing indexes, if they are exist and creates by a wrong edition. Create a new one with a different extension or save a new portion in the same file. Or (the best solution) - use the same file format for x64 and x86 edition.

So, I returned to x86 with temp lossless files

It shouldn't overwrite an existing index file unless you're using the -f switch.

TheRyuu · 9th September 2011, 21:18

Quote:

Originally Posted by rean

Also in some cases a 64 bit version uses a 32-bit-generated index files.

Are you sure about this? It should never happen (it will detect the incompatibility and reindex).

Quote:

Originally Posted by rean

use the same file format for x64 and x86 edition.

Indeed. It's been suggested before but no real work has been done on it yet.

Quote:

Originally Posted by Chumbo

It shouldn't overwrite an existing index file unless you're using the -f switch.

If you directly use ffvideosource it will overwrite without asking.

TheFluff · 9th September 2011, 21:47

Quote:

Originally Posted by rean

It seams, a 64-bit version creates incorrect indexes. I get audio position error messages or program response lost (most cases) using VirtualDub. My videos are x264-10bit encoded .mkv with a pcm audio ~ 15-30 minutes and joined using v1++v2++v3++v4... I cannot change a view position in all cases. Deleting of index files do not resolve the situation - seems indexes are not valid.

A x86 version works as charm. Also in some cases a 64 bit version uses a 32-bit-generated index files. Also switching an edition produces every reindex. It is very bad, because eats a time. Please do not overwrite existing indexes, if they are exist and creates by a wrong edition. Create a new one with a different extension or save a new portion in the same file. Or (the best solution) - use the same file format for x64 and x86 edition.

So, I returned to x86 with temp lossless files

Provide a sample file.

Regarding indexes: neither the Avisynth plugin nor ffmsindex.exe will treat an index file that was created with a different architecture as valid. As far as they are concerned the index file might as well never have existed at all. Thus, such an index file will always be silently overwritten.
Valid index files that were created with the same version of FFMS2 as the one doing the reading do have some degree of protection, but only if you specified the filename for the index file explicitly. Otherwise they too will be overwritten if they don't match the media file.

TheFluff · 10th September 2011, 04:58

Quote:

Originally Posted by rean

I have catched this bug on a simple x264-encoded file. Instructions, scripts, files, versions, screenshots are here: http://media.namerenie.info/bug/
It is my home PC, so please download them within these days.

Also please look at the bug with my file from a Sony NEX-5 (rewrapped to mkv). I cannot open it at all using ffms2.

Sorry for the big files.

I've downloaded the files but I won't have time to investigate until Sunday. I'll get back to you if nobody else has found the bug by then.

Chumbo · 10th September 2011, 16:13

Quote:

Originally Posted by TheRyuu

...
Indeed. It's been suggested before but no real work has been done on it yet.
...

+1 to use the same file format for both x86/x64 please.

Quote:

Originally Posted by TheRyuu

If you directly use ffvideosource it will overwrite without asking.

Would you kindly add this to the "wish list" to change please so ffvideosource would not recreate the index file automatically? So right now there's no point to use the CLI if using ffvideosource. Maybe add an input parameter to ffvideosource force recreating the index which defaults to not doing so if the file already exists. Thank you.

Myrsloik · 10th September 2011, 16:16

Quote:

Originally Posted by Chumbo

+1 to use the same file format for both x86/x64 please.

Would you kindly add this to the "wish list" to change please so ffvideosource would not recreate the index file automatically? So right now there's no point to use the CLI if using ffvideosource. Maybe add an input parameter to ffvideosource force recreating the index which defaults to not doing so if the file already exists. Thank you.

Hint: just specify the index name if you really have to mix stuff like that

Also, x86/x64 will probably never use the same index format or at least the work for doing so would require reading piles of libav code to check so it really does behave identically

TheFluff · 12th September 2011, 07:21

Quote:

Originally Posted by rean

Another bug. I use my function FF_Clip() from my previous bugreport with an x86 version.
I get "FFVideoSource: Insanity detected: decoder returned an empty frame" after this code:

SelectRangeEvery(50, 1, 0, false)

and some processing time on every my h264 source.

That (probably) isn't really a bug in FFMS2. The error message means exactly what it says: libavcodec returned a video frame object with no actual image data in it. Prior to 2.16 the behavior with such frames was to just crash.

Sometimes you can make this problem go away by setting threads=1.

edit: since you're already wrapping ffvideosource() etc and are dynamically loading ffms2.dll, it'd be pretty simple for you to add a global "is_x64" variable and add an arch-specific filename suffix to the index files based on that. That'd at least solve your problems with index files overwriting each other.

TheFluff · 12th September 2011, 07:39

Quote:

Originally Posted by rean

But they are really exist. I able to point to the bug-catched fame by direct point in vdub using a timeline. This message is only when I process frames from one to one.

I don't doubt that you're having the issue, I'm just saying that the bug is probably inside libavcodec, not in FFMS2 itself. Try with threads=1 as I suggested and see if the issue is still there.

I haven't had time to look at your samples yet, but I'll do that Soon(tm).

25th August 2011, 23:00	#1261 \| Link
TheFluff Excessively jovial fellow Join Date: Jun 2004 Location: rude Posts: 1,100	New test build: ffms2-r532.7z Edit: now with 100% more pthreads thanks to Myrsloik. We're slowly working on trying to fix enough bugs to motivate a release of 2.16, so test everything, submit all your weirdest sample files and win a prize if it causes a reproducible crash. Test audio in particular since we've been messing around with that a lot lately. Last edited by Guest; 7th June 2012 at 21:26. Reason: rule 4

26th August 2011, 09:25	#1262 \| Link
TheRyuu warpsharpened Join Date: Feb 2007 Posts: 787	Yet another new test build: ffms2-r533.7z All of the above plus this should fix the invalid postprocess bug (for good). Go forth and test. Last edited by TheRyuu; 26th August 2011 at 10:32.

26th August 2011, 21:48	#1263 \| Link
TheFluff Excessively jovial fellow Join Date: Jun 2004 Location: rude Posts: 1,100	FFMS 2.16 has been released. It is mostly a bugfix release, but it adds a few features as well; most importantly full support for YUV 4:4:4 as well as high bitdepth YUV. The following downloads are now available: ffms-2.16.7z - the usual binary. Includes multithreaded decoding functionality; you do no longer need a special -mt version. ffms-2.16-x64.7z - 64-bit version, for use with avs64. Note that it is a native Avisynth plugin now; no need for LoadCPlugin or LoadStdcallPlugin. ffms-2.16-src.tar.bz2 - source code. ffms-2.16_SDK.7z - package for Windows developers who want to develop programs that use FFMS2 but don't want to deal with MinGW. Full changelog since 2.15: Reimplemented output colorspace selection, this should fix all issues with the avisynth plugin when opening yuv420p10 or yuv444 material plus several other less common cases. (Myrsloik) Added FFMS_SetOutputFormatV2 to the API. This function allows you to specify PixelFormats >= 64 for use as output. (Myrsloik) Fixed a serious bug that could cause crashes and completely useless index files with h264 in matroska. (Myrsloik) Automatically detect number of decoding threads. The avisynth video source funtion already did this, now moved so the api can use it as well. (TheRyuu) Re-add the ability to target x64 with msvc since it's a bit more sane now. (TheRyuu) Fixed a bug that could cause crashes when reading audio if FFMS2 was compiled with GCC. (Myrsloik) ffmsindex will no longer crash if it cannot open the file given to it for indexing. (TheFluff) FFMS2 will no longer crash if the video decoder feeds it an empty frame (can sometimes happen when using lots of decoder threads); you'll get a nice error message instead. (TheFluff) The C-plugin can now act as both an Avisynth 2.6 plugin (including support for new colorspaces) as well as an Avisynth 2.5 one, in the same binary. (kemuri_-9) Fixed an issue that could cause opening Vorbis audio to fail because FFMS2 couldn't find an initial valid PTS. (TheFluff) FFMS2 will no longer crash if forced or tricked into using an index file generated by a FFMS2 version compiled for a different architecture. (TheRyuu) Fixed a crash when the last frame was requested using the Avisynth plugin's forced CFR mode. (Plorkyeran) Fixed various issues with decoding audio from the Ogg container without Haali's splitter. (Myrsloik, TheFluff) Fixed the "invalid postprocessing settings"; they were caused by a parsing bug in libpostproc, and a workaround has been added. (Myrsloik) Tinkered a bit with the non-MSVS build system. (Daemon404, Kovensky) Important notice for postprocessing users: Support for postprocessing in FFMS2 will be dropped in the next release. The reason is that both libav is dropping the libpostproc library from their own releases, and so we cannot continue supporting it. Last edited by TheFluff; 5th September 2011 at 21:20.

1st September 2011, 00:32	#1267 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	When cross-compiling the C plugin, the recent version.sh-related changes result in the .dll being built as ffms.dll instead of ffms2.dll. On Windows this isn't an issue, because of MSys' bash quirks or file permissions for NTFS always being executable or something. This is the problematic section: Code: version=`version.sh` API=$(echo $version \| cut -f 1 -d '.') On Linux, this results in version.sh not being found when configure gets called (so the version info is left blank and the '2' isn't inserted into the filename of the .dll), and even when having used chmod +x on version.sh, it seems to still not want to add it as an environment variable. I managed to fix it (and have it work on both Windows and Linux), by commenting out the version ENV line and adjusting the API variable to use version.sh's filename directly: Code: #version=`version.sh` API=$(echo $(./version.sh) \| cut -f 1 -d '.')

1st September 2011, 10:40	#1268 \| Link
kemuri-_9 Compiling Encoder Join Date: Jan 2007 Posts: 1,348	oops -> fixed __________________ custom x264 builds & patches \| F@H \| My Specs

31st August 2011, 17:53	#1265 \| Link
chipzoller Mr. Woof Join Date: Jan 2002 Location: USA Posts: 784	I'm not sure if this thread also doubles as the "official" support thread and so hate to cross-post, but I'm having an issue with 2.16 throwing "decoder returned an empty frame". Original thread is here and will gladly merge/delete if desired: http://forum.doom9.org/showthread.php?t=162391

1st September 2011, 12:15	#1269 \| Link
henryho_hk Registered User Join Date: Mar 2004 Posts: 889	why not simply Code: API=$(./version.sh \| /bin/cut -f 1 -d '.')

5th September 2011, 21:22	#1271 \| Link
TheRyuu warpsharpened Join Date: Feb 2007 Posts: 787	ffms-2.16-x64.7z - 64-bit version, for use with avs64. Note that it is a native Avisynth plugin now; no need for LoadCPlugin or LoadStdcallPlugin ffms-2.16_SDK.7z - updated to now include the 64-bit stuff. Little late. Have at it. Last edited by TheRyuu; 6th September 2011 at 00:47.