FFmpegSource - C-plugin

[qyot27]

Since it's easier, I'll keep this first post updated with newer builds whenever one happens.

Issues tracker (for C plugin issues only): https://github.com/qyot27/ffms2_cplugin/issues

FFMS2 C-plugin r1470+181 (https://www.mediafire.com/file/ws6pe3j6oddxwed/ffms2_r1470%252B181.7z/file)

Standard C-plugin builds are in amd64, i686, and aarch64 directories.

The c++_avsgcc directory contains builds of the C++ plugin for use with
GCC builds of AviSynth+. To use them, create a plugins_gcc or plugins64_gcc
directory and place them there instead of in plugins(+) or plugins(+)64.
The plugins*_gcc directory probably also needs to have a registry entry
added for it like the standard 32-bit and 64-bit plugin directories have.


ffmpeg version r118355+62 master-ec4d3dc5b9 HEAD-810165e432
contains: avs_pixfmts datetime merged new_pkgconfig silent_invoke versioninfo
Copyright (c) 2000-2025 the FFmpeg developers
built on Jan 30 2025 20:31:57 with gcc 14.2.0 (GCC)
libavutil 59. 55.100 / 59. 55.100
libavcodec 61. 31.101 / 61. 31.101
libavformat 61. 9.106 / 61. 9.106
libavfilter 10. 6.101 / 10. 6.101
libswscale 8. 13.100 / 8. 13.100
libswresample 5. 4.100 / 5. 4.100
libpostproc 58. 4.100 / 58. 4.100

configuration (amd64):
--prefix=/home/qyot27/mpv-build-deps/ffmpeg_build_for_ffms2/amd64
--cross-prefix=x86_64-w64-mingw32-
--enable-gpl
--enable-version3
--disable-w32threads
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--enable-libdav1d
--disable-decoder=aac_fixed
--disable-decoder=ac3_fixed
--disable-decoder=mp1
--disable-decoder=mp2
--disable-decoder=mp3
--disable-decoder=mp3adu
--disable-decoder=mp3on4
--cpu=core2
--extra-cflags='-march=core2'
--target-os=mingw32
--arch=x86_64


configuration (i686):
--prefix=/home/qyot27/mpv-build-deps/ffmpeg_build_for_ffms2/i686
--cross-prefix=x86_64-w64-mingw32-
--windres='x86_64-w64-mingw32-windres -F pe-i386'
--enable-gpl
--enable-version3
--disable-w32threads
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--enable-libdav1d
--disable-decoder=aac_fixed
--disable-decoder=ac3_fixed
--disable-decoder=mp1
--disable-decoder=mp2
--disable-decoder=mp3
--disable-decoder=mp3adu
--disable-decoder=mp3on4
--cpu=pentium3
--extra-cflags='-m32 -mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
--extra-ldflags='-m32 -L/usr/i686-w64-mingw32/lib'
--target-os=mingw32
--arch=x86


configuration (aarch64):
--prefix=$HOME/mpv-build-deps/ffmpeg_build_for_ffms2/aarch64
--cross-prefix=aarch64-w64-mingw32-
--windres="aarch64-w64-mingw32-windres -I/usr/llvm-mingw/generic-w64-mingw32/include"
--enable-gpl
--enable-version3
--disable-w32threads
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--enable-libdav1d
--disable-decoder=aac_fixed
--disable-decoder=ac3_fixed
--disable-decoder=mp1
--disable-decoder=mp2
--disable-decoder=mp3
--disable-decoder=mp3adu
--disable-decoder=mp3on4
--extra-cflags="-I/usr/aarch64-w64-mingw32/include"
--extra-ldflags="-I/usr/aarch64-w64-mingw32/lib"
--target-os=mingw32
--arch=aarch64

FFMS2 C-plugin r1315+119 (http://www.mediafire.com/file/oi31n1lcrzdvkno/ffms2_r1315%2B119-avs%2Bvsp_lastxp.7z) (Final LastXP build)

Optimized for Pentium-III and SSE (32-bit)
Optimized for Core2 (64-bit)

ffmpeg version r90798 master-21da248b5f HEAD-a56580b117
contains: lastxp_wincrypt
Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.2.0 (GCC)
libavutil 56. 13.100 / 56. 13.100
libavcodec 58. 17.100 / 58. 17.100
libavformat 58. 11.101 / 58. 11.101
libavfilter 7. 15.100 / 7. 15.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 0.102 / 5. 0.102
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

32-bit configuration:
--prefix=/home/qyot27/ffmpeg_build_for_ffms2/32bit
--cross-prefix=i686-w64-mingw32-
--enable-gpl
--enable-version3
--disable-w32threads
--enable-avresample
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--extra-cflags='-mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
--target-os=mingw32
--arch=x86

64-bit configuration:
--prefix=/home/qyot27/ffmpeg_build_for_ffms2/64bit
--cross-prefix=x86_64-w64-mingw32-
--enable-gpl
--enable-version3
--disable-w32threads
--enable-avresample
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--extra-cflags='-march=core2'
--target-os=mingw32
--arch=x86

FFmpeg has now switched from wincrypt to bcrypt, and this is too far-reaching of a change for any selective revert to handle - the system library it links against changed (and bcrypt.dll is not available on XP; it's part of Vista+'s API), and far more importantly, the code that requires the random_seed functions that the *crypt library is called in for is spread throughout the FFmpeg codebase - it's used in encoders, decoders, and [de]muxers, so unlike the winthread/pthread thing last time, this time XP compatibility is probably broken for good. While it could probably hang on for a while doing increasingly labor-intensive merge integrations to revert the bcrypt commits, make sure everything still compiles, and that no unwanted behavior arises because of the use of winthread, I'm not particularly enthused to do so, and the alternatives (Wine, ReactOS, using Wine in Windows if you can get it compiled) are more compelling in the long term.

[Reino]

There was a vote regarding this on the FFmpeg-devel mailing list a couple weeks ago (thread starts here (http://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222354.html)), so as of commit 9b121dfc32810250938021952aab4172a988cb56, support for Windows XP was dropped in FFmpeg-git, which now requires the use of the newer lock and thread APIs introduced in Vista.
I've seen the vote and as far as I know this only concerns w32threads.
https://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222869.html (https://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222869.html):
...
Windows XP is hereby not a supported build target anymore. It was
decided in a project vote that this is OK. (Technically, it could still
be built for Windows XP using an external pthread lib as of this
commit.)

I've just released another WinXP compatible FFmpeg build (https://ffmpeg.zeranoe.com/forum/viewtopic.php?p=12998#p12998) using pthreads. I haven't extensively tested it myself, but so far it's working just fine.

[lvqcl]

I've seen the vote and as far as I know this only concerns w32threads.

No, the vote (http://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222354.html) was to drop WinXP support:
"We should drop XP support, and allow unconditional use of Windows Vista APIs"

I've just released another WinXP compatible FFmpeg build using pthreads. I haven't extensively tested it myself, but so far it's working just fine.

Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.

[manolito]

Thanks so much qyot27 for this new version... :thanks:

As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again. I tested it under WinXP with a non-SSE2 capable CPU, no problems whatsoever.


Thanks again and Cheers
manolito

[qyot27]

Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.
I sort of forgot that w32threads were a separate method. And I actually do use --disable-w32threads (IIRC, because FFMS2's C++11 threading requires MinGW-w64's win32-pthreads, and I can't remember if w32threads plays nice with that), so I guess it might actually hang on a while longer.

The uncertainty about it breaking somewhere deeper at any time, though...

As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again.
It's been quite some time since I did the update to support AviSynth+'s huge number of new colorspaces in the C plugin parts, so I don't remember whether I actually fixed that differently than the previous build or not. Good to hear that it's working, though.

[StainlessS]

Thank you Q Branch, your continued support has always been appreciated (now where did I leave Moneypenny?).

[FranceBB]

@qyot27... thank you very much indeed for supporting XP :D

[TheFluff]

y'all XP diehards might want to take note of this (https://meltdownattack.com/) new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable

[manolito]

y'all XP diehards might want to take note of this (https://meltdownattack.com/) new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable

Yes, and according to this:
https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/
I will also not get a slowdown of my computer for up to 30% which these (future) patches will introduce... :devil:

Cheers
manolito

[lvqcl]

ffmpeg-related tasks are usually computationally intensive, so performance penalty for them should be close to 0.

P.S. anyway, even 30% performance hit is better than a new ransomware like WannaCry or (Not)Petya.

[Andouille]

y'all XP diehards might want to take note of this (https://meltdownattack.com/) new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable

What does this have to do with video processing ?



@qyot27... thank you

[manolito]

Good to hear that it's working, though.

After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems... :o

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.

I tried some workarounds like adding the folder from where ffms2 is loaded by AVStoDVD to the environment path variable. I also disabled DEP completely, but nothing helped. I ended up going back to the old ffms2 version from Oct 2016.


Any ideas what is happening here?

Cheers
manolito

[Midzuki]

After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems... :o

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.

Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.

[StainlessS]

ancient desktop PC running WinXP

So WXP_32 and W7_64, did anyone try WXP_64 ? [EDIT: or indeed W7_32]

[manolito]

Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.

Thanks for confirming my findings...

The thing which I do not understand is why it works under WinXP. Of course the old desktop computer has a single core CPU (Celeron Coppermine 1.1 Ghz, made in the year 2000), and it has very low system RAM (576 MB). But it runs the same version of AviSynth (2.61 Alpha VC6) as the other Win7-64 computers.

So I tried a few more things under Win7-64. First I disabled Hyperthreading, then I completely disabled multicore processing. I added the "threads=1" parameter to ffvideosource. And I tried to force AVStoDVD to only use 1 CPU core globally. Nothing helped.

I could not even get AVSMeter to create a report about the AVS script which used ffms2. AVSMeter crashed immediately.


So I have to conclude that this latest version of the qyot27 C-Plugin does not work under a 64-bit OS using the standard AVS versions (I forgot to mention that I also tried using different builds of AVS versions 2.60 and 2.61). I did not test it under AVS+ because I have no intention to upgrade to AVS+ in the forseeable future.


Cheers
manolito

[qyot27]

It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.

The 'why does it work on XP?' issue had me stumped (I'd mentioned it in passing with the build before this), but as this MSDN thread notes (https://social.msdn.microsoft.com/Forums/vstudio/en-US/21af5f8d-01a9-41cf-9134-41c12d3eab7e/malloc-crashes-my-program-in-vista-but-works-fine-in-xp?forum=vcgeneral) is more than likely down to XP's heap manager not being as comprehensive about warning users about corruptions that occur. So the bug is still there, but XP doesn't bother telling you about it.

The actual issue itself may have been a longstanding bug from back when kemuri-9 was the one doing C-plugin stuff (since the actual line that fails was virtually unchanged since then) or it was a regression that happened when the allocation method for AviSynth.dll changed recently to get rid of calloc (in which case it'd be my fault for not noticing it was trying to NULL the avs_lib after it had been freed).

[Selur]

It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.
Any new on the new build? :)

[qyot27]

Any new on the new build? :)
Yup.

FFMS2 C-plugin r1293+111

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.

EDIT 2018-04-23: Newer build available here. (https://forum.doom9.org/showthread.php?p=1839968#post1839968)

[burfadel]

What's the code separation like between the main branch and the FFMS2000 experimental branch?

[Midzuki]

Yup.

FFMS2 C-plugin r1293+111 (http://www.mediafire.com/file/h052qy7xd74v2ll/ffms2_r1293%2B111-avs%2Bvsp.7z)

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.

:goodpost:

:thanks: for the new build, now it works without causing crashes.

[manolito]

+1

This new hotfix build is a winner... :thanks:

Tested under WinXP and Win7-64bit, no problems whatsoever.

Thanks again and Cheers
manolito

[qyot27]

What's the code separation like between the main branch and the FFMS2000 experimental branch?
Wrong thread?

[burfadel]

Well, this is built from the main branch, so it's relevant here since it's a more recent build than FFMS2000.

[qyot27]

Well, this is built from the main branch, so it's relevant here since it's a more recent build than FFMS2000.
It's actually built from one of:
the c_plugin (https://github.com/FFMS/ffms2/commits/c_plugin) branch on the upstream FFMS2 repo, if the C plugin is fully caught up with the master branch
the cplugin_master (https://github.com/qyot27/ffms2/commits/cplugin_master) branch on my personal working repo, when there aren't any C-plugin-specific fixes or patches I have to apply.
the patches (https://github.com/qyot27/ffms2/commits/patches) branch, when something needs to be fixed for cplugin_master to build or for other issues with the C-plugin (read: the contents of src/avisynth_c/) to be resolved.

What this means is that, 999 times out of 1000, the FFMS2 core in the C-plugin builds will be the same as it is as of the HEAD position of upstream's master branch at the time I built it, exempting a couple spots required to get MinGW-w64 to build it at all (and those commits are old and already in the c_plugin branch).

It's not really relevant to a comparison with the upstream release builds or the test builds in the FFMS2000 thread, though, because those are the standard C++ plugin, and that's separate from the C-plugin's code. The FFMS2000 changes were merged back into the master branch in November, and the builds from after that are just normal HEAD builds from upstream/master.

[FranceBB]

I got half green screen, decoding an MPEG-2 .ts file.
It didn't happen in the former version of ffms.
It seems like it's somehow expecting high bitdepth, but MPEG-2 files are 8bit.
This is the result: Image (https://i.imgur.com/5ELRAy3.png)
It's a common MPEG-2 .ts file, except for the fact that it's 4:2:2 and it didn't go on air; there shouldn't be any problem with it anyway.
Avisynth: 2.6.1
OS: Windows XP Professional x86 with extended support

ID : 21 (0x15)
Menu ID : 1 (0x1)
Format : MPEG Video
Format version : Version 2
Format settings, BVOP : No
Format settings, Matrix : Custom
Format settings, GOP : Variable
Codec ID : 2
Duration : 23 min 9 s
Bit rate mode : Constant
Maximum bit rate : 50.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first frame : Closed

[qyot27]

Does it happen with the latest build of the C++ plugin?

[FranceBB]

Yes, it behaved exactly the same, but I just found out that the file is corrupted.
(MD5 doesn't match).
I'm just gonna transfer it again. Nothing to be worried about, my bad. ^_^

[qyot27]

FFMS2 C-plugin r1315+118 (Final LastXP build)

Optimized for Pentium-III and SSE (32-bit)
Optimized for Core2 (64-bit)

ffmpeg version r90798 master-21da248b5f HEAD-a56580b117
contains: lastxp_wincrypt
Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.2.0 (GCC)
libavutil 56. 13.100 / 56. 13.100
libavcodec 58. 17.100 / 58. 17.100
libavformat 58. 11.101 / 58. 11.101
libavfilter 7. 15.100 / 7. 15.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 0.102 / 5. 0.102
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

32-bit configuration:
--prefix=/home/qyot27/ffmpeg_build_for_ffms2/32bit
--cross-prefix=i686-w64-mingw32-
--enable-gpl
--enable-version3
--disable-w32threads
--enable-avresample
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--extra-cflags='-mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
--target-os=mingw32
--arch=x86

64-bit configuration:
--prefix=/home/qyot27/ffmpeg_build_for_ffms2/64bit
--cross-prefix=x86_64-w64-mingw32-
--enable-gpl
--enable-version3
--disable-w32threads
--enable-avresample
--disable-encoders
--disable-muxers
--disable-doc
--disable-debug
--disable-devices
--disable-avdevice
--extra-cflags='-march=core2'
--target-os=mingw32
--arch=x86

As noted above, this really is going to be the last XP-compatible build I can provide. FFmpeg has now switched from wincrypt to bcrypt, and this is too far-reaching of a change for any selective revert to handle - the system library it links against changed (and bcrypt.dll is not available on XP; it's part of Vista+'s API), and far more importantly, the code that requires the random_seed functions that the *crypt library is called in for is spread throughout the FFmpeg codebase - it's used in encoders, decoders, and [de]muxers, so unlike the winthread/pthread thing last time, this time XP compatibility is probably broken for good.

So yeah, at this point there's no point keeping my cross environment compatible with XP from the MinGW-w64 level.

EDIT 2018-04-25: Newer build available; check first post. (https://forum.doom9.org/showthread.php?p=1829061#post1829061)

[Selur]

Thanks a lot!

[Selur]

From the file name 'ffms2_r1315+117-avs+vsp_lastxp' I assumed these should also work with Vapoursynth.
Trying to use the 64bit version in Vapoursynth with:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Loading F:\TestClips&Co\files\Test-AC3-5.1.avi using FFMS2
clip = core.ffms2.Source(source="F:/TESTCL~1/files/TEST-A~1.AVI",cachefile="H:/Temp/avi_4a88093b3b83d19d00642a5a96b0af78_41.ffindex",format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Making sure input color range is set to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model (i420)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10)
# Output
clip.set_output()
I get:
Failed to evaluate the script:
Python exception: No entry point found in G:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1847, in vapoursynth.vpy_evaluateScript
File "H:\Temp\tempPreviewVapoursynthFile06_41_54_518.vpy", line 5, in <module>
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
File "src\cython\vapoursynth.pyx", line 1739, in vapoursynth.Function.__call__
vapoursynth.Error: No entry point found in G:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll

(32bit version works fine in Avisynth MT 2.6)

Cu Selur

[manolito]

Thanks so much qyot27 for this new version... :thanks:

Tested it thoroughly under Win XP and Win 7-64, works like a charm.


Maybe a little bit OT since this new version (the absolutely very very last XP version) will be perfect for some time...

What will be the alternative for stubborn XP diehards like myself once this version will fail with certain sources (and DSS2Mod will not work reliably)? This is my reasoning:

Luckily there are still three folks who still provide XP compatible FFmpeg builds (CoRoNe, Sherpya and AbeChin). Since FFmpeg decoders are updated frequently there is a good chance that these XP compatible builds will open newer source formats in the future.

So why can't we use FFmpeg.exe to decode sources and pipe them to stdout and then use some (yet to be written) AviSynth stdin source filter?

I had a look at Sashimi, but it seems awfully complex, and it relies on writing an intermediate raw file. Inserting FFmpeg decoded output into a DirectShow Filter Graph seems to be impossible (after doing some research).

What does work for me is using FFmpeg to create a lossless intermediate file (I used H264 CRF=0 with FLAC audio) and open this file with DSS2Mod and LAVFilters. Am I right to assume that there will be no seeking issues because the lossless video file only has key frames? The problem is just that the intermediate file will be huge, and creating it is time consuming. Some kind of piping it into AviSynth would be very welcome.

Any thoughts?

Cheers
manolito

[qyot27]

Maybe a little bit OT since this new version (the absolutely very very last XP version) will be perfect for some time...

What will be the alternative for stubborn XP diehards like myself once this version will fail with certain sources (and DSS2Mod will not work reliably)?
Wine under a Linux distro, ReactOS, or if you can get Wine cross-compiled*, transplanting it onto XP to run programs that require Vista+ APIs.

*I managed to partially do so once, to fix an issue with a DirectX .dll. Hopefully the situation is better than it was ca. 2010 and won't trip over itself and die during the compilation process. I don't have a lot of hope for that, but who knows?

Luckily there are still three folks who still provide XP compatible FFmpeg builds (CoRoNe, Sherpya and AbeChin). Since FFmpeg decoders are updated frequently there is a good chance that these XP compatible builds will open newer source formats in the future.

So why can't we use FFmpeg.exe to decode sources and pipe them to stdout and then use some (yet to be written) AviSynth stdin source filter?
Because you won't be able to run newer versions of FFmpeg (.exe or other programs linked to FFmpeg's libs) on XP. That was the underlying problem. It's not a configure switch that can simply turn XP support back on like --disable-w32thread could - they'll have to explicitly revert any commits that are bcrypt-related (which may mean doing complex merge integration in all those decoders every time something changes in them just to get the bcrypt commits to revert cleanly), then fix any compilation errors that might arise, and then cross your fingers that it doesn't introduce any unexpected behavior when using wincrypt.

FFmpeg commit a56580b117 (the one used in this build) is the last one that can work on XP without beginning to put in [potentially] a lot of extra leg work to keep it going. Even if it's minimal work to do so right now, it'll get progressively harder to do so.

I had a look at Sashimi, but it seems awfully complex, and it relies on writing an intermediate raw file. Inserting FFmpeg decoded output into a DirectShow Filter Graph seems to be impossible (after doing some research).

What does work for me is using FFmpeg to create a lossless intermediate file (I used H264 CRF=0 with FLAC audio) and open this file with DSS2Mod and LAVFilters. Am I right to assume that there will be no seeking issues because the lossless video file only has key frames? The problem is just that the intermediate file will be huge, and creating it is time consuming. Some kind of piping it into AviSynth would be very welcome.
I can't say. I've never used Sashimi because I've never come across a need to open raw video. Intra-only files shouldn't have any seeking issues, but in many cases that's more up to the demuxer than it is the video compression format.

[FranceBB]

Few issues with Avisynth 2.6.1 on Windows XP x86.

LoadCPlugin("ffms2.dll") in ffms2.avsi doesn't work, Load_Stdcall_Plugin("ffms2.dll") does.

If I try to index an H.264 4:2:0 8bit .mp4 file, it works flawlessly,
however when I try to index an H.264 4:2:0 10bit .mkv file, it says:

Error requesting frame xxxx
Windows error: exception access violation reading 0x00000000.

The latest stable ffms build (not ffms2000) from 2016 works fine.
I can't test the latest ffms2000 'cause I'm running XP.

Nothing special on mediainfo:

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 10@L3.1
Format settings, CABAC : Yes
Format settings, ReFrames : 10 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 1 min 32 s
Bit rate : 4 685 kb/s
Width : 1 024 pixels
Height : 576 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.331
Stream size : 51.4 MiB (91%)
Writing library : x264 core 148 r2744kMod b97ae06
Encoding settings : cabac=1 / ref=10 / deblock=1:-2:-2 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / fade_compensate=0.00 / psy_rd=0.70:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc=crf / mbtree=0 / crf=17.0000 / qcomp=0.60 / qpmin=5 / qpmax=40 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
Language : Japanese
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.601 PAL
Transfer characteristics : BT.470 System B, BT.470 System G
Matrix coefficients : BT.601

Audio
ID : 2
Format : AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_AC3
Duration : 1 min 31 s
Bit rate mode : Constant
Bit rate : 448 kb/s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 spf)
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 4.90 MiB (9%)
Language : Japanese
Service kind : Complete Main
Default : Yes
Forced : No

[manolito]

Few issues with Avisynth 2.6.1 on Windows XP x86.

LoadCPlugin("ffms2.dll") in ffms2.avsi doesn't work, Load_Stdcall_Plugin("ffms2.dll") does.


Tried to reproduce the issue, but I couldn't. Also on Win XP SP3 and AviSynth 2.61. LoadCPlugin in the ffms2.avsi works fine, I do not even have the AviSynth\plugins folder in my path.

Could it be that you have the old Avisynth_c.dll in your Plugins folder? See this post:
https://forum.doom9.org/showthread.php?p=1835702#post1835702


Cheers
manolito

[manolito]

Because you won't be able to run newer versions of FFmpeg (.exe or other programs linked to FFmpeg's libs) on XP. That was the underlying problem. It's not a configure switch that can simply turn XP support back on like --disable-w32thread could - they'll have to explicitly revert any commits that are bcrypt-related (which may mean doing complex merge integration in all those decoders every time something changes in them just to get the bcrypt commits to revert cleanly), then fix any compilation errors that might arise, and then cross your fingers that it doesn't introduce any unexpected behavior when using wincrypt.

FFmpeg commit a56580b117 (the one used in this build) is the last one that can work on XP without beginning to put in [potentially] a lot of extra leg work to keep it going. Even if it's minimal work to do so right now, it'll get progressively harder to do so.



Thanks for taking the time for explaining...
So my only hope for XP is that those 3 guys will find some tricks to maintain XP compatibility for some time to come...


Cheers
manolito

[FranceBB]

@manolito... yes, it was Avisynth_c.dll. I had it in my autoloading plugin directory. Thank you. ;)

And thanks to qyot27 for this new build.

[qyot27]

Hotfix build for the 64-bit VapourSynth issue Selur reported earlier.

EDIT 2018-04-25: Newer build available; check first post. (https://forum.doom9.org/showthread.php?p=1829061#post1829061)

[StainlessS]

Wine under a Linux distro, ReactOS, or if you can get Wine cross-compiled*, transplanting it onto XP to run programs that require Vista+ APIs.


ReactOS 0.48 is released recently. [EDIT: Ubuntu 18.1 Final due for release on 26th of this month]

With software specifically leaving NT5 behind, ReactOS is expanding its target to support NT6+ (Vista, Windows 8, Windows 10) software.


The NTFS driver coded by Trevor, during GSOC 2016/2017, has been finally added. The NTFS driver has been an ongoing effort started by Hervé and Pierre, and which needed 2 different Google Summer Of Code to reach its current state. Under the mentoring of Pierre, Trevor Thomson has been coding and documenting his titanic NTFS coding efforts. If you're interested in file systems or how NTFS works and the tools needed to understand its behavior, you shouldn't miss the blog posts of Trevor. Thanks to these efforts ReactOS is able to read NTFS partitions in a more robust way, covering NTFS specific cases, and since 0.4.8, ReactOS introduces initial NTFS writing support. The NTFS writing feature is disabled but can be enabled through registry to test its experimental support.


https://www.reactos.org/project-news/reactos-048-released

I've never got ReactOS to run as yet (dont think), and put that down to all of my systems being multi-boot (non-M$/non-Linux boot controller), am hopeful that the new version will at last work (have downloaded, not yet tried).

Thanx for the updates qyot27. :thanks:

[manolito]

I've never got ReactOS to run as yet (dont think), and put that down to all of my systems being multi-boot (non-M$ boot controller), am hopeful that the new version will at last work (have downloaded, not yet tried).

ReactOS has been on my watch list for a long time (but their development only progresses at a snail's pace). I really hope that it will be usable by the time M$ decides to kill Win7.

Please let us know once you tried it how it works for you. A lot of stuff I rely on is not available under Linux, and using Wine or have Windows installed in a VM looks clumsy to me. I just hope that ReactOS will succeed...


Cheers
manolito

[StainlessS]

I just hope that ReactOS will succeed..

Sad, but not just yet Mani,
Tried on Intel Board/bios core-duo quad m/c with all drives removed, all external cards/usb devices removed, running live USB ReactOS v0.48, and
sadly, stopped on about the 4th message
"Loading NTOSKRNL.EXE..."
"Loading HAL.DLL..."
"Loading KDCOM.DLL..."
"Loading boot drivers..."

Black Screen then nothing (monitor detects active display), CTRL/ALT/DEL reboot is disabled.

Tried on a second m/c IBM board/bios, did not remove any drives (3 internal attached), no internal cards attached, a couple of external USB cameras
attached; did not remove, same results as previous Intel m/c test, although after about 5 minutes, the fan became quite active but may have been
down to it being relatively warm day (does not usually have fan going continuous, which it did for at least 20 mins before shutting down, so CPU does seem to be
doing something).

I think on a previous attempt (perhaps with v0.47), it put up the usual long list of debug logging driver/system files, and used to hang at "Mup.Sys".

This is beginning of debug log (Safe Mode from boot F8 key) for XP SP3.

Loaded driver \WINDOWS\system32\ntkrnlpa.exe
Loaded driver \WINDOWS\system32\hal.dll
Loaded driver \WINDOWS\system32\KDCOM.DLL
Loaded driver \WINDOWS\system32\BOOTVID.dll
Loaded driver sptd.sys
Loaded driver \WINDOWS\System32\Drivers\WMILIB.SYS
Loaded driver \WINDOWS\System32\Drivers\SCSIPORT.SYS
Loaded driver ACPI.sys
Loaded driver pci.sys
Loaded driver isapnp.sys
Loaded driver pciide.sys
Loaded driver \WINDOWS\system32\DRIVERS\PCIIDEX.SYS
Loaded driver MountMgr.sys
Loaded driver ftdisk.sys
Loaded driver dmload.sys
Loaded driver dmio.sys
Loaded driver PartMgr.sys
Loaded driver VolSnap.sys
Loaded driver atapi.sys
Loaded driver disk.sys
Loaded driver \WINDOWS\system32\DRIVERS\CLASSPNP.SYS
Loaded driver fltMgr.sys
Loaded driver sr.sys
Loaded driver PxHelp20.sys
Loaded driver KSecDD.sys
Loaded driver WudfPf.sys
Loaded driver Ntfs.sys
Loaded driver inspect.sys
Loaded driver \WINDOWS\System32\DRIVERS\NDIS.SYS
Loaded driver \WINDOWS\System32\DRIVERS\TDI.SYS
Loaded driver pwdrvio.sys
Loaded driver Mup.sys

Above XP debug log for no particular reason, just showing how far it used to get (they seem to have changed/reduced debug progress messages).

It also used to fail (some prev versions) on a Dell m/c which I've recently consigned to the scrap heap (random failures, probably Motherboard,
PSU, or CPU faulty, mem seems ok).
Also dumped another Motherboard, down to only 3 now, and not set up to easily test on the 3rd board, so only above two results available.

Its strange that they have not been able to get the live image to boot on any of my machines, I presume that the image does not require
recent hardware.

ReactOS, apparently is able to run in only 96MB of RAM, M$ stuff is way too bloated.

[qyot27]

While it's a bit off-topic, when ReactOS refers to NT6+ compatibility, they're largely talking about stuff like the NT kernel itself, the system libs are already capable of running Vista+ programs if Wine can (ReactOS contributes to Wine development since they use parts of Wine themselves), but that wasn't their immediate priority because they were targeting Server 2003. I have managed to run AviSynth+ and either/both ffplay or mpv in one of last year's releases of ReactOS under a VM, and it worked fine for the kind of resource-strapped system I was running the VM on; but it was only a brief test to see if it actually could work, it wasn't anything intensive. I've not tried 0.4.8, but I've not been able to run it on the actual hardware on either of my own computers - but then again, one's using UEFI and the other uses SATA expansion card to host its boot drive, so it's probably more just that my setups are still a bit too exotic for what ROS currently has support for.

It's happily moved to both an intended 3-month release schedule and moved the source repos to Github, which to read their news posting(s) about it, seem to have attracted more outside contributions. Regardless, I remember playing with the then-current release 10 years ago (it may have been the early 0.3.x releases when I started taking notice of it?) and it's leaps and bounds ahead of where it was then.

[StainlessS]

Well this too is Off Topic, perhaps a moderator can move from post #38 to here [EDIT: Post #45] into a new thread, maybe into
'General', or 'Linux, OS X & Co' forum (my perference would be General, Maybe thread named 'ReactOS').

In my previous post, I made two screwups.

1), I had used the wrong ISO image on USB stick, as I had already written the Flash drive with intent of using
it to install test version of ReactOs on a machine that has now been scrapped due to faulty hardware.
I used the ReactOS Install ISO in testing, instead of the ReactOS Live CD ISO [originally created nearly a couple of weeks ago].

2), It seems that ReactOS does not as yet support USB devices (it seems to install some kind of USB hub drivers, but
dont support any USB devices themselves). Result was that in writing an ISO to a flash drive, it actually boots ok, but
then at some point cannot continue becase 'it dont support USB devices' [sometimes get a STOP 0x0000007B Blue screen].

I tried again after burning a CD with the Live CD image (live CD zip for v0.48 is about 75MB, expands to about 225MB ISO).
After some screwing around, I found that it boots OK with 1 Floppy, 1 IDE Pata and 2 SATA hard drives + 1 SATA SSD, +
1 PCI USB 5 port hub and 1 external USB drive attached to that add-on hub. It also works OK with Ethernet connected
and Switch/hub powered up. My ATI graphics card was removed for some reason several weeks ago so has not been tried in-situ.

Two devices were found to cause booting to fail, namely a Linksys WUSB54AG Wifi dongle, and a
TP-LINK_WN823N USB 300Mb Mini Wifi dongle (thumbnail size).

The MotherBoard used was an old Intel board DG965WH, with builtin ide, sata, usb, graphics, audio, and ethernet.
(Builtin serial, parallel, and FireWire are always disabled [EDIT: by me], as is a TPM [Trusted Platform Module, I think]).
CPU was Intel Core Duo Quad Core 2.4GHz Q6600, with 4GB RAM.
After booting into ReactOS, no USB devices worked (neither flash not external USB drives), nor the builtin Ethernet (no attempt
to install a driver for it). The Builtin Intel graphics worked just fine defaulting to 800x600 (dont recall color depth but
have impression that it was 24 or 32 bit). Tried change resolution to 1280x1024 and worked ok, did not try higher.
The CPU showed in Task Manager as Uni-Processor instead of Multi-Processor.
My NTFS partitions all showed up just fine, but I did not figure out how to enable NTFS write, read only.

It was kinda frustrating not having any device I could write to, as I had about 8 partitions all NTFS (read only) and
USB flash drives did not function at all. On starting up the Live CD, it asks for locale/keyboard, and then asks if you
want to run live or install, if you select install it produces a message "UserInit Failed to start Installer", maybe
the installer is only provided on the Install CD, or if a duel purpose Live/Install CD is created.

I removed all hard drives (in case of screwup) and temp installed an old 10GB drive and formated it as FAT32,
ran the live CD again and saved an image to the FAT32 partition, image is below.

I did not try the Live CD on my 2nd machine, it has no CD/DVD drive (replaced in drive bay with 3rd hard drive), and
not really convienient to use as it more often than not has no monitor attached, and sometimes neither keyboard nor mouse.
(access usually via 'My Network Places', or VNC [Tight-VNC/Tiger-VNC on Linux]).

So, ReactOS does actually work, but of limited use at present. think I saw somewhere that v0.50 would have NTFS working
(also will support linux partitions at some point, dont know if already so).
I'm a lot happier now I know that the project is indeed progressing, if point version increments every 4 months, then
would suggest that in about 8 months should have near full NTFS support.

https://s20.postimg.cc/ebtindin1/RTOS.jpg (https://postimages.org/)
EDIT: Above saved when desktop @ 800x600.

EDIT: By qyot27
3-month release schedule
so a 2 point increment in about 6 months, not 8.

@Manolito, Saw in the ReactOS FAQ, that minimum CPU is Pentium, so P3 is probably well plenty for you :)

EDIT: Maybe I have another try when I get another machine put together.

[FranceBB]

I personally use Linux and Windows XP in a virtual machine, for everything else I have Windows Server 2016 and I connect remotely to it via RDP to encode.
This is way off topic, but still... if you are willing to try an XP based OS, I think you should try Windows Shorthorn http://shorthornproject.com/
It's XP/Server 2003 based, but has many improvements, including a modified kernel that includes more functions and allows to run programs and drivers that wouldn't normally run on XP.
This is the closest thing you can get to an updated Windows XP.

Cheers.

[StainlessS]

Arh Damn!, another one to try, me gots bout 11 OS's that I've currently got in queue (incl 9 Linux, 1 Android, and ReactOS),
but does look very good, never heard of it before, thanx muchley FranceBB.

Tried search for a WikiPedia page on it, nuttin but cattle stuff found.

Vista eh, hope the state-of-the-art artificial stupidity managed to evade duplication.

Guess it might be lovely so long as you can turn of all of that horrid glitzy stuff, and get it back to real W95 Classic. :)

Again thanx F_BB

[qyot27]

If you scroll down, you'll notice that Shorthorn's 'newer program support' is basically that option I mentioned above about running Wine in Windows.

[FranceBB]

This is awkward...
After I removed Avisynth_C.dll I managed to load the C plugin with LoadCPlugin, but it still wasn't able to index my test file; no problem with a normal 8bit 4:2:0 H.264 .mp4.
Since it was the very first time that I was using the C plugin, I tried to remove all the other filters: same result.
I also tried to use "FFVideoSource" instead of FFMpegSource2: same behaviour.
Then, I tried older ffms2 C plugin releases, but they all prompted me to the same error, 'till I tried this one from 2016 https://forum.doom9.org/showthread.php?p=1783312#post1783312
FFMS2 C-plugin 1140+101 is working flawlessly on my machine.

Test:
FFMS2 C-plugin 1140+101 works
FFMS2 C-plugin r1293+111 exception access violation reading 0x00000000.
FFMS2 C-plugin r1293+111 exception access violation reading 0x00000000.
FFMS2 C-plugin r1315+118 exception access violation reading 0x00000000.

Not sure why.
I'm running Windows XP Professional x86 PAE.
Frameserver: Avisynth 2.6.1
CPU: Intel i7 6700HQ
RAM: 16 GB DDR4 (8x2)
GPU: NVIDIA GTX 950M
SSHD: Seagate 1TB

I don't know how to provide additional informations. Also please note that my CPU supports AVX and AVX2, but XP only uses SSE4.2, but that shouldn't be a problem. I also have PAE enabled and unlocked and I can use memory above 4GB thanks to the HAL (Hardware Abstraction Layer) that translates it from 32bit programs to 36bit hardware. Anyway, none of this should compromise the usage of ffms2 C plugin. It's weird.

[Groucho2004]

Then, I tried older ffms2 C plugin releases, but they all prompted me to the same error, 'till I tried this one from 2016 https://forum.doom9.org/showthread.php?p=1783312#post1783312
FFMS2 C-plugin 1140+101 is working flawlessly on my machine.

Test:
FFMS2 C-plugin 1140+101 works
FFMS2 C-plugin r1293+111 exception access violation reading 0x00000000.
FFMS2 C-plugin r1293+111 exception access violation reading 0x00000000.
FFMS2 C-plugin r1315+118 exception access violation reading 0x00000000.

I can reproduce that with a HEVC 420P10 clip. Specifying the correct color space makes no difference.
However, 10 bit sources seem to work fine with Avisynth+ with all plugin versions.

[qyot27]

I'm going to assume it's because I'd moved the C plugin to AviSynth+'s headers a while back. Also the note at the bottom of this post: https://forum.doom9.org/showpost.php?p=1799448&postcount=2351

[manolito]

After seeing FranceBB's post I started playing around with different ffms2 C-Plugin versions...

What I found is that the issues are not restricted to 10-bit or hi color sources. The test file from this post also caused indexing issues:
https://forum.doom9.org/showthread.php?p=1840071#post1840071

This file is broken, it does not demux or remux properly. But it can be reencoded with FFmpeg and AVS using DSS2Mod with LAVFilters. The video format is HD AVC, 8 bit, 4:2:0, interlaced TFF.

The latest ffms2 C-Plugin versions can not decode it correctly. The length is 13 sec, but with the latest ffms2 versions the conversion stops after 7 sec. Going back to the old version FFMS2 C-plugin 1140+101 which also works for FranceBB the conversion finishes without problems. Here is the script I use (generated by AVStoDVD):

LoadCPlugin("E:\Programme\AVStoDVD\Lib\ffms2.dll")
LoadPlugin("E:\Programme\AVStoDVD\Lib\LeakKernelDeint.dll")

Audio = FFAudioSource("F:\Download\kuchikirukia - won't mux.ts", track=-1)
Video = FFVideoSource("F:\Download\kuchikirukia - won't mux.ts", track=-1, fpsnum=2997, fpsden=100, seekmode=0)

Video = Video.ConvertToYV12(interlaced=true)
Video = Video.LeakKernelBob(1,7,false,false)
Video = Video.Spline36Resize(720,480)
Video = Video.SeparateFields().SelectEvery(4,1,2).Weave()

AudioDub(Video, Audio)

It looks like it is generally a bad idea to use an ffms2 version which employs the AVS+ headers under a plain vanilla AVS 2.60. For my part I will revert to the FFMS2 C-plugin 1140+101 from 2016 which always worked reliably for me.


Cheers
manolito

[qyot27]

Another new build (https://forum.doom9.org/showthread.php?p=1829061#post1829061); this time, using the C plugin under AviSynth 2.6 shouldn't error out with access violations if given high bit depth video. It will spit back the 'FFVideoSource: No suitable output format found' error since it won't do automatic downsampling anymore. The assumption, given high bit depth video, is outputting the same pix_fmt unless you use ConvertTo later in the script. The colorspace= parameter should work again, though, so you can force 8-bit output when you know you need it.

For simplicity's sake, I'll just update the first post from now on instead of burying the release posts mid-thread.


Regarding the broken TS file mentioned above, using the new build (r1315+119) and a simple script:
v=FFVideoSource("kuchikirukia - won't mux.ts")
a=FFAudioSource("kuchikirukia - won't mux.ts")
AudioDub(v,a)
All 13 seconds converted and encoded file played back fine. When using the expanded script manolito posted, it crashed at 6½ seconds (roughly). I whittled down the problem to the fpsnum=/fpsden= settings. Which is strange, since I don't think I've ever touched anything related to those in the C-plugin; anyone that can test with the main C++ FFMS2, does it happen when fpsnum/fpsden on this clip?

[FranceBB]

I see... Thanks for this new build, but I would still change a thing.
(I know, I know, you are gonna hate me right now xD).
I tried to open a 10bit file and it says "no suitable output format found", as expected.
I tried to output an 8bit dithered video from a 10bit source and it worked, as expected.
I then tried to output a 16bit video via the 10bit hack, but it didn't work.
My question is: how can I output a 16bit video (stacked - Dither Tools - or interleaved - HDRCore) in plain Avisynth via ffms2 C plugin?
I think you should enable the 10bit hack for Avisynth users, otherwise we won't be able to preserve the bitdepth of the source.

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv")

no suitable output format found

---

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv", enable10bithack=true)

FFVideoSource does not have a named argument "enable10bithack"

---

FFVideoSource("jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv", colorspace="YV12")

Works: output is dithered down 8bit 4:2:0 video

Thank you in advance.

[qyot27]

No. The 10bit hack was never in the C plugin (or mainline FFMS2, for that matter), and there's no reason for it to be put in now that we have an option that can preserve and give proper high bit depth input/output. If you want to use high bit depth, use AviSynth+.

[manolito]

When using the expanded script manolito posted, it crashed at 6½ seconds (roughly). I whittled down the problem to the fpsnum=/fpsden= settings. Which is strange, since I don't think I've ever touched anything related to those in the C-plugin; anyone that can test with the main C++ FFMS2, does it happen when fpsnum/fpsden on this clip?

Thanks for looking into it, and thanks for the new build, too... :thanks:

I tried using the very old C++ version 2.17 of ffms2 (the last version which runs on my machine), and it handled it nicely. I think I saw a short hickup pause at 6 1/2 seconds, but the conversion went on smoothly.

I was too lazy to pull out my Win7 laptop from the closet to test it with version 2.23.1 (last version on GitHub), but I think it does not really matter. The problem clearly comes from the broken source file. After transcoding this source to exactly the same format using either FFmpeg or AviSynth -> H.264 and feeding the transcoded file to AVStoDVD using the latest ffms2 C-plugin (making sure that fpsnum and fpsden were in the params list), the conversion proceeded all the way to the end.

My conclusion is that the older ffms2 versions are just more tolerant when they encounter broken source files. No big deal...


Cheers
manolito

[qyot27]

If you're using 2.17 to test, make sure to force the lavf demuxer; that's the only way to make sure it's directly comparable to current builds. The Haali demuxers (which may have been more tolerant of bad streams, I'm not sure) were removed a couple or three years ago, so newer FFMS2 builds only use libavformat.

[manolito]

OK, I did my homework, here are the results:

Not so sure about using Haali demuxers when testing version 2.17, simply because Haali is not installed on my computer. I do have avss_26.dll from DSS2Mod in my plugins folder, but would this be enough to enable Haali demuxers in ffms2?

Anyways, I fired up my Win7-64 laptop and repeated the conversion using the C++ version 2.23.1 from GitHub and the latest FFMS2000 test8 build from Myrsloik. Interesting results:

Version 2.23.1 handled the broken source file just fine, FFMS2000 crashed at the same point where the current C-plugins crash.

Working:
C++ versions 2.17 and 2.23.1
C-plugin version 1140+101

Not working:
C++ version FFMS2000 test8
All C-plugin versions after 1140

But again, this is caused by the broken source file. After transcoding this source file all ffms2 versions handled it correctly.


Cheers
manolito

[qyot27]

Ok, well if it occurs in mainline FFMS2, then the root cause is in the FFMS2 core, not in the plugins. Since it only occurs with fpsnum/fpsden, it probably involves one of the good handful of commits about timestamp or frame rate handling that came in over the course of 2017, although without bisecting it, I won't know for sure which one is the culprit. Likely it's specifically using fpsnum/fpsden on broken files like that, because the timestamps are already messed up.

My memory is really fuzzy involving the Haali stuff (but it doesn't matter since we know it happened in both the C++ and C plugins around the same time). The MPEG and OGG demuxers relied on the Haali DirectShow splitter, though, so I'm not sure if that was the *.dll or the *.ax file.

[Selur]

@qyot27: Encountered a strange problem with this (https://drive.google.com/file/d/1kmhaQxO3Au6B6G7Fuv4BppY43Q4ASxtI/view?usp=sharing) file.
It works fine with the 64bit Vapoursynth filter, but with Avisynth 2.6 MT I get dropped and repeated frames.

Using just FFVideoSource:
FFVideoSource("C:\Users\Selur\Desktop\2SOURC~1.AVI",cachefile="H:\Temp\avi_3a7584798a428c1a525d134a8f55420b_18467_1_0.ffindex",fpsnum=30000,fpsden=1001)
# current resolution: 640x480
return last
Frame with time 14 is shown as frame 14&15.
Frame with time 29 is shown as frame 29 and 30.
Frame with time 31 is missing.
Frame with time 32 is shown as frame 32&34.
Frame with time 33 is missing.
Frame with time 54 is shown as frame 54&55.
Frame with time 55 is missing.
Frame with time 56 is shown as frame 56&57.
Frame with time 59 is shown as frame 59&60.
Using just FFms2000:
same result as when using FFVideoSource.
Using just LWLibavVideoSource:
All frames are properly decoded.
Installing a Xvid vfw decoder and using AviSource:
All frames are properly decoded.

Using Vapoursynth + FFMS2K
All frames are properly decoded.
Using Vapoursynth + FFVideoSource
All frames are properly decoded.
Using Vapoursynth + LibAV
All frames are properly decoded.

Cu Selur

[manolito]

Looks like a problem with Avisynth 2.6 MT to me. Under plain vanilla AviSynth 2.60 (and 2.61 Alpha) I cannot reproduce these issues, I cannot see any dropped or repeated frames.

Have you tried to use MPEG4 Modifier by Moitah to remove the Packed Bitstream from the source file?


Cheers
manolito

[qyot27]

That it does the same thing when using FFMS2000 means that the issue is not C-plugin-specific. That it doesn't occur with VapourSynth means it's probably not in the core.

Here's the thing, though: it only happens when forcing fpsnum= and fpsden=. It doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion. So that would seem to suggest that it's an issue with the AviSynth plugin interface code, but since FFMS2000 and the C-plugin use two entirely different plugin codebases, the fpsnum/den bug when handling AVI would have to either have been implemented separately in the C-plugin, or it arises from some quirk of AVI, AviSynth, libavformat and/or libavcodec, or some combination of the three.

It does seem to be AVI-specific; I remuxed into MKV two different ways, as native MPEG4 and as VFW-compat, and both of them decoded the frames correctly even with fpsnum/den present.

[manolito]

it only happens when forcing fpsnum= and fpsden=.

This is only partly true... :devil:

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

This means that we are dealing with a very stupid bug in one of the libraries which the latest C-plugin uses. Is there any chance to find and fix this bug?


Cheers
manolito

[hello_hello]

Here's the thing, though: it only happens when forcing fpsnum= and fpsden=. It doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion.

FPSNum and FPSDen have been problematic for as long as I've been using them. I think it's due to timecode jitter and as a result, FFmpegSource sometimes drops or duplicates frames unnecessarily so it's best not to use frame rate conversion unless you need to convert the frame rate.

FFmpegSource isn't alone though. LSmash can do the same thing. Mercifully though, they don't tend to misbehave for the same sources, so usually if one doesn't work correctly, the other will.

And there's always the VFRToCFR (https://forum.doom9.org/showthread.php?t=165045) plugin.

[hello_hello]

This is only partly true... :devil:

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

I don't know how timecodes work or what the source file was, but that's probably just some sort of timecode jitter that doesn't upset FFmpegSource when it's using 2997/100.

Sometimes remuxing before indexing can fix the problem, even if you have to use a different container.

I've bumped into this problem when encoding 25fps AVIs, so it's not the NTSC frame rate as such.
After trying to convince Zathor it'd be a good idea for MeGUI to use it by default, I then tried to convince him that it wasn't so clever after-all.
https://sourceforge.net/p/megui/feature-requests/591/

[manolito]

It gets even weirder... :rolleyes:

I reverted back to the older C-plugin version 1140+101 fully expecting that the fpsnum/fpsden issue would be solved (like it did in earlier tests). But not this time. The output showed the same errors as the latest C-plugin version 1315+119.

Then I went back to the ancient Standard-plugin version 2.17, and this version handled the source correctly. Of course this is not really helpful because this old version cannot decode HEVC sources.

What does this mean? Could the issue lie in the C-plugin core after all? So far the only reliable workaround when using the C-plugin seems to be to ditch fpsnum/fpsden completely and follow the ffvideosource call immediately with a ChangeFPS command, like qyot27 recommends (I still do a lot of conversions to DVD, and VFR to CFR conversion is essential for me).


Cheers
manolito

[hello_hello]

It gets even weirder... :rolleyes:

I reverted back to the older C-plugin version 1140+101 fully expecting that the fpsnum/fpsden issue would be solved (like it did in earlier tests). But not this time. The output showed the same errors as the latest C-plugin version 1315+119.

Then I went back to the ancient Standard-plugin version 2.17, and this version handled the source correctly. Of course this is not really helpful because this old version cannot decode HEVC sources.

What does this mean? Could the issue lie in the C-plugin core after all? So far the only reliable workaround when using the C-plugin seems to be to ditch fpsnum/fpsden completely and follow the ffvideosource call immediately with a ChangeFPS command, like qyot27 recommends (I still do a lot of conversions to DVD, and VFR to CFR conversion is essential for me).

If the source was VFR, it'd be decoded at the average frame rate, the audio would no doubt go out of sync, and ChangeFPS() won't do anything to fix that. It has to be decoded at the correct frame rate.
For minor 29.970fps to 30000/1001 corrections I'd use AssumeFPS() instead. It'll probably only change the duration by a few nanoseconds anyway.
I kind of remember TIVTC uses 23.976 instead of 24000/1001 so I generally follow it with AssumeFPS() too, or I could be remembering wrong and that's a waste of time. :)

It might be the source. If you can, try remuxing it while setting the frame rate to 30000/1001 to see if that makes a difference. It might be that the source wasn't exactly 30000/1001 to begin with, and when it is, the problem will go away.

All that's assuming the source really is constant frame rate. Whenever MeGUI adds frame rate conversion to a script and it's not exactly standard NTSC or PAL, I always remove it and refresh the preview to see if the frame count and total duration changes. If the frame count stays the same and the duration only changes by a few milliseconds, then it's AssumeFPS() instead.

If it is a standard NTSC or PAL frame rate I remove it anyway as I've had too many bad experiences with dropped and incorrectly duplicated frames to use the frame rate conversion unless I really need to convert VFR to CFR.
Is AVSToDVD using MediaInfo to determine the frame rate? It's just that when it's VFR MediaInfo can confabulate all sorts of "real" frame rates.

The same MKV muxed with track statistic tags:
Frame rate mode : Variable
Frame rate : 52.594 FPS

Without them:
Frame rate mode : Constant
Frame rate : 25.000 FPS

That may be just what it displays in the GUI rather than what it tells other software, although as a test I indexed both MKVs with MeGUI/FFMS2.
For the first one MeGUI added fpsnum=26297 & fpsden=500 to the script. For the second it added fpsnum=25 & fpsden=1

I think I mentioned to Zathor at one stage that MeGUI should only add standard NTSC or PAL frame rate conversions because I don't think anything else makes sense. For example, if MediaInfo says the frame rate is 28.45fps, then it's a pretty good bet the video started life as 29.970fps, only duplicate frames were removed. The main point of the VFR to CFR conversion would be to put them back. Encoding at 28.45fps would make a mess, dropping frames that don't need dropping etc. The same probably applies to VFR sources from smart phones etc. If they're an odd average frame rate, you'd want to convert them "up" to the next highest standard NTSC or PAL frame rate. And for AVSToDVD, you're limited to NTSC or PAL frame rates anyway (I assume).

[manolito]

The source is Selur's uploaded file from this post:
https://forum.doom9.org/showthread.php?p=1845939#post1845939

It is an XviD AVI, MediaInfo says that it is 29.97 progressive. It is CFR (I believe that XviD does not support VFR).

In ffms2 the fpsnum/fpsden parameters are used to guarantee CFR output (just like "Convertfps=true" in DirectShowSource). DSS2Mod will always output CFR. I do need CFR output because the DVD specs require it.

If the source was VFR, it'd be decoded at the average frame rate, the audio would no doubt go out of sync


I strongly question this. Decoding of VFR sources at a different average frame rate may certainly ruin the video quality, but it will NOT alter the duration. So there should be no audio sync problems whatsoever.


Cheers
manolito

[qyot27]

That's partially why I said that it may even be down to a quirk of AVI. Since AVI has to adhere to a constant frame-rate anyway*, fpsnum/den conversions are probably not a good idea - actually, short of it being a format with no available VFW decoder, like ffvhuff or newer revisions of ffv1, I wouldn't use FFMS2 to handle AVI files at all. MKV and other containers don't have that restriction, or are more or less assumed to be natively VFR, even if the content's framerate doesn't vary.

*because the only real way to force VFR into AVI is by using dummy frames, which still means that the source is CFR - just at a higher framerate with the lower rate parts padded up to the higher rate of the rest of the content (or both/all of the parts padded out to the lowest common denominator).

So the way this might tie back to the previous fpsnum/den situation is that AVI parsing could be a lot more fragile in terms of what framerates it allows. If the fps is already stored as 30000/1001, it may be okay. But if it was created with a float 29.97 or an integer 2997/100 rate like was mentioned as working, forcing 30000/1001 could be all it takes to trip the problem. And that could be in the AviSynth plugin logic (but again, it would have to exist in both the C++ and C plugins, and since they're separate code it's either really old, or the problem is actually in libavformat and can only express itself in a constant framerate environment like AviSynth).

For the record, 3000/101 seems to work too, even though the end result of that is 29.7029703. It makes me wonder if AVI can even store framerates as integers, or whether it's always converted to float beforehand.

[hello_hello]

The source is Selur's uploaded file from this post:
https://forum.doom9.org/showthread.php?p=1845939#post1845939

It is an XviD AVI, MediaInfo says that it is 29.97 progressive. It is CFR (I believe that XviD does not support VFR).

Neither does the AVI container, although AVI supports some sort of "null frames" which means it can vary.

In ffms2 the fpsnum/fpsden parameters are used to guarantee CFR output (just like "Convertfps=true" in DirectShowSource). DSS2Mod will always output CFR. I do need CFR output because the DVD specs require it.

Decoders always output a constant frame rate, but which frame rate....
AVISource always outputs a CFR and it duplicates frames to replace "null frames". DirectShowSource with ConvertFPS=true does the same.
AVISource is clever enough to output the correct frame rate without being told. For a 25fps AVI with a few null frames it'll output 25fps. By default, ffms2 and LSmash would treat it as VFR and output the average frame rate, which might be 24.992 without the frame rate conversion. They spread the frames out over the total duration.

I strongly question this. Decoding of VFR sources at a different average frame rate may certainly ruin the video quality, but it will NOT alter the duration. So there should be no audio sync problems whatsoever.

Edit: Sorry, when I referred to the audio going out of sync I meant when the video is decoded at the average frame rate without any conversion. I did mention the audio sync when converting to the average frame rate though. I guess I wasn't thinking, but I made up for it by doing a nueron2 and editing my post to remove the evidence :). And then I misunderstood you initially, so explanation deleted.... I'm over-tired.

I still think my logic is sound when it comes to converting to a constant frame rate. Converting to a standard frame rate seems logical to me. ie The appropriate peak frame rate for the source and PAL or NTSC. Unless you're happy to drop frames.

These days I've embraced VFR encoding. Not that I do a lot of it as there's not a huge need for it in PAL-Land, but I doubt I'd ever convert a hybrid NTSC DVD to a constant frame rate again. I only encode with x264 though, and obviously for DVDs you don't have the same luxury.

[hello_hello]

The reason that I could not reproduce the issue was that AVStoDVD created a script which had "fpsnum=2997, fpsden=100" in it. With these values the output is flawless. Only after I changed the script to use "fpsnum=30000, fpsden=1001" I was able to reproduce the issues.

It seems to be as I thought it might. If you remux the AVI as an MKV the problem goes away. Well... I only checked two of the problem frames, but they were fine. Not that MKVs are immune, but in this case it seems to work.

Mind you I don't know why AVStoDVD was using "fpsnum=2997, fpsden=100" because MeGUI picked "fpsnum=30000, fpsden=1001".
Although as I type, there's a memory returning.... something about some players crashing when 30000/1001 is used....
I guess AVStoDVD is playing it safe.

http://www.avisynth.org.ru/docs/english/corefilters/fps.htm
In v2.57, the behaviour with respect to the framerate is a bit changed. The main issue is that users are allowed to specify the framerate as float, but the NTSC (FILM and Video) and PAL standards require ratios as framerate. Besides this AviSynth exports the framerate as a ratio, so when specifying a float, it will be converted to a ratio. The ratios of the standards are given by 24000/1001 for 23.976 (FILM) and 30000/1001 for 29.97 (Video). When specifying these floats, they are exported by AviSynth as ratios, but not as the standard ratios. One of the reasons for this is, that those floats are approximations (remember that 24000/1001 = 23.9760239760...), so how should AviSynth know how to choose the correct ratio? In order to overcome this issue, the user can use AssumeFPS(24000,1001) or simply AssumeFPS("ntsc_film").
Another problem is that the converted floats were (in v2.56 and older) exported with 64 bit precision, resulting in very large numerators and denominators, making some players crash. To overcome this, a smart float-ratio is added internally, and the framerates are approximated accurately by ratios of small numbers. For example, AssumeFPS(23.976) is converted to AssumeFPS(2997,125) as can be checked with Info.

[manolito]

OK, I see your points, but for me repacking source files depending on their formats is not an option. I mainly use AVStoDVD for DVD conversions, and the whole point of using such a software is to have solutions which work each and every time no matter what the peculiarities of the sources are...

I think that the best workaround when using ffms2 is to ditch all fpsnum/fpsden parameters from the ffvideosource call and instead follow the ffvideosource call with a ChangeFPS command. I just posted a request for the author to change the handling of ffms2 accordingly.


Cheers
manolito

[hello_hello]

I think that the best workaround when using ffms2 is to ditch all fpsnum/fpsden parameters from the ffvideosource call and instead follow the ffvideosource call with a ChangeFPS command. I just posted a request for the author to change the handling of ffms2 accordingly.

But if the source is VFR, ditching the conversion can potentially cause audio sync issues and ChangeFPS can't help.
That was why I requested MeGUI add it to scripts originally, as then it can convert VFR sources, even if not in an ideal way, but it turned out to be a bad idea due to the frame dropping/duplication issue.

If you use ChangeFPS instead you'd have to be converting from one constant frame rate to another, which is probably what happens most of the time anyway. Or maybe a checkbox might be useful. I assume if AVStoDVD is converting the frame rate it'd be converting to a DVD compliant one, but an ability to tell it the source is CFR or VFR would allow AVStoDVD to use fpsnum/fpsden or ChangeFPS as appropriate.

[manolito]

but an ability to tell it the source is CFR or VFR would allow AVStoDVD to use fpsnum/fpsden or ChangeFPS as appropriate.

Sorry but I do not really understand...

Isn't using fpsnum/fpsden exactly the same thing as using ChangeFPS? According to the ffms2 docs using fpsnum/fpsden forces the specified output fps by dropping or repeating frames. ChangeFPS does exactly the same thing. Where is the difference?

The whole point IMO is to make sure that the output is CFR no matter if the source is VFR or CFR. And if the source already is CFR then it would not matter, using fpsnum/fpsden or ChangeFPS would not do anything in this case.


Cheers
manolito

[hello_hello]

Without fpsnum/fpsden, the frames are just output at the average frame rate. There's no frames added or deleted, they're just spread out evenly. When fpsnum/fpsden is specified the video is converted to the specified frame rate, even if it's the average frame rate, adding and/or deleting them as required to maintain audio sync.
Consider a source that's 23.976 for the first half and 29.97 for the second. You could add frames to the first half or delete frames from the second half or some combination of the two, but you can't just spread the existing frames out evenly.

It makes sense because you don't want FFMS to add or remove frames unless it's specifically told to, or at least without a method to stop it from doing so, otherwise it'd be really hard to encode VFR as VFR, where you want the existing frames with no additions or deletions.

That's what I was getting at before. If the video is VFR it has to be decoded to a specified frame rate. If fpsnum/fpsden isn't specified it's not. ChangeFPS has no idea how the frames are spread out in the source.

Of course if you're happy to assume all your sources are CFR, then dropping fpsnum/fpsden is a good idea, although it mostly works fine when it's converting real VFR to CFR, as opposed to converting 29.970 to 30000/1001, which requires an extra frame roughly every ten hours, so AssumeFPS would make more sense for that sort of thing.

[manolito]

ChangeFPS has no idea how the frames are spread out in the source.

Why would it need to know this? And do you really believe that when using fpsnum/fpsden the plugin does a global analysis of the source before decoding in order to retrieve such knowlege about how frames are spread out in the source? I do not think so...

From the DirectShowSource docs:
convertfps = false (in v2.56): When setting it to true, it turns variable framerate video (vfr) into constant framerate video (cfr) by duplicating or skipping frames.

From the ffms2 docs:
This may naturally cause `FFVideoSource` to drop or duplicate frames to achieve the desired frame rate,

From qyot27:
it doesn't occur when leaving it as just the filename, even if you then proceed to use ChangeFPS() to do the same sort of framerate conversion.

I still believe that avoiding fpsnum/fpsden and instead use ChangeFPS after the ffvideosource call will render the same result (or amost the same). In the end we should leave the answer to folks like qyot27 or Myrsloik who understand the inner workings of the plugin... :p


Generally the AVStoDVD concept for creating DVD output is to always treat the source as VFR (use DSS2Mod, use convertfps=true for DirectShowSource and use fpsnum/fpsden for ffms2). This will ensure that VFR sources are handled correctly, and for CFR sources no harm will be done. Using AssumeFPS is not an option since this will alter the video duration causing audio sync errors.


I am more interested to find out which version of ffms2 was the first to show these issues. After finding that the standard C++ version 2.17 worked fine I just tried the C-plugin version 2.17. And this one also showed no problems.

Unfortunately I was unable to find C-plugin versions between 2.17 and 2.23 (GitHub does not have them, and the MediaFire links from qyot27 were expired). So all I know is that C-plugin version 2.17 works while version 2.23 (r1140+101) does not work. The issue must have been introduced somewhere between these two versions.


Cheers
manolito

[hello_hello]

Why would it need to know this? And do you really believe that when using fpsnum/fpsden the plugin does a global analysis of the source before decoding in order to retrieve such knowlege about how frames are spread out in the source? I do not think so...

If that's not the exact definition of indexing it's got to be a very close relative. DSS would output the specified constant frame rate, no doubt adding or dropping them as required based on each incoming frame duration. DSS would have to do it on the fly. FFMS2 runs an indexing pass first.
All you need is the frame count and duration and some easy maths to output the average frame rate, but I can't imagine how FFMS2 could convert VFR to CFR by duplicating/dropping frames if it doesn't know the duration of each frame in a VFR source, or in other words the speed at which the source frames are zipping by at any moment in time. ChangeFPS would just see the existing frames at "some" constant frame rate because that's what Avisynth needs, so it can't convert VFR that's been made constant simply by averaging out the frame durations, and if you tell FFMS2 to drop or duplicate frames to convert to a constant frame rate there's no need to ConvertFPS again.

It'd be impossible to re-encode VFR sources as VFR unless only the original frames are decoded. No more, no less.
The standard method is to extract the timecodes file, encode without any frame rate conversion, the script frame rate is irrelevant but the average frame rate seems a popular choice as it doesn't change the duration, give x264 the timecodes file so it knows where the frames really belong time-wise, or use the timecodes when muxing to distribute the frames in the same manner as the source.

Here's what the help file I have says (FFMS2 version 2.23.1)

##### int fpsnum = -1, int fpsden = 1
Controls the framerate of the output; used for VFR to CFR conversions.
If `fpsnum` is less than or equal to zero (the default), the output will contain the same frames that the input did, and the frame rate reported to Avisynth will be set based on the input clip's average frame duration.
If `fpsnum` is greater than zero, `FFVideoSource` will force a constant frame rate, expressed as a rational number where `fpsnum` is the numerator and `fpsden` is the denominator.
This may naturally cause `FFVideoSource` to drop or duplicate frames to achieve the desired frame rate, and the output is not guaranteed to have the same number of frames that the input did.

I just checked a VFR MKV with a section at the beginning that's 25fps while the bulk of it is 59.94fps. The 25fps section has 5877 frames and ends at 3min 55 seconds. 5877 frames at a CFR of 25fps does work out to 3min 55sec. Without FFMS converting the frame rate it still had 5877 frames but in MeGUI's preview it finished at 1min 21sec and wouldn't have looked out of place as a closing for The Benny Hill Show.
With FFMS2 converting to some constant frame rate the frame count increases or decreases accordingly but the scene ends at 3min 55sec.

That's part of the issue people seem to have when ripping DVDs with MakeMKV and then indexing them with FFMS2. Unlike DGIndex, FFMS2 ignores repeat flags by default, which means for hybrid NTSC it just outputs the existing frames, and I'm pretty sure it does so at the average frame rate.
The help file says it assumes a CFR when the RFF option is enabled, although I'd imagine that's not living too dangerously given I assume the RF flags ensure the frame rate is constant anyway.

[manolito]

If that's not the exact definition of indexing it's got to be a very close relative. DSS would output the specified constant frame rate, no doubt adding or dropping them as required based on each incoming frame duration. DSS would have to do it on the fly. FFMS2 runs an indexing pass first.

Alright, you score for ffms2, but for DSS2Mod and DSS there should be no difference when using ChangeFPS instead of "convertfps=true". In the couple of tests I did I could not see any problems with this approach.


We are dealing with two different problems here:
1. How to treat real VFR sources when CFR output is required?
2. How to deal with the regression in ffms2 where fpsnum/fpsden is broken?


For the first problem I had a dispute with Zenitram some time ago when he stopped displaying the frame rate and duration of VFR MKV files. This post by Foxyshadis
https://forum.doom9.org/showthread.php?p=1739508#post1739508
saved my life more than once. Easy to use and good quality output.

For the second problem I would really like to see current ffms2 versions fixed. It did work in version 2.17 for both the standard and the C-plugin, so it must be possible to find the error and fix it. If this is not going to happen then I will need to employ the workaround using ChangeFPS instead of fpsnum/fpsden. I am a result oriented person, I will use "whatever works", but the method has to be universal, it has to work for all kinds of sources without deeply analyzing them.


Cheers
manolito

[qyot27]

To be fair, the situation where I said ChangeFPS would be preferable is not VFR-to-CFR, it was AVI files like that sample specifically (I also don't really know how the fps conversions in FFMS2 perform their logic, it was a general statement about framerate).

The process of using AssumeFPS when the framerate is close enough to standard* is the most correct way, but for safety's sake I always go with sync_audio=true and then run SSRC or ResampleAudio so that the audio frequency gets fixed back to a standard 44100 or 48000 Hz, depending on what exactly it was that I was encoding for. But if you're trying to preserve sync whilst absolutely not touching the audio, the only solution is to drop or insert duplicate frames to do it.

*Even though the minor framerate differences won't result in any audible pitch shift; the shift is there, but human ears really can't tell if 44100 was slowed down to, say, 44030 before getting re-normalized to 44100; not unless you do it dozens of times in a row. Thousandths or tens of thousandths of a frame, here: I usually would find myself having to do it to correct whole 30->29.97 or 24->23.976, but we're talking about stuff like 2997/100->30000/1001; 24/30->NTSC Film/Video is a 1.001% shift, and I couldn't hear the difference - simplified 23.976/29.97->NTSC Film/Video is a difference of 0.0001%.


I kind of feel like we're getting a bit into the weeds here, though.

[hello_hello]

When the source frame rate is already DVD complaint, AVStoDVD should be aware of it and not add any ffms2 frame rate conversion to the script. Ideally it should have no effect if it's added, but obviously that's not what happens all the time.

For VFR frame rates that aren't in the DVD compliance ballpark, and when DSS isn't an option, there's really no other way than to convert it to a CFR as it's decoded. Lsmash has the same conversion options as ffms2, so maybe if your source is VFR and there's incorrectly dropped/duplicated frames in the output, an ability to pick the indexer might be enough. I've definitely seen Lsmash go nuts in the past and duplicate/drop frames it shouldn't (ie when the specified frame rate is the same as the source frame rate) so I'm not sure it's an ffms2 specific bug, but fortunately I don't think I've had a source that neither indexers could convert successfully. Maybe it's just luck.

Still feeling obligated to argue about the ability of ChangeFPS to convert a VFR source to CFR correctly.....

When the VFR source is decoded without ffms2 converting it to a constant frame rate, FFMS2 outputs the average frame rate.
The average frame rate is pretty much the VFR equivalent of AssumeFPS, only because it's the average frame rate it can do the assuming without changing the duration. Still, it'll effectively speed up some sections and slow down others, just like AssumeFPS() changes the speed of constant frame rate video.

If you happen to get lucky and you can decode a VFR source at the average frame rate without effecting the audio sync, then of course you can convert that to a different frame rate with ConvertFPS and still not effect the audio sync any further, but it doesn't mean it converted VFR to CFR. When the frame rate varies enough or even early enough that the audio slowly loses sync when decoding at average frame rate, ConvertFPS won't fix it. It'll output the frame rate you tell it to, but the audio sync won't change.

[qyot27]

Also, I don't know why I didn't do this earlier, but there does appear to be something wonky with that sample file. MKVToolNix accepts it as input, but ffmpeg won't remux it into MKV, because it complains about the file's timestamps (it will allow it to mux into another AVI, but still throws a non-fatal warning about the timestamps in the file):
[avi @ 00000226993850c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

And from running a grep on the FFMS2 source code in the avisynth, avisynth_c, and vapoursynth directories, I'm starting to suspect that this is entirely external to FFMS2, and that the issue is partially in libavformat and partially in AviSynth itself. The fps-related hits in all three of those directories were all pretty similar, and nothing seemed to stick out as being really different, which points to this disconnect between AviSynth and VapourSynth probably being more about how those programs themselves interact with FFMS2 than something entirely in FFMS2's hands. I haven't tested VapourSynth, though (mostly because I already installed Python 3.7 for all my Python things, and don't want to also have to 3.6 on here too; I'll wait until the Windows builds are built against 3.7).

[manolito]

First of all this issue applies to the standard plugin as well as to the C-plugin, I'll post it here because I usually use the C-plugin.

I've been thinking for some time that aac_latm is an evil format, but since it has become one of the two audio standards in German DVB-T2 broadcasts I have no choice but dealing with it.

I uploaded a short sample file here:
https://www51.zippyshare.com/v/KY10rZkC/file.html

This format is used by half of the public stations (the other half use E-AC3). It is not this specific file, it's all captured files with this audio format. I also doubt that my receiver has anything to do with it (a Xoro with the latest firmware). The file plays without errors in all my software players, MediaInfo has no problems with it. And using LAVFilters with DSS2Mod also handles it nicely.

Only ffaudiosource has issues, and it applies to all current versions up to the latest 2.23.1 of the standard plugin and r1315 of the C-plugin. Tested on 2 different computers under WinXP and Win7-64bit.

This is the problem:
When using ffaudiosource("filename", track=-1) I get the error "No audio track found". When I use "track=1" or "track=2" the error is "Track has no audio samples". (I did verify that the 2 audio tracks are track #1 and track #2).

First I thought that ffms2 was just unable to decode this audio format, but this is not true. When I use the shortcut "ffmpegsource2" instead of "ffaudiosource" then the audio is decoded properly. This is true for the C-plugin where the shortcut comes from "ffms2.avsi" as well as for the current standard-plugin where the shortcut is integrated into the DLL.

Can anyone explain this weird behavior? I certainly do not have a clue... :devil:


Cheers
manolito

[qyot27]

FFMS2 C-plugin r1327+120 (https://forum.doom9.org/showthread.php?p=1829061#post1829061) (directs to first post in this thread)

The reason it took me so long before posting another build was that I was doing a pretty deep refactoring of the FFmpeg and mpv build guide I maintain, to switch all of the entries to out-of-tree builds (unless that was impossible; I think a grand total of 5 pieces couldn't be, out of 74-ish?). The result is that the 32-bit and 64-bit instructions in the guide can now use a single download of the source code, which saves download time/bandwidth, and it saves storage space.

Then, because I'm apparently a glutton for punishment, I'd gotten so fed up with having to build the 32-bit environment under a VM that I decided to try doing a multilib environment again (thus saving time at the beginning, because only one cross environment needs to be built, as well as being able to avoid all the nasty performance hits of a VM). This was...significantly trickier, as it exposed some pretty deep problems with how some of the pieces in FFmpeg's dependency chain have their build systems configured - and most of it caused headaches for the 32-bit parts. I had to resort to disabling assembly code on some of those 32-bit parts because their build systems won't tell nasm/yasm/gcc to build the asm for i686 in a multilib environment, so it tries building it as amd64 instead, and that causes it to fail to link at the end. Hopefully there's actually a way to force it without just having to wait on those projects to fix their autoconf setups (because it seemed to hit autoconf projects the most).

And then, because the Universe hates me, the computer I had been using as my main setup for the last couple years (a Quantum Byte mini-PC I bought in 2015) ended up getting fried due to a faulty power strip when I was just under halfway done with the move to multilib. So I had to order a new mini-PC (a Byte3), and go through setting up both the OSes *and* then get back to and finish that move to multilib in the guide. At least the new one runs 64-bit Windows 10 and I immediately migrated the Windows install on it to an SSD, so that crap shouldn't happen again (and enough free space to allow for installing MSVC, so I can get back to helping with AviSynth+ development...again, hopefully).


Only ffaudiosource has issues, and it applies to all current versions up to the latest 2.23.1 of the standard plugin and r1315 of the C-plugin. Tested on 2 different computers under WinXP and Win7-64bit.

This is the problem:
When using ffaudiosource("filename", track=-1) I get the error "No audio track found". When I use "track=1" or "track=2" the error is "Track has no audio samples". (I did verify that the 2 audio tracks are track #1 and track #2).

First I thought that ffms2 was just unable to decode this audio format, but this is not true. When I use the shortcut "ffmpegsource2" instead of "ffaudiosource" then the audio is decoded properly. This is true for the C-plugin where the shortcut comes from "ffms2.avsi" as well as for the current standard-plugin where the shortcut is integrated into the DLL.
It works with the build I just posted...sort of. I always index with ffmsindex first, and using the normal ffmsindex -t -1 ARD.mts to index the video and audio fails with an error message about the audio track.

Referring to the documentation, I set the errorhandling value to 2:
ffmsindex -t -1 -s 2 ARD.mts
This allows ffmsindex to complete the indexing. -t -1 indexes all the tracks, but the error specifically occurs on track 2 - setting -t 1 (for just track 1) without -s 2, it can index successfully. The resulting FFAudioSource script with track=1 plays back.

It also - probably incorrectly - will play back the same track if you select track=2 with that first -t -1 -s 2 script.

But -t 2 without -s 2 errors out; -t 2 -s 2 does allow indexing to complete, but FFAudioSource complains about the track not containing samples.

[manolito]

Thanks qyot27 for looking into this issue... :thanks:

This also explains why using the shortcut "ffmpegsource2" works (because indexing will be done before the ffaudiosource call). Of course using this workaround is not really satisfactory if the second track cannot be decoded.


Cheers
manolito

[FranceBB]

Thank you for the new build! :D
Still, adding enable10bithack would be really useful for our poor Avisynth (not Avisynth+) users. :')

[StvG]

Comparing 2.23.1 and 1327+120 in AvsPmod (Avisynth+ r2728), the outputs are different. Is it a bug or because of newer ffmpeg?
Comparing 2.23.1 and 1140+101 in AvsPmod (Avisynth+ r2728), the outputs are identical.
Checked with 8/10bit progressive video (23.976/24.000).
Simple script:FFVideoSource("input.mkv")

[qyot27]

You're going to have to be more specific. Are you sure that's not just the fact that newer versions of the C plugin don't automatically downsample to 8bit? Did you compare 2.23.1 and the latest mainline FFMS2 build (https://forum.doom9.org/showthread.php?t=174469)?

[StvG]

What I mean - https://imgbox.com/g/JWldJPTvIb

Script:FFVideoSource("HEVC.10bit.mkv") #2.23.1
ConvertBits(8)

LoadCPlugin("ffms2_r1140+101-avs+vsp\64-bit\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="r1140.ffindex")
ConvertBits(8)

LoadCPlugin("ffms2_r1327+120-avs+vsp\64bit\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="r1327.ffindex")
ConvertBits(8)

LoadPlugin("ffms2000-test8\64\ffms2.dll")
FFVideoSource("HEVC.10bit.mkv",cachefile="test8.ffindex")
ConvertBits(8)


It happens with AVC 8bit videos too.
Also some frames of the video could be ok, other frames could be off.

[qyot27]

Okay, it looks like there's a frame skew compared to the other builds.

FFMS2000 test8 is from March 3rd, and there are only a couple of extant C-plugin builds from after that point (including r1327+120, which is from July 10th). Try with the 'lastxp' build in the first post (r1315+119). Since r1315+119 is from April 25th, if it doesn't exhibit this, it was introduced *after* April. If it does show the issue, it happened sometime between March 3rd and April 25th.

There were multiple commits that occurred on FFMS2's master branch during mid-March dealing with H.264 PAFF and interlaced HEVC, frame delays, and such. This is stuff that could simply make the frame counts different from then on, and would be visible on a hypothetical FFMS2000 test9 or the next point release. It doesn't necessarily mean it's a bug, just a change in behavior.

[StvG]

r1315+119 has the same behavior as r1327+120. http://imgbox.com/g/DAz5CXMPN4 - AVC 8bit video, some of the frames are ok, other frames are off.

[qyot27]

Then it occurred somewhere in the five frame-related commits in March*. All of those commits relate to how frames are handled in terms of delay, half-frames/interlacing, discarded frames, and so on. Considering that, is the output from newer FFMS2 builds just different or is playback affected negatively?

*https://github.com/FFMS/ffms2/commit/3d5867dc63dc69b98c3940860d2d32ed87d7c9ee
https://github.com/FFMS/ffms2/commit/13d9f01f5af62a30fe162b0068cb3c18d4db401a
https://github.com/FFMS/ffms2/commit/60860f7b3d53884acb79ceef30ea22ed534d01ba
https://github.com/FFMS/ffms2/commit/0562ba3c64a8e36cfe5950b41b0cb6eaf5f5a81a
https://github.com/FFMS/ffms2/commit/c870059ddb2131e6bf20c89b164aa69e5531c0e8

[StvG]

When the frames are off, usually r1327 shows 2-3 frames ahead compared to 2.23.1 and the other versions. So playback seems ok. Both the first and the last frame always are matching.
Seems some I frames decoded by 2.23.1 are decoded as B frames by r1327 and those are places where the mismatch is starting until the next I frame, which could be decoded either as I frame (frames are matching again) or as B frame (the mismatch is continuing).
Also tested some files encoded with x264 and always everything seems ok with these files.

[sneaker_ger]

Can you upload a sample?

[StvG]

Sample (https://a.doko.moe/gpkoqx.mkv)

[sneaker_ger]

The sample is not cut cleanly (open GOP not cut on IDR). There are 3 stray bframes left in the beginning.

[StvG]

The sample is cut with mkvtoolnix (https://gitlab.com/mbunkus/mkvtoolnix/wikis/Splitting-imprecise). 2.23.1 shows 4 I frames in the beginning.

[sneaker_ger]

Yes, but there aren't 4 I frames. 1 I and 3 B frames. It seems to freeze the I frame for 4 frames total. Now, the 2 strategies to deal with this broken file:
1.) freeze first I frame/recovery point for 4 frames replacing the 3 stray b frames or
2.) skipping/deleting the 3 stray b frames
are both valid in itself. You have to do something.

The questions would be:
A.) is av sync maintained
B.) what happens at the end of the clip (both show a length of 501 frames)

[StvG]

The sample is a part of randomly picked trailer just for testing.

Another sample (https://a.doko.moe/ypqmje.mkv) (should be fine now). When you go forward and backward through the video picking random frames, you will see at some moment that frames by 2.23.1 and r1327 are off. Then if you go backward to find the last I frame (let's call X the frame number) showed by 2.23.1, you will see that X frame by r1327 is B. If you refresh the video by r1327, the frame is still B type. Then if you go backward again to a place where frames by 2.23.1 and r1327 are the same and go forward frame by frame, X frame by r1327 will be now I frame and there will no further mismatch until you start picking randomly frames again. I encoded (x264) this sample with both 2.23.1 and r1327, and compared the encoded files - no matter how long I'm randomly picking forward/backward frames, the frames by 2.23.1 and r1327 are always matching.

[sneaker_ger]

Thx. I see both problems with r1327+120.

[StvG]

Btw FFMS2000 test8 (https://forum.doom9.org/showthread.php?t=174469) has similar behavior in a lighter form.

It seems the ffmpeg version also has influence. Github tag 2.23 built with the latest ffmpeg master branch has the same behavior as FFMS2000 test 8 - both have a bit different behavior than 2.23.1 release posted here (https://github.com/FFMS/ffms2/releases).
Tag 2.23 built with the latest release/3.4 ffmpeg branch has the same behavior as 2.23.1 github release.
It seems FFMS2000 test 8 behavior starts from this commit (https://github.com/FFMS/ffms2/commit/f3fb0e0c8910321bfef542836e4391e57ad3c6fb), because version compiled from this tree (https://github.com/FFMS/ffms2/tree/5eb741e45ab333bfb9799c243bf1156d0719a843) with ffmpeg release/3.4 has the same behavior as the github release.

P.S. This problem obviously is not specifically related to the c-plugin, so a staff could move the whole conversion to the appropriate thread?

Edit: Reverting back these commits*, build from master branch with the latest ffmpeg has the same behavior as 2.23.1.

*https://github.com/FFMS/ffms2/commit/f3fb0e0c8910321bfef542836e4391e57ad3c6fb
https://github.com/FFMS/ffms2/commit/6c8bc8cbca0060a454507cc4ac2f582183d3af6d
https://github.com/FFMS/ffms2/commit/13d9f01f5af62a30fe162b0068cb3c18d4db401a

[Selur]

A fresh build to support av1 would be great. :)
Thanks!

[qyot27]

It was about time for a new build anyway.

FFMS2 C-plugin r1327+119 (https://forum.doom9.org/showthread.php?p=1829061#post1829061)
(the backtrack of the C-plugin rev # is nothing to worry about, just a mix-up with the working trees on my local setup - it's been resolved on Github)

I tested it against the AV1 samples on https://www.elecard.com/videos, it seems to work.

[Selur]

Thanks!

[amayra]

Thank you qyot27

[FranceBB]

FFMS2 C-plugin r1315+119

I get an error "Unhandled C++ exception" if I try to open a .thd audio file.
It's a TrueHD Dolby Atmos 7.1 audio file of about 4.7 GB.
FFAudioSource starts indexing, it reaches about 512 MB of RAM and then it crashes.
Can you try to index a similar TrueHD Dolby Atmos 7.1 audio file and reproduce the issue?
It happens on the old ffms2 version as well (I can't test ffms2000 with XP).
If you want I can upload the file.

MediaInfo

Audio
Format : MLP FBA 16-ch
Format/Info : Meridian Lossless Packing FBA with 16-channel presentation
Commercial name : Dolby TrueHD with Dolby Atmos
Codec ID : A_TRUEHD
Duration : 2 h 29 min
Bit rate mode : Variable
Bit rate : 4 534 kb/s
Maximum bit rate : 6 930 kb/s
Channel(s) : 8 channels
Channel layout : L R C LFE Ls Rs Lb Rb
Sampling rate : 48.0 kHz
Frame rate : 1 200.000 FPS (40 SPF)
Compression mode : Lossless
Stream size : 4.73 GiB (9%)
Title : TrueHD Atmos 7.1
Language : English
Default : Yes
Forced : No
Number of dynamic objects : 13
Bed channel count : 1 channel
Bed channel configuration : LFE

[qyot27]

I've added it to the first post, but I finally opted to enable the issues tracker on my personal FFMS2 repo on Github to track C plugin-related issues:
https://github.com/qyot27/ffms2_cplugin/issues

[qyot27]

New build. (https://forum.doom9.org/showthread.php?p=1829061#post1829061)

The AV1 decoder in this build is libdav1d. FFmpeg is four months newer.

[Taurus]

@ qyot27
A big Thank You for maintaining ffmpegsource.
Really appreciated :thanks::thanks::thanks:

[StainlessS]

@ qyot27
A big Thank You for maintaining ffmpegsource.
Really appreciated :thanks::thanks::thanks:

+ 1 on that, thanx muchly. :)

[StvG]

Thanks.

I compared the latest c-plugin version you posted vs ffms2 (non c-plugin) with QTGMC(sharpness=*) in AvsPmod. C-plugin eats ~4.4GB RAM while the othe build eats ~1.3GB RAM.
* - when values from 0.1 to 1.6 are tested
C-plugin continues to eat RAM for every new tested option until there is no free RAM. The non c-plugin doesn't have this behavior. I had crashes with c-plugin while using non c-plugin runs without problems in the same scenario.
Can take a look for fix/solution?

[Groucho2004]

I compared the latest c-plugin version you posted vs ffms2 (non c-plugin) with QTGMC(sharpness=*) in AvsPmod. C-plugin eats ~4.4GB RAM while the othe build eats ~1.3GB RAM.
* - when values from 0.1 to 1.6 are tested
C-plugin continues to eat RAM for every new tested option until there is no free RAM. The non c-plugin doesn't have this behavior. I had crashes with c-plugin while using non c-plugin runs without problems in the same scenario.
Can take a look for fix/solution?
Have you tried VirtualDub2 and/or AVSMeter to verify this? Maybe the problem lies with AVSPmod.

[qyot27]

The problem lies with the client program re-loading the script over and over again without necessarily releasing the AviSynth script environment completely, which also unloads the C-plugin and flushes the memory. It's been an 'issue'* for years and trying to drill down to exactly where it lies is maddening.

*I mean, not for users of programs like x264 or FFmpeg, since those only load and unload the script environment once per program instance. VirtualDub can exhibit it eventually if you open the script and then repeatedly hit F2. Every time, it tacks an additional 70MB onto VirtualDub's memory allocation.

[manolito]

A forum member at VideoHelp uploaded this 12-bit HEVC sample which causes problems with the current ffms2 C_Plugins:
https://forum.videohelp.com/attachments/48349-1552354471/hevc_test_12bit.mp4

The latest C-Plugin version which can handle this file is FFMS2 C-plugin 1140+101 from 2016. All later versions either cannot find the function "ffaudiosource", or they quit with an access violation.

Repacking the file from MP4 to MKV does not help either. The file itself looks healthy, plays fine in all my media players, and MKVToolNix does not complain when remuxing it to MKV.

Any thoughts? I always thought that the C-Plugin v.1140 is the most reliable one and I always use it, but it feels weird to me that for decoding a "modern" 12-bit source the old version handles it gracefully while all the newer version fail.

Cheers
manolito

[qyot27]

I can't reproduce this. It works fine here with the most recent build I've posted (from 2019-01-20 / r1334+120).

Those errors sound like something else is wrong in the AviSynth environment, though.

[filler56789]

FWIW, on my PC the latest FFMS2 C-plugin returns this message:

"FFVideoSource: no suitable output format found"

— whereas the DLL compiled on 2018/January/09 loads that video but makes VirtualDub crash when I click the play button.

[manolito]

"FFVideoSource: no suitable output format found"


Yes, that's the error I get when using te latest C-Plugin version, too. Win7 -64, everything else 32-bit.

[filler56789]

UPDATE:

adding colorspace="YV12" to FFVideoSource() is the solution to the PEBKAC :o situation.

Another new build (https://forum.doom9.org/showthread.php?p=1829061#post1829061); this time, using the C plugin under AviSynth 2.6 shouldn't error out with access violations if given high bit depth video. It will spit back the 'FFVideoSource: No suitable output format found' error since it won't do automatic downsampling anymore. The assumption, given high bit depth video, is outputting the same pix_fmt unless you use ConvertTo later in the script. The colorspace= parameter should work again, though, so you can force 8-bit output when you know you need it.

[manolito]

Thanks, this explains it...

But for software like AVStoDVD where the user cannot alter the FFVideoSource parameter to add a colorspace value this means that for high bit depth video the current ffms2 C-Plugins will not work. I will recomend to all A2D users to revert back to ffms2_r1140+101-avs+vsp.7z to avoid these issues.

[qyot27]

New build. (https://forum.doom9.org/showthread.php?p=1829061#post1829061)

Includes the AV1 parsing fix patches, along with two months' newer versions of FFmpeg and libdav1d.

[manolito]

Thanks for the new build r1340... :thanks:

I would like to make this fool proof for AVStoDVD, this means that I need to add the colorspace="YV12" param to the AVS file created by AVStoDVD somehow (not too hard, I already do intercept the encoder call).

The question is if this is really worth the trouble. The older version r1140 works nicely so far, does the new version r1340 offer any real world advantages? Does it support source formats which the older version doesn't? Or does it fix some serious bugs?

Cheers
manolito

[qyot27]

Does it support source formats which the older version doesn't?
AV1 is probably the most significant one. (https://en.wikipedia.org/wiki/AV1) r1140+101 was uploaded in October 2016, more than a year before the AV1 spec reached feature freeze, and a year and a half before the initial release of its reference software. Even just newer versions of libaom (the reference software) or libdav1d could be significant on just the topic of AV1; it's a shame that FFMS2 doesn't have something in place to switch among multiple libavcodec decoders (if available), depending on which currently gets better performance. FFmpeg itself didn't support decoding with libaom (http://git.videolan.org/?p=ffmpeg.git;a=commit;h=99cc3cf7a26cd4113e5047034577027d8c6f07a2) until March 2018, and didn't support libdav1d until November 2018 (http://git.videolan.org/?p=ffmpeg.git;a=commit;h=beaa350e24167f4ff31275a6114693f5ce7cd409).

[manolito]

Thanks for this info...

Looks like AV1 might become an important video format in the near future, source filters will need to support it. My efforts to add the colorspace parameter to the AVS script in A2D were only halfway successful. I can manipulate the encoding script, but for the preview a separate script is used which I cannot access. Hopefully MrC will find some time to fix it...


Cheers
manolito

[Ktzf01]

Thank you qyot 27 for these nice ffms2 builds. I find these very helpful. I was wondering if sometime a build with libaom (instead of libdav1d) would be available, just for comparison.

[manolito]

I'm not sure if a different thread for this post would be more appropriate (like the regular ffms2 thread or the AVS+ thread), but I chose this thread anyways because I know who I am talking to...:D

After installing the current AVS+ test build I did a couple of speed benchmarks using different source filters, and like before I stumbled over some odd behavior using ffvideosource in MT mode.

My tests were all 32-bit, latest AVS+ test version, several C-plugin versions of ffms2. I also tried LWLibavVideoSource and DSS2Mod. CPU was an Intel Core i5 3rd generation, I used 4 MT threads. Input was a HD MP4 file, I used AVStoDVD to convert it to DVD. The script was as simple as it gets, just the source filter, YV12 conversion and the resizer.

When I specify ffms2 in AVStoDVD, it generates this call:
Video = FFVideoSource("C:\Download\Test.mp4", track=-1, fpsnum=25, fpsden=1, colorspace="YV12", seekmode=2)

The fpsnum and fpsden params are used to make sure that the source filter issues CFR (DVD cannot handle VFR, and if the source is already CFR then these params should not cause any damage). Seekmode=2 is from a patch I did because the default Seekmode=0 was just too slow for my taste.

These are the speeds I get: (The values in brackets are without MT)
DSS2Mod: 115fps (95fps)
LWLibavVideoSource: 115fps (97fps)
FFVideoSource r1140: 59fps (97fps)
FFVideoSource r1340: 36fps (97fps)

This looks pretty bad for FFVideoSource, and the following fix worked for me:
Video = FFVideoSource("C:\Download\Test.mp4", track=-1, colorspace="YV12", seekmode=2).ChangeFPS(25)
I took away the fpsnum and fpsden params and used ChangeFPS to ensure CFR output. Suddenly the speed jumped up to 117fps.

This weird behavior only happens when MT is used. There must be something in the implementation of fpsnum and fpsden which seems to cause a race condition or so in AVS+ MT mode. Do you have an explanation for this?

And do you think that there are any catches when using ChangeFPS? According to the ffms2 docs all the fpsnum and fpsden params do is inserting or dropping frames to reach the desired CFR fps, and ChangeFPS does exactly the same thing. I have not found any visual problems so far using this workaround.


Cheers
manolito

[qyot27]

The behavior is a little more predictable. It happens with just a plain invocation of FFMS2:
FFVideoSource("test-av1opus.mp4")
This script opening a 1080p30 AV1/Opus test file averages ~260fps on my Apollo Lake Celeron J3455. AviSynth+ r2883, so prior to the recent commits.

Add Prefetch(4), and the average speed drops to ~10fps. Add ChangeFPS(25), it rises back up over 200fps.

But here's the thing: if ChangeFPS(25) is used and Prefetch(4) isn't, the speed is still over 200fps. If, however, ChangeFPS' linear parameter is set to false (the default is true) while using Prefetch, the speed drops to the same low number.

What this is saying, more likely than not, is that it has something to do with frame or memory caching (or in other words, ChangeFPS essentially nullifies the speed drop from FFMS2 being run in an MT script because by default, it's doing linear access). It also isn't isolated to the C plugin; the same sort of thing occurs with the August 11th build of the C++ plugin that HolyWu posted. fpsnum/den actually have nothing to do with it, and can be worked around using RequestLinear() from the TIVTC package:
FFVideoSource("test-av1opus.mp4",fpsnum=25000,fpsden=1000)
RequestLinear()
Prefetch(4)
The result in avsmeter: 226fps.

Beyond that, whether it's FFMS2's core reacting badly to MT or AviSynth+ being too aggressive with FFMS2 specifically, I can't say. Why FFMS' seekmode parameter wasn't working is one question. It would have supposedly forced linear access, but there was no speed up.

With AviSynth+GCC r2831, FFmpeg reports the following (since avsmeter can't run GCC builds):
plain script: 159fps
with Prefetch(4): 32fps
with ChangeFPS(25): 142fps
with ChangeFPS(25) and Prefetch(4): 170fps (!!!)
Now that result is interesting. It's the only one here where activating MT actually resulted in an increased fps, rather than the same fps or significantly lower. Unfortunately, you give up almost all plugin compatibility when using AviSynth+GCC, so it's more for checking whether there's something weird going on with the compilers (and based on this alone, there actually might be, although I wouldn't know what it is exactly).

[manolito]

Thanks for your research, much appreciated... :)

I can reproduce your findings that following the FFVideoSource call with RequestLinear brings back full speed.

Meanwhile I repeated my tests using LWLibavVideoSource, and to a lesser degree it shows the same behavior. Instead of 59 or 36 fps with FFVideoSource I get 80 fps with LWLibavVideoSource, but following the call with RequestLinear or ChangeFPS also brings back the full speed.

So it looks like it is an FFmpeg thing. Is this something the FFmpeg devs should fix, or should I ask pinterf for a workaround in AVS+? Or should I just live with it and use RequestLinear after the source filter as my standard procedure?


Cheers
manolito

[qyot27]

New build (https://forum.doom9.org/showthread.php?p=1829061#post1829061).

Five months newer versions of FFmpeg and libdav1d.

[manolito]

Thanks for the new builds...

If you now decided to also make C++ plugin versions, it is probably not a good idea to only promote them in this C-Plugin thread.

For the C++ plugins the situation already is a little bit confusing. We have current builds from HolyWu, StvG and now also from you (Myrsloik seems to have abandoned ffms2000).

Hard to find out the subtle differences between all these builds for the average Joe...


Cheers
manolito

[StainlessS]

Hard to find out the subtle differences between all these builds for the average Joe...
Yep, and some indication of the coder that did the build [in file name] would not be a complete waste of time.
[version resource in all future versions would also be nice, some have them, some dont]

& thanks Q27.

[qyot27]

For the C++ plugins the situation already is a little bit confusing. We have current builds from HolyWu, StvG and now also from you (Myrsloik seems to have abandoned ffms2000).
ffms2000 got merged into the upstream master branch in Nov 2017. That's why there's no new builds of it: all the builds, even the C plugin, have had those changes in them for over a year and a half.

Hard to find out the subtle differences between all these builds for the average Joe...

I dunno about the others, but you'd find out pretty quickly that these aren't normal C++ builds if you tried to use them. They'll only work with GCC builds of AviSynth+*, and thus should only ever be put in the plugins_gcc/plugins64_gcc folder(s). Very very niche at this point.

*of course, the normal C plugin builds work with GCC builds of AviSynth+ too, but that's sort of boring.

[Myrsloik]

Thanks for the new builds...

If you now decided to also make C++ plugin versions, it is probably not a good idea to only promote them in this C-Plugin thread.

For the C++ plugins the situation already is a little bit confusing. We have current builds from HolyWu, StvG and now also from you (Myrsloik seems to have abandoned ffms2000).

Hard to find out the subtle differences between all these builds for the average Joe...


Cheers
manolito

Why would I bother making builds when everyone else is already doing it? Seems like a lot of duplicated effort to me.

[ChaosKing]

Why would I bother making builds when everyone else is already doing it? Seems like a lot of duplicated effort to me.

You could make official releases :p

[Reino]

Hello qyot27,

Despite 'ffms2_r1315%2B119-avs%2Bvsp_lastxp.7z' being the last WinXP compatible SSE build you released I was wondering if https://github.com/qyot27/ffms2_cplugin (https://github.com/qyot27/ffms2_cplugin) itself is still WinXP compatible and suitable for non-SSE2 cpus.
I've tried to build FFM2 for Avisynth (https://msfn.org/board/topic/177308-ffmpeg-windows-xp-updated-builds-by-corone/?do=findComment&comment=1181347), but upon invoking FFVideoSource() I'm getting this typical error which leads me to believe it's actually not suitable for non-SSE2 cpus anymore. Or am I totally wrong?

[FranceBB]

qyot27, as you can see from the MSDN topic (https://msfn.org/board/topic/177308-ffmpeg-windows-xp-updated-builds-by-corone/page/2/?tab=comments#comment-1181358), CoRoNe (who's the one maintaining ffmpeg xp compatible binaries (https://rwijnsma.home.xs4all.nl/files/ffmpeg/?C=M;O=D) updated) actually managed to compile ffms2 targeting XP and when I tried it, FFAudioSource worked like a charm. When I tried to use FFVideoSource, it even created the .ffindex file relative to the video file I was trying to index, however it failed to open the uncompressed video stream in Avisynth and reported an access violation.

I'm just gonna report a few replies from MSDN to here for completeness:


I git cloned https://github.com/qyot27/ffms2_cplugin.git, patched a couple of files in order for compilation-errors to go away, made the library and executable and packed the necessary files.
You could try ffm2-2.18-736-g43dc804_avs+vsp-win32-xpmod-sse.7z (http://rwijnsma.home.xs4all.nl/files/ffms2/ffm2-2.18-736-g43dc804_avs+vsp-win32-xpmod-sse.7z), with the emphasis on "try".

Although the compilation-process went fine, I'm getting an application error pop-up "The instruction at "0x6ef94386" referenced memory at "0x00000000". The memory could not be "written"." the moment I try to open an avs-file like this...

FFVideoSource("<full-path-random-video>")
...with FFmpeg. This probably means there's SSE2 code in FFMS2 (not sure though). I'm not a C-coder, so unless someone could point me to the exact cause, I'm afraid this is all I could do.

'ffmsindex.exe' and "FFAudioSource()" did work fine during my tests though.



The following script works:

video=AVISource("Z:\Isilon\421.avi")
audio=FFAudioSource("Z:\Isilon\421.avi")
AudioDub(video, audio)

which means that FFAudioSource is working:

https://i.imgur.com/bw6BJaD.png


however if I try to use FFVideoSource directly it reports an access violation, just like the one you got:


FFVideoSource("Z:\Isilon\421.avi")


https://i.imgur.com/A73dqiE.png
https://i.imgur.com/YEasdT0.png

The weird thing is that it tries to do something and in fact the indexer does index the file and it creates the relative .ffindex file like: Analog_bad_source.avi.ffindex or test.ffindex which means that it's indeed working fine, but something fails when it tries to load the uncompressed video stream into Avisynth. Besides, it doesn't even have anything missing when I look at it in dependency walker. I mean, we're almost there, it can work on XP!

Did you compile it with /Zc:threadSafeInit as well?


No. Would it matter?


Yep, many Avisynth plugins can compile but don't work on XP without /Zc:threadSafeInit as XP doesn't have a way to handle the way they use to handle threads, therefore /Zc:threadSafeInit is needed to make them thread-safe and compatible with XP. If you could try to compile it again but with /Zc:threadSafeInit I can test it.

@qyot27... do you have any insights about what is causing the access violation? 'Cause, you know, it's working fine for everything but the very last part of showing the video inside Avisynth / ffmpeg etc.


@CoRoNe... for the records, I'm testing it with an SSE4.2 capable CPU on Windows XP, so, for now, my bets are on /Zc:threadSafeInit

[qyot27]

Nothing in the C-plugin or the FFMS2 core itself causes it to be incompatible with XP. It's entirely due to the dependencies linked into it (read: FFmpeg) and the compilation environment.

/Zc:threadSafeInit is completely meaningless to the C plugin, that's an MSVC option. The C plugin is built with MinGW-w64/GCC (maybe Clang, but I've never tried).

The GCC optimization level is probably at fault. Most of the builds I've provided use --optlevel=0 (and even --enable-debug) because something in -O2 screws over the src/avisynth_c/ sources specifically.

[Reino]

This is the first time I had to compile something with a debug-option for it to work. --optlevel=0 didn't work, but --enable-debug --extra-ldflags=-s does! Do you know why -O2 is so problematic?

Alright, ffm2-2.18-736-g43dc804_avs+vsp+ffmpeg-4.3-3133-g1128aa8-win32-xpmod-sse.7z (http://rwijnsma.home.xs4all.nl/files/ffms2/ffm2-2.18-736-g43dc804_avs+vsp+ffmpeg-4.3-3133-g1128aa8-win32-xpmod-sse.7z) it is.
So, FFMS2 2.18-736-g43dc804 (https://github.com/qyot27/ffms2_cplugin/commit/43dc8040cfc50677422cd48268a84f139d708acb) compiled with FFmpeg 4.3-3133-g1128aa8 (https://github.com/FFmpeg/FFmpeg/commit/1128aa875367f66ac11adc30364d5652919a2591). Both built with MinGW-w64 7.0.0 and GCC 9.3.0.
FFmpeg buildconf:
--arch=x86
--target-os=mingw32
--prefix=$mingw_w64_x86_64_prefix
--cross-prefix=$cross_prefix
--extra-cflags='-march=pentium3 -mtune=athlon-xp -O2 -mfpmath=sse -msse'
--pkg-config=pkg-config
--pkg-config-flags=--static
--enable-gpl
--enable-version3
--disable-bcrypt
--disable-debug
--disable-doc
--disable-htmlpages
--disable-manpages
--disable-podpages
--disable-schannel
--disable-txtpages
--disable-w32threads
--disable-avdevice
--disable-avfilter
--disable-devices
--disable-encoders
--disable-filters
--disable-hwaccels
--disable-muxers
--disable-network
--disable-programs
--disable-sdl2
--enable-libaom

[qyot27]

I've been stumped by it for years. If I could pinpoint exactly which option or combination of options in the optimizations causes it (it's not -O2 specifically, any level of optimization does it, -O2 is just the default¹), then I could try to hunt down the part of the plugin code that causes the issue. There was a point, back like 8 years ago, that it wasn't necessary. Updates to MinGW-w64 and/or GCC brought on the need to start messing around with the optlevel and then debug (--enable-debug is only needed for 32-bit builds, oddly enough; the 64-bit builds can get by with just --optlevel=0; --enable-debug also turns off the optimizations, but also sets -g in the C(XX)FLAGS, and stranger is that if you're stripping the symbols right back out at link time, that should make it equivalent to --optlevel=0, but it clearly isn't). That it seems to not cause problems for audio might help figuring out what part is at fault here; I wouldn't be surprised if it ends up being something in ff_vidsource.c.

¹because it's not needed for the core or, IIRC, even for the C++ or (now really deprecated) AvxSynth plugin. At least on Linux, anyway.

[Groucho2004]

- deleted -

[FranceBB]

Alright, ffm2-2.18-736-g43dc804_avs+vsp+ffmpeg-4.3-3133-g1128aa8-win32-xpmod-sse.7z (http://rwijnsma.home.xs4all.nl/files/ffms2/ffm2-2.18-736-g43dc804_avs+vsp+ffmpeg-4.3-3133-g1128aa8-win32-xpmod-sse.7z) it is.


OMG OMG OMG!
IT WORKS!!

https://i.imgur.com/x06fCqE.png

Other test with AVSPmod new ffms2 index: Image (https://i.imgur.com/xnvumMo.png)

Yes!! It works!!
annuntio vobis gaudium magnum: habemus ffms2!!
And with modern indexers, XP is more alive than ever! :D
Thank you, thank you, thank you so much!

@StainlesSs, Manolito, come here, we gotta celebrate (with social distancing)!! xD

https://youtu.be/roRQ2mNwMMQ

[StainlessS]

Great stuff, thiings looking up :)

[videoh]

http://rationalqm.us/misc/beat.jpg

[amayra]

http://rationalqm.us/misc/beat.jpg

I wonder why you are not banned yet?

[videoh]

Probably the mods have the sense of humor that you lack.

http://rationalqm.us/misc/road_ends.jpg

http://rationalqm.us/misc/change.jpg

[filler56789]

^ And probably "sense of humor" is a euphemism for «unconditional love» :sly:

[Groucho2004]

^ And probably "sense of humor" is a euphemism for «unconditional love» :sly:I'm not so sure about that.

Anyway, even though I was using XP intermittently until a few months ago, I find myself unable to do anything useful with it now. Lots of software/features I use just don't work (any more). I certainly enjoy doing everything 64 bit, especially working with raw images in Photoshop/Adobe Camera Raw which was a nightmare on 32 bit Windows (Adobe products consume memory like DonnyT does burgers :)). Not to mention that there are no drivers for contemporary graphics cards.

[videoh]

^ And probably "sense of humor" is a euphemism for «unconditional love» :sly: Sorry for being so lovable.

Not to mention that there are no drivers for contemporary graphics cards. That's a biggie.

[Reino]

Yes!! It works!!
That's good to hear! ;)
Thanks for the confirmation.

[manolito]

Took me a little while to test it thoroughly, but I can confirm that this build of ffms2 works nicely... :D

Under WinXP with a Non-SSE2 CPU no poblems with all my workflows.

Then I tested it under Win7-64 and AVS+ 3.51 32-bit. Also no unexpected issues, especially comparing it to the latest qyot27 XP build (which is already 2 years old) I did not find problems.

The problems I did find have nothing to do with this build, they are present in all ffms2 builds I have tested (C-plugins and also standard C++ plugins by StvG and MeteorRain). These problems occur when trying to decode a HD HEVC video plus AAC-LATM audio file in an MTS container (German DVB-T2 video captures).

When trying to decode the MTS file I got the error that no audio track was found. After repacking the MTS to MKV using the current MKVToolNix the audio track was detected and converted without any error messages, but the resulting audio was distorted and crackling so it was unusable.

Changing the audio source filter to either DSS2Mod with LAVAudio or to LWLibavAudioSource fixed the distortion. But under LWLibavAudioSource I could not get it running stable with multithreading.

The weird thing is that FFmpeg itself has no problem to decode the AAC-LATM track to WAV. My workaround for such sources is to use FFmpeg to repack the captured MTS to MKV using Copy for video and PCM for audio. This MKV decodes nicely without any quality issues under all my source filters.


So thanks again to CoRoNe for the continued support for us retro XP users. :thanks:


Cheers
manolito

[Emulgator]

Many thanks, CoRoNe !
This opens a detour path where highways have become unavailable.

[Reino]

If I could pinpoint exactly which option or combination of options in the optimizations causes it (it's not -O2 specifically, any level of optimization does it, -O2 is just the default¹), then I could try to hunt down the part of the plugin code that causes the issue.
I've done some testing and -fomit-frame-pointer (https://github.com/qyot27/ffms2_cplugin/blob/c_plugin/configure#L480-L484) proves to be the culprit:
if [ "$debug" != "yes" -a "$gprof" != "yes" ]; then
CFLAGS="$CFLAGS -fomit-frame-pointer"
CXXFLAGS="$CXXFLAGS -fomit-frame-pointer"
LDFLAGS="$LDFLAGS -s"
fi
This fixes that for now:
diff --git a/configure b/configure
index 376bcb8..cc07783 100755
--- a/configure
+++ b/configure
@@ -478,8 +478,8 @@ if [ "$pic" = "yes" ] ; then
fi

if [ "$debug" != "yes" -a "$gprof" != "yes" ]; then
- CFLAGS="$CFLAGS -fomit-frame-pointer"
- CXXFLAGS="$CXXFLAGS -fomit-frame-pointer"
+ CFLAGS="$CFLAGS -fno-omit-frame-pointer"
+ CXXFLAGS="$CXXFLAGS -fno-omit-frame-pointer"
LDFLAGS="$LDFLAGS -s"
fi
if [ "$debug" = "yes" ]; then

It does appear to be a GCC option of significance, because the 'ffms2.dll' I compiled earlier with --enable-debug --extra-ldflags=-s and with $CFLAGS containing -O0 is 23.413.760 bytes in size, while a new 'ffms2.dll' compiled with this fix and despite $CFLAGS containing -O2 is 23.564.800 bytes in size.

[qyot27]

I thought I'd tested that a few years ago, but maybe not. I recall that x264 used to have issues with -fomit-frame-pointer, not sure if it still does.

[Reino]

For those interested, I've made a new Windows XP compatible non-SSE2 build. See https://rwijnsma.home.xs4all.nl/files/ffms2/ (https://rwijnsma.home.xs4all.nl/files/ffms2/).

[manolito]

:thanks: :thanks:

[FranceBB]

Thank you very much indeed! :D
The only thing that was slowly killing XP was the lack of updated indexers, but with ffms2 still supported, XP is gonna live for a very long time! :D

[kedautinh12]

For those interested, I've made a new Windows XP compatible non-SSE2 build. See https://rwijnsma.home.xs4all.nl/files/ffms2/ (https://rwijnsma.home.xs4all.nl/files/ffms2/).

Can you build x64 ver??

[StainlessS]

Can you build x64 ver??
++ 1, thanks.

[Reino]

Can you build x64 ver??No, I can't. Sorry.

[manolito]

@qyot27

Just the other day I made some seek tests with several source filters using Atak's SeekTester utility. See here:
https://forum.doom9.org/showthread.php?p=1927119#post1927119

The puzzling thing for me is that your latest FFMS2 C-plugin 1342+129 from last year is the only ffms2 build which handles VP9 video without any seek errors. Amazing...

What did you do differently than all the other guys like Reino, Myrsloik or StvG? Can you share your secret?


Cheers
manolito

[qyot27]

I simply paid attention to the pull requests opened over on the main FFMS2 repo and merged in PR#323 (https://github.com/FFMS/ffms2/pull/323), which is noted in the PR as not being a final version of the patches (so it's not merged into the default c_plugin branch when cloned), and on the C-plugin repo I added a couple extra patches on top to handle a version check failure and making it aware of AV1. The official builds won't include a WIP patchset, the StvG builds apparently reverted to an even earlier version of FFMS2's VP9 parsing instead of using PR323, and unless Reino used the patches_plusvp9av1 branch (https://github.com/qyot27/ffms2_cplugin/commits/patches_plusvp9av1), those are basically the same as the unpatched upstream version (if that branch was used, then maybe something changing in FFmpeg's libraries might be the cause).

I obliquely mentioned that those patches were included way back here (https://forum.doom9.org/showthread.php?p=1869630#post1869630).

[FranceBB]

I too had issues with ffms2 on VP9 streams, but, as pretty much everyone does, as soon as I find a problem with ffms2, I use LSMASH.
Using LWLibavVideo and Audio Source on the very same VP9 stream solved the issue as it was indexed correctly and I'm talking about the latest XP Compatible version of LSMASH which is quite old now.
Very interesting...

[StvG]

I simply paid attention to the pull requests opened over on the main FFMS2 repo and merged in PR#323 (https://github.com/FFMS/ffms2/pull/323), which is noted in the PR as not being a final version of the patches (so it's not merged into the default c_plugin branch when cloned), and on the C-plugin repo I added a couple extra patches on top to handle a version check failure and making it aware of AV1. The official builds won't include a WIP patchset, the StvG builds apparently reverted to an even earlier version of FFMS2's VP9 parsing instead of using PR323, and unless Reino used the patches_plusvp9av1 branch (https://github.com/qyot27/ffms2_cplugin/commits/patches_plusvp9av1), those are basically the same as the unpatched upstream version (if that branch was used, then maybe something changing in FFmpeg's libraries might be the cause).

I obliquely mentioned that those patches were included way back here (https://forum.doom9.org/showthread.php?p=1869630#post1869630).
The patches from your patches_plusvp9av1 branch are included in my build - https://forum.doom9.org/showthread.php?p=1921581#post1921581 ( https://pastebin.com/0wS5syun - lines from #23 to #39; from #275 to #291; from #347 to #366).

[qyot27]

Along with multiple other pieces that alter PTS/DTS and discontinuity handling (likely what Myrsloik meant when realfinder tried to submit it upstream (https://github.com/FFMS/ffms2/pull/367)), which there is no patch history or topic separation for to show what got changed relative to any other patches.

[StvG]

Just tested...

FFMS2 C-plugin 1342+129 isn't seek accurate with default threads here - 1st image - c-plugin, 2nd image - cpp-plugin.

https://images2.imgbox.com/96/b1/rXEBX4g3_o.png
https://images2.imgbox.com/fe/55/Y58CKw9u_o.png

As you can see cpp-plugin has the same amount of errors as my build ( https://forum.doom9.org/showthread.php?p=1927067#post1927067 ) - 6 errors.

[manolito]

This is weird...

What is the value for your default threads? For my CPU I get 4 default threads, and the test finishes without any errors.

(This applies to the C-Plugin. The CPP-Plugin from this build is not really useable).

[qyot27]

CPU used? Because running it just now with v1.2 of that program didn't flag anything. i5-9400, so 6 cores total.

As manolito hinted at, I'm betting it's something related to # of default cores/threads on a particular setup. There's also this:
https://forum.doom9.org/showthread.php?p=1921643#post1921643

Which seems like it wouldn't be too hard to get it to misbehave on a Threadripper but be completely fine on a typical consumer CPU with a traditionally moderate # of cores. Whether the issue is really in the testing setup or the plugin or in libavcodec/format itself is another matter.

[StvG]

@manolito, with my build you will have default threads 3 (cpu threads * 0.75).

Tested with v1.2 and the results for the all 3 builds are the same.
Tested with i9 7900x on vm with 16 vcores and 16GB RAM.
All 3 builds with threads=10 have the same amount of errors (166 for c-plugin and 6 for cpp-plugins).
All 3 builds with threads=8 are seek accurate.

Edit: @manolito, do you use 32-bit or 64-bit?

Edit2: Tested on vm with 4 vcores.
c-plugin doesn't have seek errors.
my build with default threads (3) has 14 errors.
my build with threads=4 doesn't have seek errors.
@manolito, you can try with manual set threads=4.

[qyot27]

New build (https://forum.doom9.org/showthread.php?p=1829061#post1829061), for the first time in a while.

[manolito]

@qyot27
Just tested your new build, no problems so far, everything works. No seek issues with my default 4 threads. Thanks a lot...

@StvG
with my build you will have default threads 3 (cpu threads * 0.75)
And this seems to cause problems. Uneven # of threads are not handled well by libavcodec. And it also contradicts the documentation which says:
The number of decoding threads to request from libavcodec.
Setting it to less than or equal to zero means it defaults to the number of logical CPU's reported by Windows.

I only use 32-bit AVS+ (latest pinterf 3.6.2 test3) and AVS plugins.

You are right, specifying threads=4 fixes all seek errors for me. The same goes for threads=2 and threads=1. Looks like only uneven thread numbers cause the errors.


Unfortunately the Reino XP compatible ffms2 builds do not react to specifying different # of threads. Too bad...


A little OT for this thread - I also tested your latest LSMASH build which caused corrupted frames upon seek operations. As you already said, these errors disappear when using 2 threads. But using 4 threads does not help at all. Using only 1 thread also gets rid of the seek problems.


So it looks like the number of libavcodec threads is the key. Go figure...


Cheers
manolito

[StvG]

threads=5/7 doesn't give errors. Only threads=3 gives errors when threads are <=8. It's the same for c-plugin.

About LSMASH - I didn't changed anything than applying the pull request for frame properties and changing exlibs (removed mfx, added dav1d and bcrypt).
It's interesting that official build with default threads has ~same speed as my build with 2 threads - ( https://forum.doom9.org/showthread.php?p=1927055#post1927055 ).

[Reino]

Unfortunately the Reino XP compatible ffms2 builds do not react to specifying different # of threads. Too bad...I've compiled ffms2 (and FFmpeg) on an old AMD Athlon XP cpu (single core of course) for use on similar old hardware, so this is no surprise.

[manolito]

I've compiled ffms2 (and FFmpeg) on an old AMD Athlon XP cpu (single core of course) for use on similar old hardware, so this is no surprise.

Yes, I know, but this is not really my point. What I wanted to say is that the StvG ffms2 builds when dealing with a VP9 stream do sometimes have seek errors, but this depends on the number of libavcodec threads. Your builds (and also qyot27's older XP compatible builds) do not react to specifying different numbers of threads. In fact they do not handle VP9 streams at all. This is probably caused by the XP compatible libraries, not much anyone can do about it...

The other thing is that I do not believe that compiling on an old single core Athlon necessarily prevents multithreading. Your XP compatible FFmpeg builds sure do multithreading nicely when they are run on a modern CPU.


Cheers
manolito

[manolito]

Just noticed that Reino has built new XP compatible versions of ffms2 and also FFmpeg... :thanks:

Will start testing right away.

[FranceBB]

Just noticed that Reino has built new XP compatible versions of ffms2 and also FFmpeg... :thanks:

Will start testing right away.

Ah! Thanks for the heads up, mate! I'll test it straight away too! :)

[manolito]

Just finished my usual test routine for these latest ffms2 and FFmpeg builds. Pretty boring, because there were no problems whatsoever, everything JUST WORKED.

Thanks again Reino for taking care of us retro WinXP users...

[FranceBB]

Just finished my usual test routine for these latest ffms2 and FFmpeg builds. Pretty boring, because there were no problems whatsoever, everything JUST WORKED.


Same here, everything is working fine. I didn't have time to test it with VP9, though, to see if there were any problems (you know, the ones raised with parallelism etc).


Thanks again Reino for taking care of us retro WinXP users...

+1
Thank you, Reino, for keeping ffms2 C plugin updated. Without it, we wouldn't have any updated indexer on XP, so this is of vital importance. :)

[Reino]

Always good to hear. Appreciated.

[manolito]

I didn't have time to test it with VP9, though, to see if there were any problems (you know, the ones raised with parallelism etc).

Sorry, I forgot to mention that for VP9 video sources this build behaves the same as the previous build from Sept. 2020. Atak's seek tester reports hundred of errors. So for decoding VP9 under WinXP you need to use a different source filter. FWIW the latest XP compatible version of LSMASH lsmash-r784-150316-XP does handle VP9 sources without any seek errors. This version is included in the current MeGUI-32bit download.

[FranceBB]

Sorry, I forgot to mention that for VP9 video sources this build behaves the same as the previous build from Sept. 2020. Atak's seek tester reports hundred of errors.

Ok, so just like the old one, got it.



So for decoding VP9 under WinXP you need to use a different source filter. FWIW the latest XP compatible version of LSMASH lsmash-r784-150316-XP does handle VP9 sources without any seek errors.

Yeah, that's exactly what I've done last time and it's what I'm gonna do in the future too, then.

[Reino]

Is this a WinXP issue? Or does this only happen with my binaries? If not, then have you considered reporting this issue (bug?) to the libvpx developers?

[manolito]

Is this a WinXP issue? Or does this only happen with my binaries?

I don't think that it is a direct WinXP issue. Have a look here:
https://forum.doom9.org/showthread.php?p=1927119#post1927119
All tests were done under Win7-64.

The StvG versions up to Oct. 2020 as well as the latest official Myrsloik version 2.40 have these problems, too. The newest StvG version from Nov.23.2020 fixed it, though.

The older qyot27 C-Plugin versions do have the issue up to the latest final XP compatible version. All builds after this are fine.

So I tend to think that some of the patches used by qyot27 in the current non-XP versions took care of the problem. Probably these patches cannot be used for XP compatible versions.

[qyot27]

New build (https://forum.doom9.org/showthread.php?p=1829061#post1829061). Mostly boring, just nearly 6 months' newer FFmpeg.

[manolito]

I think I found a small discrepancy between the ffms2 documentation and the actual plugin behavior:

For the "colorspace" parameter in ffvideosource the doc says that "YV12" can be replaced by "YUV420P8". Does not work for me, ffms2 throws an error if I do this.

The same thing happens with Reino's latest XP compatible C-Plugin (I did not test StvG's latest C++ build).

No big deal, but it would be nice if this was fixed, because LWLibavVideoSource has no problem with it.

[qyot27]

I haven't touched the docs whatsoever to describe the functionality of the C plugin. The docs are for the C++ plugin (generally speaking, though, the options and whatnot are the same, save for the C++ plugin having the FFmpegSource2/FFMS2 function inside the plugin now instead of in the .avsi...but the .avsi is more or less self-documenting).

You can see the difference pretty easily here:
https://github.com/ffms/ffms2/blob/master/src/avisynth/avssources.cpp#L39
https://github.com/qyot27/ffms2_cplugin/blob/bc1f95cf9fad2cf5622b49ee8f120c92d169c13d/src/avisynth_c/avs_utils.c#L33

That said, it was an oversight that I didn't alias the newer names for the 2.5/2.6 pix_fmts into the table in avisynth_c/avs_utils.c when I added all the others.

[qyot27]

New build, first post. Not sure what happened with the branch numbering.

[manolito]

:thanks:

[Reino]

@Reino

could you have a look at this post:
https://forum.doom9.org/showthread.php?p=1943204#post1943204Done (https://rwijnsma.home.xs4all.nl/files/ffms2/). Untested though.

[FranceBB]

Done (https://rwijnsma.home.xs4all.nl/files/ffms2/). Untested though.

Thank you!! :D

[qyot27]

New build, first post. Includes both Windows and macOS.

[kedautinh12]

Thanks

[qyot27]

New build, first post.