Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd January 2018, 00:29   #1  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 901
FFmpegSource - C-plugin

FFMS2 C-plugin r1293+111 (EDIT 2018-01-08: hotfix build)

Optimized for Pentium-III and SSE (32-bit)
Optimized for Core2 (64-bit)

ffmpeg version N-89607-g52d70aded6 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
libavutil 56. 7.100 / 56. 7.100
libavcodec 58. 8.100 / 58. 8.100
libavformat 58. 3.100 / 58. 3.100
libavfilter 7. 7.100 / 7. 7.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

32-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/32bit
    --cross-prefix=i686-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
    --target-os=mingw32
    --arch=x86
64-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/64bit
    --cross-prefix=x86_64-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-march=core2'
    --target-os=mingw32
    --arch=x86_64

There was a vote regarding this on the FFmpeg-devel mailing list a couple weeks ago (thread starts here), so as of commit 9b121dfc32810250938021952aab4172a988cb56, support for Windows XP was dropped in FFmpeg-git, which now requires the use of the newer lock and thread APIs introduced in Vista. 3.4.x point releases should still support XP, but 3.5.x releases won't (because new release series are branched directly from the master branch as of X date, and that hasn't happened yet).

Due to this, this build of FFMS2 will likely be the last XP-compatible build I provide, since there's no way to get around the incompatibility in FFmpeg. This doesn't mean the FFMS2 source code itself is no longer compatible with XP (for the time being), but it would have to be forever locked to an older version of FFmpeg, reducing its utility in coming years (XP already doesn't support using AVX-512 instructions, so any speedups from that wouldn't work anyway). The problem with the builds is slightly different, in that it's not feasible to maintain two MinGW-w64/GCC cross environments under the same OS: one for current Windows and one for XP. Since FFmpeg - as the largest piece in my cross-compile chain - was really the only reason I kept MinGW-w64's Secure API support disabled, once GCC 7.3 is released and I update my environment, nothing in the environment will support XP. I also suspect that enabling the Secure API might fix some bugs and crash behavior I've noticed in various spots (or at the very least help pinpoint that the bugs lie somewhere other than in the fact I kept that disabled).

I'm not changing the build options for 32-bit, though, so future builds should still work on plain SSE setups for those users that have switched to using AviSynth under Wine, those brave enough to attempt running ReactOS, or the insane that have Windows 7 installed on a Pentium III.


To that end, I'm also throwing up a full build of the last XP-compatible commit to FFmpeg-git, with the also-new x264-multidepth patches cherrypicked into the last XP branchpoint, since those patches for x264 fell after the drop (release 3.4 also has the x264 patches, but cherry-picking into the git version is more up-to-date).

FFmpeg itself
x264 (for the LAVF and FFMS inputs; FFMS is a couple commits behind the plugin build above, since I built this one a few days ago)

The last Windows XP compatible commit to FFmpeg is:
Code:
git checkout -b lastxp 2aee5a87233277027bb281372f888b5246a58e77
cherry-pick x264 multi-bitdepth patches for full build
Code:
git cherry-pick -1 89f704cabab446afc8ba6ecea76714a51b1df32b && \
git cherry-pick -1 2a111c99a60fdf4fe5eea2b073901630190c6c93 && \
git cherry-pick -1 7e60c74329353db28db00552028bc88cd2a52346
I have a 'lastxp' branch on my Github repo with the above already set up with the x264 patches included:
Code:
git clone -b lastxp git://github.com/qyot27/FFmpeg.git

Last edited by qyot27; 9th January 2018 at 10:31. Reason: Edit: hotfix build
qyot27 is offline   Reply With Quote
Old 3rd January 2018, 01:18   #2  |  Link
CoRoNe
Registered User
 
CoRoNe's Avatar
 
Join Date: Nov 2005
Posts: 645
Quote:
Originally Posted by qyot27 View Post
There was a vote regarding this on the FFmpeg-devel mailing list a couple weeks ago (thread starts here), so as of commit 9b121dfc32810250938021952aab4172a988cb56, support for Windows XP was dropped in FFmpeg-git, which now requires the use of the newer lock and thread APIs introduced in Vista.
I've seen the vote and as far as I know this only concerns w32threads.
https://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222869.html:
Quote:
...
Windows XP is hereby not a supported build target anymore. It was
decided in a project vote that this is OK. (Technically, it could still
be built for Windows XP using an external pthread lib as of this
commit.)
I've just released another WinXP compatible FFmpeg build using pthreads. I haven't extensively tested it myself, but so far it's working just fine.
__________________
My hobby website
CoRoNe is offline   Reply With Quote
Old 3rd January 2018, 02:29   #3  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 40
Quote:
Originally Posted by CoRoNe View Post
I've seen the vote and as far as I know this only concerns w32threads.
No, the vote was to drop WinXP support:
"We should drop XP support, and allow unconditional use of Windows Vista APIs"

Quote:
Originally Posted by CoRoNe View Post
I've just released another WinXP compatible FFmpeg build using pthreads. I haven't extensively tested it myself, but so far it's working just fine.
Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.
lvqcl is offline   Reply With Quote
Old 3rd January 2018, 02:53   #4  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,061
Thanks so much qyot27 for this new version...

As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again. I tested it under WinXP with a non-SSE2 capable CPU, no problems whatsoever.


Thanks again and Cheers
manolito
manolito is offline   Reply With Quote
Old 3rd January 2018, 05:30   #5  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 901
Quote:
Originally Posted by lvqcl View Post
Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.
I sort of forgot that w32threads were a separate method. And I actually do use --disable-w32threads (IIRC, because FFMS2's C++11 threading requires MinGW-w64's win32-pthreads, and I can't remember if w32threads plays nice with that), so I guess it might actually hang on a while longer.

The uncertainty about it breaking somewhere deeper at any time, though...

Quote:
Originally Posted by manolito
As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again.
It's been quite some time since I did the update to support AviSynth+'s huge number of new colorspaces in the C plugin parts, so I don't remember whether I actually fixed that differently than the previous build or not. Good to hear that it's working, though.
qyot27 is offline   Reply With Quote
Old 3rd January 2018, 08:03   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,270
Thank you Q Branch, your continued support has always been appreciated (now where did I leave Moneypenny?).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 3rd January 2018, 21:33   #7  |  Link
FranceBB
FranceBB
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 220
@qyot27... thank you very much indeed for supporting XP
FranceBB is offline   Reply With Quote
Old 4th January 2018, 00:01   #8  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 980
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable

Last edited by TheFluff; 4th January 2018 at 00:09.
TheFluff is offline   Reply With Quote
Old 4th January 2018, 01:48   #9  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,061
Quote:
Originally Posted by TheFluff View Post
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable
Yes, and according to this:
https://www.theregister.co.uk/2018/0...u_design_flaw/
I will also not get a slowdown of my computer for up to 30% which these (future) patches will introduce...

Cheers
manolito
manolito is offline   Reply With Quote
Old 4th January 2018, 02:28   #10  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 40
ffmpeg-related tasks are usually computationally intensive, so performance penalty for them should be close to 0.

P.S. anyway, even 30% performance hit is better than a new ransomware like WannaCry or (Not)Petya.

Last edited by lvqcl; 4th January 2018 at 03:24.
lvqcl is offline   Reply With Quote
Old 4th January 2018, 05:33   #11  |  Link
Andouille
Registered User
 
Andouille's Avatar
 
Join Date: May 2012
Posts: 53
Quote:
Originally Posted by TheFluff View Post
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable
What does this have to do with video processing ?



@qyot27... thank you
Andouille is offline   Reply With Quote
Old 4th January 2018, 08:17   #12  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,061
Quote:
Originally Posted by qyot27 View Post
Good to hear that it's working, though.
After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems...

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.

I tried some workarounds like adding the folder from where ffms2 is loaded by AVStoDVD to the environment path variable. I also disabled DEP completely, but nothing helped. I ended up going back to the old ffms2 version from Oct 2016.


Any ideas what is happening here?

Cheers
manolito
manolito is offline   Reply With Quote
Old 4th January 2018, 13:26   #13  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,420
Quote:
Originally Posted by manolito View Post
After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems...

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.
Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.
Midzuki is offline   Reply With Quote
Old 4th January 2018, 13:55   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,270
Quote:
ancient desktop PC running WinXP
So WXP_32 and W7_64, did anyone try WXP_64 ? [EDIT: or indeed W7_32]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 5th January 2018, 06:41   #15  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,061
Quote:
Originally Posted by Midzuki View Post
Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.
Thanks for confirming my findings...

The thing which I do not understand is why it works under WinXP. Of course the old desktop computer has a single core CPU (Celeron Coppermine 1.1 Ghz, made in the year 2000), and it has very low system RAM (576 MB). But it runs the same version of AviSynth (2.61 Alpha VC6) as the other Win7-64 computers.

So I tried a few more things under Win7-64. First I disabled Hyperthreading, then I completely disabled multicore processing. I added the "threads=1" parameter to ffvideosource. And I tried to force AVStoDVD to only use 1 CPU core globally. Nothing helped.

I could not even get AVSMeter to create a report about the AVS script which used ffms2. AVSMeter crashed immediately.


So I have to conclude that this latest version of the qyot27 C-Plugin does not work under a 64-bit OS using the standard AVS versions (I forgot to mention that I also tried using different builds of AVS versions 2.60 and 2.61). I did not test it under AVS+ because I have no intention to upgrade to AVS+ in the forseeable future.


Cheers
manolito
manolito is offline   Reply With Quote
Old 5th January 2018, 08:15   #16  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 901
It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.

The 'why does it work on XP?' issue had me stumped (I'd mentioned it in passing with the build before this), but as this MSDN thread notes is more than likely down to XP's heap manager not being as comprehensive about warning users about corruptions that occur. So the bug is still there, but XP doesn't bother telling you about it.

The actual issue itself may have been a longstanding bug from back when kemuri-9 was the one doing C-plugin stuff (since the actual line that fails was virtually unchanged since then) or it was a regression that happened when the allocation method for AviSynth.dll changed recently to get rid of calloc (in which case it'd be my fault for not noticing it was trying to NULL the avs_lib after it had been freed).
qyot27 is offline   Reply With Quote
Old 8th January 2018, 19:13   #17  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,362
Quote:
It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.
Any new on the new build?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 9th January 2018, 04:11   #18  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 901
Quote:
Originally Posted by Selur View Post
Any new on the new build?
Yup.

FFMS2 C-plugin r1293+111

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.
qyot27 is offline   Reply With Quote
Old 9th January 2018, 04:48   #19  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,192
What's the code separation like between the main branch and the FFMS2000 experimental branch?
burfadel is offline   Reply With Quote
Old 9th January 2018, 06:24   #20  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,420
Quote:
Originally Posted by qyot27 View Post
Yup.

FFMS2 C-plugin r1293+111

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.


for the new build, now it works without causing crashes.
Midzuki is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.