Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd January 2018, 00:29   #1  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 970
FFmpegSource - C-plugin

Since it's easier, I'll keep this first post updated with newer builds whenever one happens.

Issues tracker (for C plugin issues only): https://github.com/qyot27/ffms2_cplugin/issues

FFMS2 C-plugin 1327+119

Optimized for Pentium-III and SSE (32-bit)
Optimized for Core2 (64-bit)

ffmpeg version r91961+1 master-63c69d51c7 HEAD-ef69b9e604
contains: versioninfo
Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.1.0 (GCC)
libavutil 56. 19.101 / 56. 19.101
libavcodec 58. 30.100 / 58. 30.100
libavformat 58. 18.101 / 58. 18.101
libavfilter 7. 31.100 / 7. 31.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100

32-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/32bit
    --cross-prefix=x86_64-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --enable-libaom
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-m32 -mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
    --extra-ldflags='-m32 -L/usr/i686-w64-mingw32/lib'
    --target-os=mingw32
    --arch=x86
64-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/64bit
    --cross-prefix=x86_64-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --enable-libaom
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-march=core2'
    --target-os=mingw32
    --arch=x86_64
FFMS2 C-plugin r1315+119 (Final LastXP build)

Optimized for Pentium-III and SSE (32-bit)
Optimized for Core2 (64-bit)

ffmpeg version r90798 master-21da248b5f HEAD-a56580b117
contains: lastxp_wincrypt
Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.2.0 (GCC)
libavutil 56. 13.100 / 56. 13.100
libavcodec 58. 17.100 / 58. 17.100
libavformat 58. 11.101 / 58. 11.101
libavfilter 7. 15.100 / 7. 15.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 0.102 / 5. 0.102
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

32-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/32bit
    --cross-prefix=i686-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-mfpmath=sse -march=pentium3 -msse -mtune=pentium3'
    --target-os=mingw32
    --arch=x86
64-bit configuration:
Code:
    --prefix=/home/qyot27/ffmpeg_build_for_ffms2/64bit
    --cross-prefix=x86_64-w64-mingw32-
    --enable-gpl
    --enable-version3
    --disable-w32threads
    --enable-avresample
    --disable-encoders
    --disable-muxers
    --disable-doc
    --disable-debug
    --disable-devices
    --disable-avdevice
    --extra-cflags='-march=core2'
    --target-os=mingw32
    --arch=x86
FFmpeg has now switched from wincrypt to bcrypt, and this is too far-reaching of a change for any selective revert to handle - the system library it links against changed (and bcrypt.dll is not available on XP; it's part of Vista+'s API), and far more importantly, the code that requires the random_seed functions that the *crypt library is called in for is spread throughout the FFmpeg codebase - it's used in encoders, decoders, and [de]muxers, so unlike the winthread/pthread thing last time, this time XP compatibility is probably broken for good. While it could probably hang on for a while doing increasingly labor-intensive merge integrations to revert the bcrypt commits, make sure everything still compiles, and that no unwanted behavior arises because of the use of winthread, I'm not particularly enthused to do so, and the alternatives (Wine, ReactOS, using Wine in Windows if you can get it compiled) are more compelling in the long term.

Last edited by qyot27; 24th October 2018 at 18:46. Reason: EDIT 2018-09-15: r1327+119
qyot27 is offline   Reply With Quote
Old 3rd January 2018, 01:18   #2  |  Link
CoRoNe
Registered User
 
CoRoNe's Avatar
 
Join Date: Nov 2005
Posts: 645
Quote:
Originally Posted by qyot27 View Post
There was a vote regarding this on the FFmpeg-devel mailing list a couple weeks ago (thread starts here), so as of commit 9b121dfc32810250938021952aab4172a988cb56, support for Windows XP was dropped in FFmpeg-git, which now requires the use of the newer lock and thread APIs introduced in Vista.
I've seen the vote and as far as I know this only concerns w32threads.
https://ffmpeg.org/pipermail/ffmpeg-devel/2017-December/222869.html:
Quote:
...
Windows XP is hereby not a supported build target anymore. It was
decided in a project vote that this is OK. (Technically, it could still
be built for Windows XP using an external pthread lib as of this
commit.)
I've just released another WinXP compatible FFmpeg build using pthreads. I haven't extensively tested it myself, but so far it's working just fine.
__________________
My hobby website
CoRoNe is offline   Reply With Quote
Old 3rd January 2018, 02:29   #3  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 71
Quote:
Originally Posted by CoRoNe View Post
I've seen the vote and as far as I know this only concerns w32threads.
No, the vote was to drop WinXP support:
"We should drop XP support, and allow unconditional use of Windows Vista APIs"

Quote:
Originally Posted by CoRoNe View Post
I've just released another WinXP compatible FFmpeg build using pthreads. I haven't extensively tested it myself, but so far it's working just fine.
Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.
lvqcl is offline   Reply With Quote
Old 3rd January 2018, 02:53   #4  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,300
Thanks so much qyot27 for this new version...

As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again. I tested it under WinXP with a non-SSE2 capable CPU, no problems whatsoever.


Thanks again and Cheers
manolito
manolito is offline   Reply With Quote
Old 3rd January 2018, 05:30   #5  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 970
Quote:
Originally Posted by lvqcl View Post
Looks like adding --disable-w32threads option is currently enough to restore ffmpeg compatibility with WinXP. But it can break with any new commit.
I sort of forgot that w32threads were a separate method. And I actually do use --disable-w32threads (IIRC, because FFMS2's C++11 threading requires MinGW-w64's win32-pthreads, and I can't remember if w32threads plays nice with that), so I guess it might actually hang on a while longer.

The uncertainty about it breaking somewhere deeper at any time, though...

Quote:
Originally Posted by manolito
As opposed to the previous version this one works under plain vanilla AviSynth v2.6 again.
It's been quite some time since I did the update to support AviSynth+'s huge number of new colorspaces in the C plugin parts, so I don't remember whether I actually fixed that differently than the previous build or not. Good to hear that it's working, though.
qyot27 is offline   Reply With Quote
Old 3rd January 2018, 08:03   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,982
Thank you Q Branch, your continued support has always been appreciated (now where did I leave Moneypenny?).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 3rd January 2018, 21:33   #7  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 393
@qyot27... thank you very much indeed for supporting XP
FranceBB is offline   Reply With Quote
Old 4th January 2018, 00:01   #8  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,049
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable

Last edited by TheFluff; 4th January 2018 at 00:09.
TheFluff is offline   Reply With Quote
Old 4th January 2018, 01:48   #9  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,300
Quote:
Originally Posted by TheFluff View Post
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable
Yes, and according to this:
https://www.theregister.co.uk/2018/0...u_design_flaw/
I will also not get a slowdown of my computer for up to 30% which these (future) patches will introduce...

Cheers
manolito
manolito is offline   Reply With Quote
Old 4th January 2018, 02:28   #10  |  Link
lvqcl
Registered User
 
Join Date: Aug 2015
Posts: 71
ffmpeg-related tasks are usually computationally intensive, so performance penalty for them should be close to 0.

P.S. anyway, even 30% performance hit is better than a new ransomware like WannaCry or (Not)Petya.

Last edited by lvqcl; 4th January 2018 at 03:24.
lvqcl is offline   Reply With Quote
Old 4th January 2018, 05:33   #11  |  Link
Andouille
Registered User
 
Andouille's Avatar
 
Join Date: May 2012
Posts: 60
Quote:
Originally Posted by TheFluff View Post
y'all XP diehards might want to take note of this new shiny CPU exploit that you will most likely never get a patch for
if you do dangerous things such as "using a web browser with javascript enabled" you are vulnerable
What does this have to do with video processing ?



@qyot27... thank you
Andouille is offline   Reply With Quote
Old 4th January 2018, 08:17   #12  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,300
Quote:
Originally Posted by qyot27 View Post
Good to hear that it's working, though.
After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems...

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.

I tried some workarounds like adding the folder from where ffms2 is loaded by AVStoDVD to the environment path variable. I also disabled DEP completely, but nothing helped. I ended up going back to the old ffms2 version from Oct 2016.


Any ideas what is happening here?

Cheers
manolito
manolito is offline   Reply With Quote
Old 4th January 2018, 13:26   #13  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,477
Quote:
Originally Posted by manolito View Post
After a long testing session I have to report that this new version indeed works nicely, but only under WinXP. Under Win7-64bit it causes problems...

This seems weird, normally it works the other way around, so I was sceptical and tested it on 3 different machines. My ancient desktop PC running WinXP and two newer machines with Intel Core i5 CPUs running under Win7-64bit.

I used AVStoDVD for testing. This software comes with ffms2_r1140+101-avs+vsp.7z from Oct 2016. Replacing ffms2 with this new build causes no problems under WinXP. But on the two Win7 machines the encoders (HCenc and FFmpeg) as well as Wavi crash after they have finished their work. The terrible Windows popup telling the user that the software has stopped working because of some problem. And the software needs to be closed down. No further explanation.
Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.
Midzuki is offline   Reply With Quote
Old 4th January 2018, 13:55   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,982
Quote:
ancient desktop PC running WinXP
So WXP_32 and W7_64, did anyone try WXP_64 ? [EDIT: or indeed W7_32]
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is online now   Reply With Quote
Old 5th January 2018, 06:41   #15  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,300
Quote:
Originally Posted by Midzuki View Post
Just confirming, yes the latest C plugin is problematic. Tested with avs2yuv and x265.exe under 64-bit Windows 7. avs2yuv crashed after the encoding was finished.
Thanks for confirming my findings...

The thing which I do not understand is why it works under WinXP. Of course the old desktop computer has a single core CPU (Celeron Coppermine 1.1 Ghz, made in the year 2000), and it has very low system RAM (576 MB). But it runs the same version of AviSynth (2.61 Alpha VC6) as the other Win7-64 computers.

So I tried a few more things under Win7-64. First I disabled Hyperthreading, then I completely disabled multicore processing. I added the "threads=1" parameter to ffvideosource. And I tried to force AVStoDVD to only use 1 CPU core globally. Nothing helped.

I could not even get AVSMeter to create a report about the AVS script which used ffms2. AVSMeter crashed immediately.


So I have to conclude that this latest version of the qyot27 C-Plugin does not work under a 64-bit OS using the standard AVS versions (I forgot to mention that I also tried using different builds of AVS versions 2.60 and 2.61). I did not test it under AVS+ because I have no intention to upgrade to AVS+ in the forseeable future.


Cheers
manolito
manolito is offline   Reply With Quote
Old 5th January 2018, 08:15   #16  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 970
It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.

The 'why does it work on XP?' issue had me stumped (I'd mentioned it in passing with the build before this), but as this MSDN thread notes is more than likely down to XP's heap manager not being as comprehensive about warning users about corruptions that occur. So the bug is still there, but XP doesn't bother telling you about it.

The actual issue itself may have been a longstanding bug from back when kemuri-9 was the one doing C-plugin stuff (since the actual line that fails was virtually unchanged since then) or it was a regression that happened when the allocation method for AviSynth.dll changed recently to get rid of calloc (in which case it'd be my fault for not noticing it was trying to NULL the avs_lib after it had been freed).
qyot27 is offline   Reply With Quote
Old 8th January 2018, 19:13   #17  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,658
Quote:
It's been fixed locally. I'll wait to post a new build until all this recent C-plugin discussion has been split off to a new thread, though.
Any new on the new build?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 9th January 2018, 04:11   #18  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 970
Quote:
Originally Posted by Selur View Post
Any new on the new build?
Yup.

FFMS2 C-plugin r1293+111

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.

EDIT 2018-04-23: Newer build available here.

Last edited by qyot27; 23rd April 2018 at 14:02.
qyot27 is offline   Reply With Quote
Old 9th January 2018, 04:48   #19  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,235
What's the code separation like between the main branch and the FFMS2000 experimental branch?
burfadel is offline   Reply With Quote
Old 9th January 2018, 06:24   #20  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,477
Quote:
Originally Posted by qyot27 View Post
Yup.

FFMS2 C-plugin r1293+111

It's built against the same FFmpeg as last time, so I won't bother with that infoblock again.


for the new build, now it works without causing crashes.
Midzuki is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.