Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th July 2011, 22:50   #1201  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
Quote:
Originally Posted by TheFluff View Post
FFMS2 can't output YV24 because Avisynth (2.5) doesn't support it. Avisynth 2.6 does, but so far I haven't looked into what needs to be done on FFMS2's side to make it a working 2.6 plugin. If you want the support like RIGHT NOW you need to write a patch yourself (any contributions are welcome), but if you're okay with waiting we'll get around to it eventually.
due to how broken/incomplete the avisynth 2.6 interface is right now, you'd have to not only take the avisynth.h but also the implementation code behind it (i think it was interface.cpp) to get any 2.6 plugins to compile atm.

this will, as a by-product, be likely to break backwards compatibility, preventing use in 2.5.
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 19th July 2011, 04:46   #1202  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Yes it was never the intention for 2.6 plugins to work in 2.5.

Cunning plugin authors may crib and allow this to work for their plugin, but in general it should not be expected.


Note: 2.5 plugins are intended to be fully supported in 2.6 within the constraints of the 2.5 framework.
IanB is offline   Reply With Quote
Old 19th July 2011, 11:56   #1203  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 913
Quote:
Originally Posted by IanB View Post
Yes it was never the intention for 2.6 plugins to work in 2.5.

Cunning plugin authors may crib and allow this to work for their plugin, but in general it should not be expected.
That's really annoying. I really don't want to maintain two separate interfaces (three, if you count the avs64 C interface that kemuri-_9 maintains). How much clever ifdeffing is required, really?
TheFluff is offline   Reply With Quote
Old 23rd July 2011, 11:08   #1204  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
ffms2-r507.7z
ffms2-r507-x64.7z

64-bit binary no longer requires LoadCPlugin. It's just a regular avisynth plugin now.
I changed the name slightly to try and reflect this. My compiles will no longer require LoadCPlugin (obviously all old avs64 compiles will) but archives labeled as "x64" will be regular plugins.

Code:
LoadPlugin(X:\path\to\ffms2.dll")
For both 32 and 64-bit binaries.

Changes since last build:
-Updated libav to rev. b4cfb82.
-Update project files in svn to msvc2010.
-x64 target in msvc.
-Auto-detect number of threads if threads < 1. (The avisynth plugin already did this, this functionality is now extended to those using the api as well).

Now, msvc x64 and mingw-w64 don't exactly play nice now, but they play together well enough to make the builds possible now.

Also now included with the builds is some stuff for devs (.lib for linking, ffms.h header file, and pdb's for debugging).

Last edited by TheRyuu; 24th July 2011 at 03:50.
TheRyuu is offline   Reply With Quote
Old 23rd July 2011, 22:26   #1205  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 913
Quote:
Originally Posted by TheRyuu View Post
64-bit binary no longer requires LoadCPlugin. It's just a regular avisynth plugin now.
Wrong. It depends on who compiled it and with what compiler.
TheFluff is offline   Reply With Quote
Old 24th July 2011, 10:14   #1206  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by TheFluff View Post
Wrong. It depends on who compiled it and with what compiler.
Edited post to better announce this fact. Although you have to be pretty retarded to not realize it.

Also for future releases I'm going to do a separate 'SDK' package or something which contains release/debug builds and associated libs/headers etc...

Last edited by TheRyuu; 24th July 2011 at 10:17.
TheRyuu is offline   Reply With Quote
Old 31st July 2011, 06:53   #1207  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
shenanigans mentioned here are now committed, enjoy (and report found bugs).

I've also renamed the avs64 branch to c_plugin in a preemptive to avoid arguments like the above.
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 1st August 2011, 11:37   #1208  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 883
Is there some difference between the way the MSVC and C-plugin builds handle pthreads? Because any time I try to use the C-plugin with more than one thread (of which I mean 'leave it at the defaults', but as I'm using it on a Core 2 Duo...), HCenc crashes almost immediately. If I use the distributed MSVC builds, make the C-plugin use threads=1, or if I compile with win32threads, the crash doesn't happen. If there isn't really a difference, then I'll know it's an HCenc-side issue.
qyot27 is offline   Reply With Quote
Old 1st August 2011, 15:32   #1209  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 913
Quote:
Originally Posted by qyot27 View Post
Is there some difference between the way the MSVC and C-plugin builds handle pthreads? Because any time I try to use the C-plugin with more than one thread (of which I mean 'leave it at the defaults', but as I'm using it on a Core 2 Duo...), HCenc crashes almost immediately. If I use the distributed MSVC builds, make the C-plugin use threads=1, or if I compile with win32threads, the crash doesn't happen. If there isn't really a difference, then I'll know it's an HCenc-side issue.
The handling on FFMS2's side isn't different but the two variants are compiled by different people with different compiler versions in different ways, so it's not at all impossible that kemuri-_9's pthreads compile has some funny issue while TheRyuu's does not. I leave further investigation to kemuri-_9 since the c-plugin is his playground.
TheFluff is offline   Reply With Quote
Old 2nd August 2011, 10:49   #1210  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
I actually never build ffmpegsource against an ffmpeg that was built with pthreads on windows.

so I can't say the c plugin is very much tested in that regards, though like TheFluff has stated, it shouldn't really have a weird issue like that.

Though, this is reminding me of an issue i've seen previously with pthreads crashing x264 when built with msvc (though this was x86_64 specific).

file a bug in googlecode for it to track this, and i'll try and find time to investigate (and temporarily put aside my distaste in requiring to build ffmpeg against pthreads for the full mt support)
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 2nd August 2011, 11:27   #1211  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 883
Would it possibly be entirely up to the version of pthreads, though? Mainly because x264 doesn't exhibit the issue, even with the C-plugin using pthreads and calling more than 1 thread. Then again, x264 is built against the same version of pthreads that ffmpeg was. That was partially why I thought it might actually be something about HCenc interacting with it in some way, except it still seems isolated to when I'm using the C-plugin.

I suppose if I rolled back to a much older revision of pthreads and the crash doesn't occur, that would show it's somewhere in there, right? It was happening with stuff built against a pthreads from April 30th, 2011, and also with the current CVS version as of this past week. I used to use a build from October 2009, and I think I might do a test with it built against that one to see if the crash still happens. My memory's not good enough to recall if the builds I used to do against that one also crashed in this manner.

Last edited by qyot27; 2nd August 2011 at 11:32.
qyot27 is offline   Reply With Quote
Old 2nd August 2011, 13:29   #1212  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
It could be the version of pthreads.

I base this on the fact that the pthread issue i previously mentioned was only for a newer version, whereas an older verison did not exhibit the issue (i think the major difference for my issue was whether that autostatic stuff was there or not).
ffmpegsource nor ffmpeg handle static-but-not-non-autostatic pthread-win32 correctly, however, so you probably can't go back before that if you want to link pthread-win32 statically.
you could try using the shared lib version of pthread-win32 if you need to go back before that addition though.
in fact, trying the share lib version may be a helpful test overall...
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 2nd August 2011, 20:30   #1213  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 883
Well, the build from October 2009 caused an error during the compile process, even though it is an autostatic build (autostatic with the old patch that added it, rather than as part of the trunk CVS code); it just must be too old now to compile a current ffmpeg against. I'll have to see about building it against a shared version.
qyot27 is offline   Reply With Quote
Old 2nd August 2011, 23:05   #1214  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
i would also like to point out that TheRyuu does not typically use the standard pthread-win32 when he builds.
he generally uses a version that's based off the win32 thread support i wrote for x264, which is available here

could also trying to see if this version causes a crash or not.
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 3rd August 2011, 05:15   #1215  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by kemuri-_9 View Post
i would also like to point out that TheRyuu does not typically use the standard pthread-win32 when he builds.
he generally uses a version that's based off the win32 thread support i wrote for x264, which is available here

could also trying to see if this version causes a crash or not.
Incorrect, of all the builds I've done: 32bit builds all have standard pthreads-w32. 64bit c-plugin builds all have standard pthreads-w32.
Only 64bit msvc builds (which I've posted one build ever, as of this post) have the version based off of win32 threads to avoid some weird linker issues with standard pthreads.

All of the above have and use autostatic for libav.

ffms2-r517.7z

Changes:
Built with libav rev. 62ee0e6
Implement a working version of avcodec_find_best_pix_fmt. This should fix all automatic selection of output colorspace in avisynth.
Make FFMS_Index reference counted and shared between sources instead of partially copied.
Fix deprecation in libavformat usage. Replace av_find_stream_info with avformat_find_stream_info.

Last edited by TheRyuu; 3rd August 2011 at 06:00.
TheRyuu is offline   Reply With Quote
Old 6th August 2011, 23:10   #1216  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 883
I wasn't able to get to testing with the different pthreads builds until today, but the results are below.
  • Static pthreads-win32 (July 2011) crashes HCenc (at 3%) on sampling pass
  • Shared pthreads-win32 crashes HCenc (at 3%) on sampling pass with both:
    The pthreadGC2.dll that matched with the version of the library (July 2011) FFMS2 was built against.
    A pthreadGC2.dll from 2008 that was laying around in C:\Windows\system32
  • pw32threads crashes HCenc (at 3%) on sampling pass
The initial stuff as well:
  • w32threads = no crash
  • pthreads-win32 (July 2011) with threading disabled (threads=1) = no crash




As a different test, I decided to use a different file. With the static pthreads-win32 build of r512 (C-plugin).

It crashed at 7% in the sampling pass, because I think the issue has to do with the length of the file and where the threading would actually divvy up the frames.

However, I then remuxed the file from FLV to MKV using ffmpeg and reran the test. No crash with the MKV remux.

I'd overlooked the fact that the MSVC test I'd run earlier was with an MKV remux, so I reran it with the FLV instead. MSVC (r517, as distributed on googlecode) crashed on the FLV at 7%, just like the C-plugin did. As already stated, no crash with the MKV.

It would seem that it's an issue with the FLV demuxer (I'd guess FLV just uses libavformat, since using -m lavf when indexing still resulted in HCenc crashing at 7%) choking with pthreads-win32 when using more than one thread, but it still only happens with HCenc; x264 seems completely unaffected by it, which I wouldn't think would happen if it was something intrinsic to libavformat.

In all cases the video and audio were H.264 and AAC.
qyot27 is offline   Reply With Quote
Old 7th August 2011, 00:34   #1217  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 913
http://code.google.com/p/ffmpegsourc...s/detail?id=50

Possibly the same issue. Can you redo the test with r518 and see if you get a nice error message instead of a crash?
TheFluff is offline   Reply With Quote
Old 7th August 2011, 00:48   #1218  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
huh, i guess it's not surprising that w32threads does not exhibit the issue, as w32threads with ffmpeg still do not allow full mt support like building with pthreads.
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 7th August 2011, 05:02   #1219  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 883
r518 still crashes HCenc at 7%, no inkling of an error message during playback of the script with MPC when it passes the 7% mark.



As an aside, I normally do an svn merge in those times between the C-plugin's HEAD updates, but doing a plain merge currently fails due to the recent revisions concerning MSVC x64. Readout below:
Code:
svn merge http://ffmpegsource.googlecode.com/svn/trunk
--- Merging r506 into '.':
   C build-msvc/ffms2_include_dirs.props
   C build-msvc/ffmsindex.vcxproj.filters
   C build-msvc/ffms2.vcxproj
   C build-msvc/ffms2.sln
   C build-msvc/ffmsindex.vcxproj
   C build-msvc/ffms2.vcxproj.filters
D    build-msvc/ffms2_include_dirs.vsprops
   C build-msvc/ffms2.vcproj
   C build-msvc/ffmsindex.vcproj
svn: Attempt to add tree conflict that already exists at 'build-msvc/ffmsindex.vcxproj'
svn: Error reading spooled REPORT request response
I can work around that by specifying the revisions with -r, but if left alone it wants to merge from back at r506 instead of starting at the last C-plugin revision.
qyot27 is offline   Reply With Quote
Old 7th August 2011, 11:57   #1220  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 913
Okay, then I guess we get to do this the hard way.

Quote:
Originally Posted by qyot27 View Post
It would seem that it's an issue with the FLV demuxer (I'd guess FLV just uses libavformat, since using -m lavf when indexing still resulted in HCenc crashing at 7%)
FLV uses libavformat, yes. We don't have a special demuxer for it.

So, let me sum up what you're saying to see if I got it right:
  • The crash only affects HCenc; not x264 or MPC
  • The crash only appears if you use a FFMS2 compiled with pthreads, and only if you're actually using threads >1
  • It only affects FLV; remuxing to MKV makes the problem disappear
  • The crash happens with both the native and the C-style plugins

When you tested with r518, are you sure you actually merged in the changes from r518 in trunk to the C-plugin branch?

Also, do you have Visual Studio installed? If you do, compile FFMS2 in debug mode (or at least get ahold of a .pdb file somewhere and stick it with the .dll) and try attaching Visual Studio to the HCenc process and see if you can get a stack trace when it crashes.

Last edited by TheFluff; 7th August 2011 at 11:59.
TheFluff is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:29.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.