FFmpegSource

[burfadel]

Originally Posted by JEEB

Built a standard ffms2 with Release configuration. Available here.
.

Thanks! very much appreciated

[TheFluff]

Originally Posted by sneaker_ger

Yes, you're probably correct:
http://forum.doom9.org/showthread.ph...12#post1558212

So I presume the problem is already known and we have to be patient.

Yes, it's the same problem that has been known since forever; we only recently figured out the root cause. There's no fix yet, though.

[qyot27]

Revisiting the crash issue with certain MKV files, it seems to be related specifically to the way MinGW handles (or rather, mangles) setjmp and longjmp in matroskaparser.c when any sort of optimizations in gcc are used (as there are a couple of optimizations I saw listed in gcc's docs that seemed to do something to -jmp calls), and that files that lack certain information in their metadata or headers or whatever happen to be what can set it off, probably because those files need those calls to not be broken by MinGW, and that's only true when optimizations are turned off. It also now happens with trunk builds linked into x264, not just the C-interface AviSynth plugin, but only when built with MinGW. Cygwin and native Linux builds are wholly unaffected.

I did manage to get a backtrace a couple of weeks ago, below:

$ gdb ffmsindex.exe
GNU gdb (GDB) 7.4
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying" and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from c:\dap\vid\Incoming Files\ffms2-avs_r658-optdebug\ffmsindex.exe...done.

(gdb) r -t -1 ffms2crashtest.mkv
Starting program: c:\dap\vid\Incoming Files\ffms2-avs_r658-optdebug\ffmsindex.exe -t -1 ffms2crashtest.mkv
[New Thread 3956.0xf78]
Indexing, please wait... 0%
Program received signal SIGSEGV, Segmentation fault.
0x77c3554a in msvcrt!_abnormal_termination ()
   from C:\WINDOWS\system32\msvcrt.dll

(gdb) bt
#0  0x77c3554a in msvcrt!_abnormal_termination ()
   from C:\WINDOWS\system32\msvcrt.dll

(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x77c3552a to 0x77c3556a:
   0x77c3552a <msvcrt!_abnormal_termination+19>:        push   %ecx
   0x77c3552b <msvcrt!_abnormal_termination+20>:        or     $0x8b,%al
   0x77c3552d <msvcrt!_abnormal_termination+22>:        push   %edx
   0x77c3552e <msvcrt!_abnormal_termination+23>:        or     $0x39,%al
   0x77c35530 <msvcrt!_abnormal_termination+25>:        push   %ecx
   0x77c35531 <msvcrt!_abnormal_termination+26>:        or     %dh,0x5(%ebp)
   0x77c35534 <msvcrt!_abnormal_termination+29>:        mov    $0x1,%eax
   0x77c35539 <msvcrt!_abnormal_termination+34>:        ret
   0x77c3553a <msvcrt!_abnormal_termination+35>:        push   %ebx
   0x77c3553b <msvcrt!_abnormal_termination+36>:        push   %ecx
   0x77c3553c <msvcrt!_abnormal_termination+37>:        mov    $0x77c5f990,%ebx
   0x77c35541 <msvcrt!_abnormal_termination+42>:        jmp    0x77c3554d <msvcrt!_abnormal_termination+54>
   0x77c35543 <msvcrt!_abnormal_termination+44>:        push   %ebx
   0x77c35544 <msvcrt!_abnormal_termination+45>:        push   %ecx
   0x77c35545 <msvcrt!_abnormal_termination+46>:        mov    $0x77c5f990,%ebx
=> 0x77c3554a <msvcrt!_abnormal_termination+51>:        mov    0x8(%ebp),%ecx
   0x77c3554d <msvcrt!_abnormal_termination+54>:        mov    %ecx,0x8(%ebx)
   0x77c35550 <msvcrt!_abnormal_termination+57>:        mov    %eax,0x4(%ebx)
   0x77c35553 <msvcrt!_abnormal_termination+60>:        mov    %ebp,0xc(%ebx)
   0x77c35556 <msvcrt!_abnormal_termination+63>:        push   %ebp
   0x77c35557 <msvcrt!_abnormal_termination+64>:        push   %ecx
   0x77c35558 <msvcrt!_abnormal_termination+65>:        push   %eax
   0x77c35559 <msvcrt!_abnormal_termination+66>:        pop    %eax
   0x77c3555a <msvcrt!_abnormal_termination+67>:        pop    %ecx
   0x77c3555b <msvcrt!_abnormal_termination+68>:        pop    %ebp
   0x77c3555c <msvcrt!_abnormal_termination+69>:        pop    %ecx
   0x77c3555d <msvcrt!_abnormal_termination+70>:        pop    %ebx
   0x77c3555e <msvcrt!_abnormal_termination+71>:        ret    $0x4
   0x77c35561 <msvcrt!_abnormal_termination+74>:        int3
   0x77c35562 <msvcrt!_abnormal_termination+75>:        int3
   0x77c35563 <msvcrt!_abnormal_termination+76>:        int3
   0x77c35564 <msvcrt!_abnormal_termination+77>:        int3
   0x77c35565 <msvcrt!_abnormal_termination+78>:        int3
   0x77c35566 <msvcrt!_assert+0>:       mov    %edi,%edi
   0x77c35568 <msvcrt!_assert+2>:       push   %ebp
   0x77c35569 <msvcrt!_assert+3>:       mov    %esp,%ebp
End of assembler dump.

(gdb) info all-registers
eax            0x6ed86b2e       1859676974
ecx            0x77c39bc6       2009308102
edx            0xffffff01       -255
ebx            0x77c5f990       2009463184
esp            0x22f24c 0x22f24c
ebp            0x0      0x0
esi            0x22ffe0 2293728
edi            0x0      0
eip            0x77c3554a       0x77c3554a <msvcrt!_abnormal_termination+51>
eflags         0x10246  [ PF ZF IF RF ]
cs             0x1b     27
ss             0x23     35
ds             0x23     35
es             0x23     35
fs             0x3b     59
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            <invalid float value>    (raw 0x022200f6a7e800618c9c)
st3            -1       (raw 0xbfff8000000000000000)
st4            -1       (raw 0xbfff8000000000000000)
st5            9.9999999999999995e-021  (raw 0x3fbcbce5086492111aeb)
st6            23.976024167640553       (raw 0x4003bfcee5c240f97000)
st7            3.455309232230784e+018   (raw 0x403cbfcee5c240f97002)
fctrl          0xffff037f       -64641
fstat          0xffff0420       -64480
ftag           0xffffffff       -1
fiseg          0x1b     27
fioff          0x6f2f05c6       1865352646
foseg          0xffff0023       -65501
fooff          0x22f428 2290728
fop            0x77c    1916
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x3, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0x0, 0x0, 0x0,
    0x70, 0xf8, 0x22, 0x0}, v8_int16 = {0x103, 0x0, 0x0, 0x0, 0xff, 0x0,
    0xf870, 0x22}, v4_int32 = {0x103, 0x0, 0xff, 0x22f870}, v2_int64 = {
    0x103, 0x22f870000000ff}, uint128 = 0x0022f870000000ff0000000000000103}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x3, 0x1, 0x0, 0x0, 0x3b, 0x0, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0,
    0x6c, 0xf0, 0x22, 0x0}, v8_int16 = {0x103, 0x0, 0x3b, 0x0, 0x1, 0x0,
    0xf06c, 0x22}, v4_int32 = {0x103, 0x3b, 0x1, 0x22f06c}, v2_int64 = {
    0x3b00000103, 0x22f06c00000001},
  uint128 = 0x0022f06c000000010000003b00000103}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x3c, 0xa3, 0x61, 0x0, 0x98, 0xe4, 0x22, 0x0, 0x0, 0x1, 0x0,
    0x0, 0xe0, 0xff, 0x22, 0x0}, v8_int16 = {0xa33c, 0x61, 0xe498, 0x22,
    0x100, 0x0, 0xffe0, 0x22}, v4_int32 = {0x61a33c, 0x22e498, 0x100,
    0x22ffe0}, v2_int64 = {0x22e4980061a33c, 0x22ffe000000100},
  uint128 = 0x0022ffe0000001000022e4980061a33c}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x2c, 0xed, 0x22, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
    0xdc, 0xf0, 0x22, 0x0}, v8_int16 = {0xed2c, 0x22, 0x0, 0x0, 0x0, 0x0,
    0xf0dc, 0x22}, v4_int32 = {0x22ed2c, 0x0, 0x0, 0x22f0dc}, v2_int64 = {
    0x22ed2c, 0x22f0dc00000000}, uint128 = 0x0022f0dc00000000000000000022ed2c}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x9c, 0x8c, 0x61, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
    0xd4, 0xee, 0x22, 0x0}, v8_int16 = {0x8c9c, 0x61, 0x0, 0x0, 0x0, 0x0,
    0xeed4, 0x22}, v4_int32 = {0x618c9c, 0x0, 0x0, 0x22eed4}, v2_int64 = {
    0x618c9c, 0x22eed400000000}, uint128 = 0x0022eed4000000000000000000618c9c}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0,
    0x8000000000000000}, v16_int8 = {0x47, 0x95, 0x61, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0, 0x0, 0x0, 0x20, 0xe9, 0x90, 0x7c}, v8_int16 = {0x9547,
    0x61, 0x0, 0x0, 0x0, 0x0, 0xe920, 0x7c90}, v4_int32 = {0x619547, 0x0,
    0x0, 0x7c90e920}, v2_int64 = {0x619547, 0x7c90e92000000000},
  uint128 = 0x7c90e920000000000000000000619547}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x3, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
    0xf0, 0xf0, 0x22, 0x0}, v8_int16 = {0x103, 0x0, 0x0, 0x0, 0x0, 0x0,
    0xf0f0, 0x22}, v4_int32 = {0x103, 0x0, 0x0, 0x22f0f0}, v2_int64 = {0x103,
    0x22f0f000000000}, uint128 = 0x0022f0f0000000000000000000000103}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x9c, 0x8c, 0x61, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0x0, 0x0, 0x0,
    0x70, 0xf8, 0x22, 0x0}, v8_int16 = {0x8c9c, 0x61, 0x0, 0x0, 0xff, 0x0,
    0xf870, 0x22}, v4_int32 = {0x618c9c, 0x0, 0xff, 0x22f870}, v2_int64 = {
    0x618c9c, 0x22f870000000ff}, uint128 = 0x0022f870000000ff0000000000618c9c}
mxcsr          0x1f80   [ IM DM ZM OM UM PM ]
mm0            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm1            {uint64 = 0x0, v2_int32 = {0x0, 0x0}, v4_int16 = {0x0, 0x0,
    0x0, 0x0}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}
mm2            {uint64 = 0xf6a7e800618c9c, v2_int32 = {0x618c9c, 0xf6a7e8},
  v4_int16 = {0x8c9c, 0x61, 0xa7e8, 0xf6}, v8_int8 = {0x9c, 0x8c, 0x61, 0x0,
    0xe8, 0xa7, 0xf6, 0x0}}
mm3            {uint64 = 0x8000000000000000, v2_int32 = {0x0, 0x80000000},
  v4_int16 = {0x0, 0x0, 0x0, 0x8000}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x80}}
mm4            {uint64 = 0x8000000000000000, v2_int32 = {0x0, 0x80000000},
  v4_int16 = {0x0, 0x0, 0x0, 0x8000}, v8_int8 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x80}}
mm5            {uint64 = 0xbce5086492111aeb, v2_int32 = {0x92111aeb,
    0xbce50864}, v4_int16 = {0x1aeb, 0x9211, 0x864, 0xbce5}, v8_int8 = {0xeb,
    0x1a, 0x11, 0x92, 0x64, 0x8, 0xe5, 0xbc}}
mm6            {uint64 = 0xbfcee5c240f97000, v2_int32 = {0x40f97000,
    0xbfcee5c2}, v4_int16 = {0x7000, 0x40f9, 0xe5c2, 0xbfce}, v8_int8 = {0x0,
    0x70, 0xf9, 0x40, 0xc2, 0xe5, 0xce, 0xbf}}
mm7            {uint64 = 0xbfcee5c240f97002, v2_int32 = {0x40f97002,
    0xbfcee5c2}, v4_int16 = {0x7002, 0x40f9, 0xe5c2, 0xbfce}, v8_int8 = {0x2,
    0x70, 0xf9, 0x40, 0xc2, 0xe5, 0xce, 0xbf}}

(gdb)

A comparison between files using mkvinfo -v, stuff written by x264 (files that cause the crash) lack Segment size information (as 'size unknown'), Seek head, EbmlVoid data, and the final Cues entry. Both the mkvmerge and libavformat test files (which work correctly) have those entries intact and the Segment size actually contains a numeric value.



On a partially related note, a patch that I cobbled together for the C plugin that allows the user to more easily select the optimization level during configure.
https://github.com/qyot27/ffms2-cint...923d369af65fca

[cjplay]

Just a quick thank you. I downloaded the build for 666 from a few posts up and wish to thank the developers for fixing an issue I've been having with ProResHQ 10-bit decoding. I don't know which build it was, but it was after 644, so thank you.

-CJ

PS - Sign of the beast is the build that works??? Holy !

[JEEB]

For the RGB-in-H.264 folk another build. Available here.

ffms2: r671, libav: git-0aaa45e with D404's related backport (web view).

libav configuration:
--prefix=/ffms --disable-network --disable-encoders --disable-muxers --disable-hwaccels --disable-indevs --disable-outdevs --extra-cflags='-U__STRICT_ANSI__' --disable-debug --enable-runtime-cpudetect

If this backport doesn't get into mainline libav soon enough I will herp a derp.

[Yellow_]

Originally Posted by TheFluff

Edit: for reference, FFMS2 now behaves like this: if the input pixel format is one of the YUVJ* ones, the video is assumed to be fullrange, regardless of what the CodecContext->color_range metadata flag might say. For all other pixel formats, FFMS2 uses whatever the CodecContext->color_range flag says. If that's unspecified, limited range is assumed.

What are the characteristics of a yuvj420p source compared to yuv420p? Is yuvj JFIF? Is chroma placement different? Is chroma normalised over 0 - 255 along with luma? I'm struggling to find any decent info on what yuvj is.

If yuvj420p is simply deemed full range luma 4:2:0 progressive because it's flagged 'fullrange' so then is it no different from yuv420p with full luma unflagged that makes up a big proportion of YCC certainly video off cameras. Is it just suggested differing handling for playback, if the flag's honoured at all.

The result I see importing flagged full range YCC into an NLE's for example is it treats it as 0 - 255 levels, scale luma to 16 - 235 so it can then expand again to 0 - 255 RGB for color processing.

For avisynth if it's just a luma thing then makes no difference if it's yuvj420p or yuv420p as long as it's not scaled at decompression? Am I correct in thinking that?

[kemuri-_9]

my understanding is that the J stands for "JPEG" which uses pc/full range in comparison to the standard ones which are tv/limited range.

you seem to be thinking that the range of values only applies to luma, but it actually applies to chroma as well...

[Yellow_]

Originally Posted by kemuri-_9

my understanding is that the J stands for "JPEG" which uses pc/full range in comparison to the standard ones which are tv/limited range.

Yes, that's my understanding too, but only with regard to playback handling, surely unless specifically requested there should be no messing with levels just transcoding but there is. We've all seen sources with levels outside of 16 - 235/240. XDCAM off a $20,000 Canon C300 for example. It's not flagged as such it's not h264. ffmpeg / CS5 treat the XDCAM as yuv420p but it still has full luma, 16 -240 chroma.

An example, this native Canon DSLR source file to me is just YCbCr 4:2:0, it uses a BT601 color matrix and BT709 transfer. It has luma that goes above 235 and chroma that stays well inside 16 - 240 even as saturated yellow/gold as it is:

http://www.yellowspace.webspace.virg....com/Gold.aMOV

ffmpeg gives it a pixel format of yuvj420p. Adobe Premiere CS5 treats it as full range so scales luma on decompression for the YCC to RGB conversion. But the chroma is 16 - 240 so can it still be jpeg / JFIF, can chroma be normalised this way so it appears to be just typical yuv420p?.

If I remux the Gold.MOV above with komisars patched build of MP4Box and turn off the fullrange flag it's handled by CS5 and ffmpeg as yuv420p not yuvj. The files histogram look better in the conversion to RGB treated as yuv420p and forcing ffmpeg to use full luma than it does just as yuvj420p or as yuvj420p and forcing full luma.

So this is my query, am I missing something, is chroma placement different ie: centre for jpeg vs left for mpeg or simply is it because it's flagged fullrange in the h264 VUI Option? By the camera firmware simply to suggest playback handling as it's perhaps considered a 'consumer' camera and h264 gives possibility to flag it.

you seem to be thinking that the range of values only applies to luma, but it actually applies to chroma as well...

Actually no I'm not under that impression. I'm aware of it, for example xvYCC is over full luma and chroma and JFIF has luma and chroma normalised over full range if I understand correctly, this is my point I think, why yuvj420p if the source files are not full chroma, even well saturated. ColorYUV(analyze=true) gives no chroma outside 16 - 240 with any of my yuvj420p deemed source files, that's using ffmpegsource2 or QTInput. Is this some dumb ffmpeg behaviour over the use of the fullrange flag?

Or I have I misinterpreted you answer, do you mean chroma is scaled as well as luma when decompressing yuvj?

Thanks for the reply.

[qyot27]

Using komisar's gcc 4.6.2 package or the i686-w64-mingw32-gcc 4.6.3 provided by Ubuntu 12.04 results in the C plugin being utterly broken. It probably affects distributions of MinGW-w64's gcc by other sources, and I wouldn't be at all surprised if it has to do with how gcc itself was compiled, but trying to work around this is driving me batty. The 4.6.1 in 11.10 didn't have any of this mess. On that same token, FFmpeg and x264 also get wonky with >4.6.1, but that can be adjusted with much of the same method (I physically moved all *.dll.a files in /usr/lib/gcc/i686-w64-mingw32/4.6 to a subdirectory so it wouldn't try to use them). The trunk seems to be entirely unaffected, as it loaded video fine as part of x264, provided the aforementioned moving of *.dll.a stuff first so that those builds don't require extra .dlls.

With komisar's 4.6.2, it can be fairly easily circumvented by passing -static-libgcc -static-libstdc++ to --extra-ldflags, and seems to work fine. Unfortunately, my attempts on Ubuntu at inserting a version check into configure that ensures these options are used for gcc >=4.6 resulted in getting ffmsindex-related link errors when both options were specified together (the library seemed to build right, except that I didn't actually test whether it worked; it compiled, which was as far as I went). The check seemed to be okay, but ffmsindex did not like the use of -static-libstdc++ - leaving it out isn't an option, though, since it causes annoying external dependency issues and if those are accounted for, I still ran into it causing Access Violation errors in AviSynth when trying to load scripts using the plugin. I didn't try to dig enough into how to get -static-libstdc++ to be used for the library only, and omitted from ffmsindex's link step.

EDIT: Yeah, so I screwed that description up. It's -static-libgcc that ffmsindex doesn't like, not static-libstdc++.

EDIT 2012-05-21: Finally resolved by recompiling the MinGW-w64 environment with Zeranoe's build script. It also confirmed that it was the shared crap that was screwing up the repo versions, as I made sure to have it build fully static. The resultant builds were fine, just like they were before. And it only took 6 hours to get set up.

[JEEB]

There've been some additional commits over at ffms2, so a new build (also the RGB-in-H.264 stuff got put into libav as well quite some time ago). Available here.

ffms2: r680, libav: git-f132248

libav configuration:
--prefix=/ffms --disable-network --disable-encoders --disable-muxers --disable-hwaccels --disable-indevs --disable-outdevs --extra-cflags='-U__STRICT_ANSI__' --disable-debug --enable-runtime-cpudetect

[Reel.Deel]

Thank you very much JEEB!

[jmac698]

Did interlaced h.264 get fixed yet?

[JEEB]

Originally Posted by jmac698

Did interlaced h.264 get fixed yet?

I don't think anything has gotten fixed in relation to interlaced H.264 in ffms, but when I tested my build in Aegisub with a random "interlaced H.264 in m2ts" clip, I seem to get a very "stable/slow" way of seeking at least (seems to be gradually decoded from the beginning to the frame I asked), and the output seems to be fine

Your Mileage May Vary

[tebasuna51]

Originally Posted by jmac698

Did interlaced h.264 get fixed yet?

Same problem with this sample:

Originally Posted by sneaker_ger

The following interlaced H.264 sample will not work correctly in ffms2 (tested 2.17, r644, r666):
http://www.mediafire.com/?ubxfydr4owr2rmo

1 thread: jumping back and forth

[burfadel]

Originally Posted by JEEB

There've been some additional commits over at ffms2, so a new build (also the RGB-in-H.264 stuff got put into libav as well quite some time ago). Available here.

Thanks for the new build, much appreciated!

[Skauneboy]

Could any kind soul please compile a x64 build? Thanks!

[rogerdpack]

I assume FFMS2 can't be used for live unindexed capture sources, like
FFVideoSource("-f dshow -i video=screen-capture-recorder")
If it could some day that would might be kind
Thanks much.
-roger-

[Plorkyeran]

Why not just use DirectShowSource there? There are potential usecases for FFMS with no index and no seeking allowed, but that doesn't appear to be one of them.

[rogerdpack]

I am not a pro at it, but it appears DirectShowSource your only option is a .grf file, and I don't know how to easily make a .grf file specify "which size and color format" you want out of the pins, etc. But I haven't looked into it too closely yet.

[TheRyuu]

Originally Posted by Skauneboy

Could any kind soul please compile a x64 build? Thanks!

ffms2-r683.7z

Built with ffmpeg rev. d1384c0
Contains both 32 and 64-bit binaries.