FFmpegSource

[AMED]

I'm currently using FFMS v579 with MeGUI and i have noticed some random blocking when encoding a bluray VC1 stream in MKV.



These blocks in Frame 947 do actually come out on the final encode as well.

General
Unique ID                      : 151178532468450782821249184430171641178 (0x71BBED1F9198E6CA80E93BBF4664555A)
Complete name                  : D:\backup\CINDERELLA_MAN\CINDERELLA_MAN.mkv
Format                         : Matroska
Format version                 : Version 1
File size                      : 24.7 GiB
Duration                       : 2h 24mn
Overall bit rate               : 24.5 Mbps
Encoded date                   : UTC 2011-10-23 01:15:30
Writing application            : eac3to
Writing library                : Haali DirectShow Matroska Muxer 1.11.96.14

Video
ID                             : 1
Format                         : VC-1
Format profile                 : AP@L3
Codec ID                       : WVC1
Codec ID/Hint                  : Microsoft
Duration                       : 2h 24mn
Bit rate                       : 24.0 Mbps
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate                     : 23.976 fps
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.483
Stream size                    : 24.2 GiB (98%)

Uploading a sample right now....

Sample
http://www.mediafire.com/?8cy5i8179hunmcy

MD5: 604190103a687e07bddd243d83ec0295
SHA1: 1b1eee32e5e8a765de0b4980ba0a06d57f4766c2

[TheRyuu]

Originally Posted by forclip

Can someone explain me please, what does it means:

"and is valid" - how it is calculating that the index-file is (still) valid? I'm asking this because when I use FFAudioSource like this (because of this):

FFIndex("my_file")
FFAudioSource("my_file")

everything is OK for the first time. But after replacing my_file with another file without changing its name and without deleting the index-file, FFIndex still returns 0 and don't re-indexing this (totally different!) file. So FFIndex can't see the difference.. However, FFAudioSource can see the difference and re-creating the index, but due to "error handling mode 1" it is useless for some kind of files..

Why FFIndex can't see that the source-file was changed?

(Probably) Fixed in r580.

Give it a shot: ffms2-r580.7z
Built with libav ec6d743.

[forclip]

Originally Posted by TheRyuu

(Probably) Fixed in r580.

Probably - yes . But this fix also must be added to ffmsindex.cpp I guess, so ffmsindex.exe will check the index file the same way?

[TheRyuu]

Originally Posted by forclip

Probably - yes . But this fix also must be added to ffmsindex.cpp I guess, so ffmsindex.exe will check the index file the same way?

ffmsindex will always throw an exception if the index file already exists.

[forclip]

Originally Posted by TheRyuu

ffmsindex will always throw an exception if the index file already exists.

How about adding an option to change this behavior and check that this index file is "valid" for the source file?

Also, I must notice that after r580 the FFmpegSource2() function, when "cachefile" is specified, is working not as expected acordingly to

The arguments do the same thing as in FFVideoSource and FFAudioSource; see those functions for details

As expected is: since "cachefile" was specified, this function will throw an error if this index isn't belong to the source file. But instead this index file will be overwritten - I want this behavior for FFIndex(), and not for FFmpegSource2(), but since call to FFIndex is part of FFmpegSource2(), we get what we get.

So again. How about adding an option to ffmsindex.exe and to FFIndex, that will turn on\off this "file signature check" (off = everything works as before r580)? And then FFMS2.avsi can be modified to call FFIndex without this check, so FFmpegSource2() will work as before r580. At least for me with my workflow I would like to have such an option for ffmsindex and for ffindex..

Also the first call of FFindex in FFMS2.avsi seems to missing the "utf8=utf8" atribute (at line 31 column 40).

[pandv2]

About this:

add sanity check for cases where the video decoder returns an empty frame.
"fixes" issue #50 by virtue of not crashing anymore, but just returning a nice
error message instead.

I get this error in a video (in 2 frames), but previously (2.15 version) the same video (same file, not touched) decodes ok. FFMS2 doesn't crash. I can't say if there are frame corruption or not (because it's a unattended process) but not error and not crash.

I am using the library. Not avisynth.

______________________________________________________

With the lastest beta r580, if I try to set the output format (with FFMS_SetOutputFormatV or FFMS_SetOutputFormatV2) to any planar format (I tested all) the program crashes with a memory violation (trying to write in protected area). I tested with non planar formats (rgb & yuv) and it worked.

[TheFluff]

Originally Posted by pandv2

About this:

add sanity check for cases where the video decoder returns an empty frame.
"fixes" issue #50 by virtue of not crashing anymore, but just returning a nice
error message instead.

I get this error in a video (in 2 frames), but previously (2.15 version) the same video (same file, not touched) decodes ok. FFMS2 doesn't crash. I can't say if there are frame corruption or not (because it's a unattended process) but not error and not crash.

I am using the library. Not avisynth.

That's not good at all and sounds like a regression in either FFMS2 or libavcodec. Can you provide a sample file?

Originally Posted by pandv2

With the lastest beta r580, if I try to set the output format (with FFMS_SetOutputFormatV or FFMS_SetOutputFormatV2) to any planar format (I tested all) the program crashes with a memory violation (trying to write in protected area). I tested with non planar formats (rgb & yuv) and it worked.

That's probably some silly typo somewhere. I'll take a look.

[MatLz]

On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...),
this script does show something wrong :

a=mpeg2source("v.d2v") # or avisource("v_uncompressed_by_virtualdub.avi")
b=ffvideosource("v.mkv") # muxing the stream in a mkv does fix the ffms2 frame inaccuracy
subtract(a,b)
histogram("luma") # needed for humans eyes

I remember when testing some ffdshow settings, I did try the "use speedup tricks" and got some funky pixels like that (but much more visible).
Dunno if it is related since both ffms2 and ffdshow use ffmpeg.

[jmac698]

I'm fairly new to using ffms2, but I just discovered something simple that seems impossible: I can't open a 5.1 and make it dolby stereo. There is no way to downmix to DPL/2. Also can you add a simple feature in the PP string for contrast adjust on import, cont:100
You have functions to support different colorspaces so don't ignore audio.

[arestarh]

Originally Posted by AMED

I'm currently using FFMS v579 with MeGUI and i have noticed some random blocking when encoding a bluray VC1 stream in MKV.



These blocks in Frame 947 do actually come out on the final encode as well.

General
Unique ID                      : 151178532468450782821249184430171641178 (0x71BBED1F9198E6CA80E93BBF4664555A)
Complete name                  : D:\backup\CINDERELLA_MAN\CINDERELLA_MAN.mkv
Format                         : Matroska
Format version                 : Version 1
File size                      : 24.7 GiB
Duration                       : 2h 24mn
Overall bit rate               : 24.5 Mbps
Encoded date                   : UTC 2011-10-23 01:15:30
Writing application            : eac3to
Writing library                : Haali DirectShow Matroska Muxer 1.11.96.14

Video
ID                             : 1
Format                         : VC-1
Format profile                 : AP@L3
Codec ID                       : WVC1
Codec ID/Hint                  : Microsoft
Duration                       : 2h 24mn
Bit rate                       : 24.0 Mbps
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate                     : 23.976 fps
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.483
Stream size                    : 24.2 GiB (98%)

Uploading a sample right now....

Sample
http://www.mediafire.com/?8cy5i8179hunmcy

MD5: 604190103a687e07bddd243d83ec0295
SHA1: 1b1eee32e5e8a765de0b4980ba0a06d57f4766c2

Same issue. Does anyone can confirm this ?
I used build 579.
Thanks.

[TheFluff]

Originally Posted by jmac698

I'm fairly new to using ffms2, but I just discovered something simple that seems impossible: I can't open a 5.1 and make it dolby stereo. There is no way to downmix to DPL/2. Also can you add a simple feature in the PP string for contrast adjust on import, cont:100
You have functions to support different colorspaces so don't ignore audio.

FFMS2 does not support any channel mixing operations at all, because libavcodec doesn't support that sort of stuff either. Adding some simple audio processing options such as resampling, bitdepth conversion and channel mixing (either via an external library or via our own code) has been on the todo list for a long time. The reason we support colorspace conversion and resizing is that ffmpeg comes with libswscale.

Video postprocessing is mostly the same thing, it's provided via libpostproc and we don't have any control over what features it supports. Not that it matters, because the ffmpeg devs hate it and are going to remove it, and thus so are we.

[TheFluff]

Originally Posted by MatLz

On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...),
this script does show something wrong :

a=mpeg2source("v.d2v") # or avisource("v_uncompressed_by_virtualdub.avi")
b=ffvideosource("v.mkv") # muxing the stream in a mkv does fix the ffms2 frame inaccuracy
subtract(a,b)
histogram("luma") # needed for humans eyes

I remember when testing some ffdshow settings, I did try the "use speedup tricks" and got some funky pixels like that (but much more visible).
Dunno if it is related since both ffms2 and ffdshow use ffmpeg.

If that's supposed to be a bug report I'll have to say it's a remarkably bad one. At least have the decency to include a sample file and a screenshot.

That being said, AFAIK MPEG2 decoders are not required to have bit-exact output so it's not entirely surprising that two different decoders could have subtly different outputs.

[MatLz]

Originally Posted by TheFluff

If that's supposed to be a bug report I'll have to say it's a remarkably bad one. At least have the decency to include a sample file and a screenshot.

Originally Posted by MatLz

On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...)

Originally Posted by TheFluff

That being said, AFAIK MPEG2 decoders are not required to have bit-exact output so it's not entirely surprising that two different decoders could have subtly different outputs.

Dunno... I thought they should output identical thing, like avc ones do.

[TheRyuu]

Originally Posted by MatLz

Dunno... I thought they should output identical thing, like avc ones do.

iDCT bro.

[TheFluff]

Originally Posted by MatLz

Dunno... I thought they should output identical thing, like avc ones do.

MPEG2 does not specify exactly how the IDCT algorithm should work. Bit-exact decoding is not required. Ever noticed how you can choose between like five different IDCT algorithms in DVD2AVI? Yeah, all of those give different outputs.

See also http://guru.multimedia.cx/the-mpeg124-and-h26123-idct/

[LigH]

Something similar (but not related to iDCT) happened between DivX 5.02 and 5.03 too, probably a change in the rounding (Oct. 2003). That resulted in what I called "orange-blue bug". But quite soon after I reported it (March 2004), ffdshow was able to adapt its decoding.

[TheFluff]

Originally Posted by LigH

The MeGUI thread discusses an issue with FFindex which crashes on uncompressed YUY2 / YV12 videos — not only with dozens of GB in size.

I don't think you ever got an official reply to this, but Chikuzen is correct; FFMS2 does not support uncompressed formats (because libavcodec handles them weirdly). Crashes are possible. Indexing such files should work just fine, but when I checked the MeGUI threa I did not see any indications that it's the indexing process itself that is crashing.

[TheFluff]

Originally Posted by LigH

Something similar (but not related to iDCT) happened between DivX 5.02 and 5.03 too, probably a change in the rounding (Oct. 2003). That resulted in what I called "orange-blue bug". But quite soon after I reported it (March 2004), ffdshow was able to adapt its decoding.

Since you apparently need histogram("luma") to even see the differences MatLz is speaking of, it does not appear to be an issue of anywhere near that magnitude.

[MatLz]

The real visible things without enhancing the luma differences are mostly in flat areas.

Thx for the explanations.

Btw, could be possible to easily implement the ffms2 idct algorithm in mpeg2source ?
Using 1 thread (to be fair) does show it is 20% faster.

[TheRyuu]

Originally Posted by MatLz

The real visible things without enhancing the luma differences are mostly in flat areas.

Thx for the explanations.

Btw, could be possible to easily implement the ffms2 idct algorithm in mpeg2source ?
Using 1 thread (to be fair) does show it is 20% faster.

DGDecode or Mpeg2Dec3 are the recommended methods for sourcing mpeg2 material. For VOBS and transport streams I would trust them more than ffms2.

You get something like 6-7 different iDCT options with both DGDecode and Mpeg2Dec3. They both default to whatever is found in the d2v file but can be overridden in avisynth. DGIndex defaults to Skal SSE and DVD2AVI defaults to SSE2MMX. The one most like libav's default is probably Simple iDCT although that's just a guess based on libav's naming of a constant.

Mpeg2Dec3 is normally faster than DGDecode especially when seeking. I prefer DVD2AVI's default choice (with mpeg2dec3) for all things mpeg2. YMMV and opinions also.