Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th October 2011, 09:08   #1341  |  Link
AMED
Registered User
 
AMED's Avatar
 
Join Date: Sep 2004
Location: Auckland, New Zealand
Posts: 466
I'm currently using FFMS v579 with MeGUI and i have noticed some random blocking when encoding a bluray VC1 stream in MKV.



These blocks in Frame 947 do actually come out on the final encode as well.

Code:
General
Unique ID                      : 151178532468450782821249184430171641178 (0x71BBED1F9198E6CA80E93BBF4664555A)
Complete name                  : D:\backup\CINDERELLA_MAN\CINDERELLA_MAN.mkv
Format                         : Matroska
Format version                 : Version 1
File size                      : 24.7 GiB
Duration                       : 2h 24mn
Overall bit rate               : 24.5 Mbps
Encoded date                   : UTC 2011-10-23 01:15:30
Writing application            : eac3to
Writing library                : Haali DirectShow Matroska Muxer 1.11.96.14

Video
ID                             : 1
Format                         : VC-1
Format profile                 : AP@L3
Codec ID                       : WVC1
Codec ID/Hint                  : Microsoft
Duration                       : 2h 24mn
Bit rate                       : 24.0 Mbps
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate                     : 23.976 fps
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.483
Stream size                    : 24.2 GiB (98%)
Uploading a sample right now....

Sample
http://www.mediafire.com/?8cy5i8179hunmcy

MD5: 604190103a687e07bddd243d83ec0295
SHA1: 1b1eee32e5e8a765de0b4980ba0a06d57f4766c2
__________________
A Man Eating Duck

Last edited by AMED; 28th October 2011 at 09:50.
AMED is offline   Reply With Quote
Old 28th October 2011, 10:10   #1342  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by forclip View Post
Can someone explain me please, what does it means:

"and is valid" - how it is calculating that the index-file is (still) valid? I'm asking this because when I use FFAudioSource like this (because of this):
Code:
FFIndex("my_file")
FFAudioSource("my_file")
everything is OK for the first time. But after replacing my_file with another file without changing its name and without deleting the index-file, FFIndex still returns 0 and don't re-indexing this (totally different!) file. So FFIndex can't see the difference.. However, FFAudioSource can see the difference and re-creating the index, but due to "error handling mode 1" it is useless for some kind of files..

Why FFIndex can't see that the source-file was changed?
(Probably) Fixed in r580.

Give it a shot: ffms2-r580.7z
Built with libav ec6d743.
TheRyuu is offline   Reply With Quote
Old 28th October 2011, 10:41   #1343  |  Link
forclip
Registered User
 
Join Date: Dec 2009
Posts: 63
Quote:
Originally Posted by TheRyuu View Post
(Probably) Fixed in r580.
Probably - yes . But this fix also must be added to ffmsindex.cpp I guess, so ffmsindex.exe will check the index file the same way?
forclip is offline   Reply With Quote
Old 29th October 2011, 03:48   #1344  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by forclip View Post
Probably - yes . But this fix also must be added to ffmsindex.cpp I guess, so ffmsindex.exe will check the index file the same way?
ffmsindex will always throw an exception if the index file already exists.
TheRyuu is offline   Reply With Quote
Old 29th October 2011, 14:04   #1345  |  Link
forclip
Registered User
 
Join Date: Dec 2009
Posts: 63
Quote:
Originally Posted by TheRyuu View Post
ffmsindex will always throw an exception if the index file already exists.
How about adding an option to change this behavior and check that this index file is "valid" for the source file?

Also, I must notice that after r580 the FFmpegSource2() function, when "cachefile" is specified, is working not as expected acordingly to
Quote:
The arguments do the same thing as in FFVideoSource and FFAudioSource; see those functions for details
As expected is: since "cachefile" was specified, this function will throw an error if this index isn't belong to the source file. But instead this index file will be overwritten - I want this behavior for FFIndex(), and not for FFmpegSource2(), but since call to FFIndex is part of FFmpegSource2(), we get what we get.

So again. How about adding an option to ffmsindex.exe and to FFIndex, that will turn on\off this "file signature check" (off = everything works as before r580)? And then FFMS2.avsi can be modified to call FFIndex without this check, so FFmpegSource2() will work as before r580. At least for me with my workflow I would like to have such an option for ffmsindex and for ffindex..

Also the first call of FFindex in FFMS2.avsi seems to missing the "utf8=utf8" atribute (at line 31 column 40).
forclip is offline   Reply With Quote
Old 30th October 2011, 19:10   #1346  |  Link
pandv2
Registered User
 
Join Date: Sep 2011
Posts: 34
About this:

add sanity check for cases where the video decoder returns an empty frame.
"fixes" issue #50 by virtue of not crashing anymore, but just returning a nice
error message instead.

I get this error in a video (in 2 frames), but previously (2.15 version) the same video (same file, not touched) decodes ok. FFMS2 doesn't crash. I can't say if there are frame corruption or not (because it's a unattended process) but not error and not crash.

I am using the library. Not avisynth.

______________________________________________________

With the lastest beta r580, if I try to set the output format (with FFMS_SetOutputFormatV or FFMS_SetOutputFormatV2) to any planar format (I tested all) the program crashes with a memory violation (trying to write in protected area). I tested with non planar formats (rgb & yuv) and it worked.

Last edited by pandv2; 30th October 2011 at 21:04. Reason: Another possible bug found
pandv2 is offline   Reply With Quote
Old 31st October 2011, 03:08   #1347  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by pandv2 View Post
About this:

add sanity check for cases where the video decoder returns an empty frame.
"fixes" issue #50 by virtue of not crashing anymore, but just returning a nice
error message instead.

I get this error in a video (in 2 frames), but previously (2.15 version) the same video (same file, not touched) decodes ok. FFMS2 doesn't crash. I can't say if there are frame corruption or not (because it's a unattended process) but not error and not crash.

I am using the library. Not avisynth.
That's not good at all and sounds like a regression in either FFMS2 or libavcodec. Can you provide a sample file?

Quote:
Originally Posted by pandv2 View Post
With the lastest beta r580, if I try to set the output format (with FFMS_SetOutputFormatV or FFMS_SetOutputFormatV2) to any planar format (I tested all) the program crashes with a memory violation (trying to write in protected area). I tested with non planar formats (rgb & yuv) and it worked.
That's probably some silly typo somewhere. I'll take a look.
TheFluff is offline   Reply With Quote
Old 31st October 2011, 21:04   #1348  |  Link
MatLz
I often say "maybe"...
 
MatLz's Avatar
 
Join Date: Jul 2009
Location: France
Posts: 583
On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...),
this script does show something wrong :
Code:
a=mpeg2source("v.d2v") # or avisource("v_uncompressed_by_virtualdub.avi")
b=ffvideosource("v.mkv") # muxing the stream in a mkv does fix the ffms2 frame inaccuracy
subtract(a,b)
histogram("luma") # needed for humans eyes

I remember when testing some ffdshow settings, I did try the "use speedup tricks" and got some funky pixels like that (but much more visible).
Dunno if it is related since both ffms2 and ffdshow use ffmpeg.
MatLz is offline   Reply With Quote
Old 31st October 2011, 22:07   #1349  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,862
I'm fairly new to using ffms2, but I just discovered something simple that seems impossible: I can't open a 5.1 and make it dolby stereo. There is no way to downmix to DPL/2. Also can you add a simple feature in the PP string for contrast adjust on import, cont:100
You have functions to support different colorspaces so don't ignore audio.
jmac698 is offline   Reply With Quote
Old 1st November 2011, 12:17   #1350  |  Link
arestarh
Registered User
 
arestarh's Avatar
 
Join Date: Nov 2008
Location: Ukraine
Posts: 34
Quote:
Originally Posted by AMED View Post
I'm currently using FFMS v579 with MeGUI and i have noticed some random blocking when encoding a bluray VC1 stream in MKV.



These blocks in Frame 947 do actually come out on the final encode as well.

Code:
General
Unique ID                      : 151178532468450782821249184430171641178 (0x71BBED1F9198E6CA80E93BBF4664555A)
Complete name                  : D:\backup\CINDERELLA_MAN\CINDERELLA_MAN.mkv
Format                         : Matroska
Format version                 : Version 1
File size                      : 24.7 GiB
Duration                       : 2h 24mn
Overall bit rate               : 24.5 Mbps
Encoded date                   : UTC 2011-10-23 01:15:30
Writing application            : eac3to
Writing library                : Haali DirectShow Matroska Muxer 1.11.96.14

Video
ID                             : 1
Format                         : VC-1
Format profile                 : AP@L3
Codec ID                       : WVC1
Codec ID/Hint                  : Microsoft
Duration                       : 2h 24mn
Bit rate                       : 24.0 Mbps
Width                          : 1 920 pixels
Height                         : 1 080 pixels
Display aspect ratio           : 16:9
Frame rate                     : 23.976 fps
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Compression mode               : Lossy
Bits/(Pixel*Frame)             : 0.483
Stream size                    : 24.2 GiB (98%)
Uploading a sample right now....

Sample
http://www.mediafire.com/?8cy5i8179hunmcy

MD5: 604190103a687e07bddd243d83ec0295
SHA1: 1b1eee32e5e8a765de0b4980ba0a06d57f4766c2
Same issue. Does anyone can confirm this ?
I used build 579.
Thanks.
arestarh is offline   Reply With Quote
Old 1st November 2011, 14:57   #1351  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by jmac698 View Post
I'm fairly new to using ffms2, but I just discovered something simple that seems impossible: I can't open a 5.1 and make it dolby stereo. There is no way to downmix to DPL/2. Also can you add a simple feature in the PP string for contrast adjust on import, cont:100
You have functions to support different colorspaces so don't ignore audio.
FFMS2 does not support any channel mixing operations at all, because libavcodec doesn't support that sort of stuff either. Adding some simple audio processing options such as resampling, bitdepth conversion and channel mixing (either via an external library or via our own code) has been on the todo list for a long time. The reason we support colorspace conversion and resizing is that ffmpeg comes with libswscale.

Video postprocessing is mostly the same thing, it's provided via libpostproc and we don't have any control over what features it supports. Not that it matters, because the ffmpeg devs hate it and are going to remove it, and thus so are we.
TheFluff is offline   Reply With Quote
Old 1st November 2011, 15:00   #1352  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by MatLz View Post
On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...),
this script does show something wrong :
Code:
a=mpeg2source("v.d2v") # or avisource("v_uncompressed_by_virtualdub.avi")
b=ffvideosource("v.mkv") # muxing the stream in a mkv does fix the ffms2 frame inaccuracy
subtract(a,b)
histogram("luma") # needed for humans eyes

I remember when testing some ffdshow settings, I did try the "use speedup tricks" and got some funky pixels like that (but much more visible).
Dunno if it is related since both ffms2 and ffdshow use ffmpeg.
If that's supposed to be a bug report I'll have to say it's a remarkably bad one. At least have the decency to include a sample file and a screenshot.

That being said, AFAIK MPEG2 decoders are not required to have bit-exact output so it's not entirely surprising that two different decoders could have subtly different outputs.
TheFluff is offline   Reply With Quote
Old 1st November 2011, 15:14   #1353  |  Link
MatLz
I often say "maybe"...
 
MatLz's Avatar
 
Join Date: Jul 2009
Location: France
Posts: 583
Quote:
Originally Posted by TheFluff View Post
If that's supposed to be a bug report I'll have to say it's a remarkably bad one. At least have the decency to include a sample file and a screenshot.
Quote:
Originally Posted by MatLz View Post
On all mpeg2 I can have (dvd rips, dvb recordings, fresh HCEnc encodes...)

Quote:
Originally Posted by TheFluff View Post
That being said, AFAIK MPEG2 decoders are not required to have bit-exact output so it's not entirely surprising that two different decoders could have subtly different outputs.
Dunno... I thought they should output identical thing, like avc ones do.
MatLz is offline   Reply With Quote
Old 1st November 2011, 15:28   #1354  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by MatLz View Post
Dunno... I thought they should output identical thing, like avc ones do.
iDCT bro.
TheRyuu is offline   Reply With Quote
Old 1st November 2011, 15:29   #1355  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by MatLz View Post
Dunno... I thought they should output identical thing, like avc ones do.
MPEG2 does not specify exactly how the IDCT algorithm should work. Bit-exact decoding is not required. Ever noticed how you can choose between like five different IDCT algorithms in DVD2AVI? Yeah, all of those give different outputs.

See also http://guru.multimedia.cx/the-mpeg124-and-h26123-idct/
TheFluff is offline   Reply With Quote
Old 1st November 2011, 15:54   #1356  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,659
Something similar (but not related to iDCT) happened between DivX 5.02 and 5.03 too, probably a change in the rounding (Oct. 2003). That resulted in what I called "orange-blue bug". But quite soon after I reported it (March 2004), ffdshow was able to adapt its decoding.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 1st November 2011 at 15:59.
LigH is offline   Reply With Quote
Old 1st November 2011, 16:14   #1357  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by LigH View Post
The MeGUI thread discusses an issue with FFindex which crashes on uncompressed YUY2 / YV12 videos not only with dozens of GB in size.
I don't think you ever got an official reply to this, but Chikuzen is correct; FFMS2 does not support uncompressed formats (because libavcodec handles them weirdly). Crashes are possible. Indexing such files should work just fine, but when I checked the MeGUI threa I did not see any indications that it's the indexing process itself that is crashing.
TheFluff is offline   Reply With Quote
Old 1st November 2011, 16:16   #1358  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by LigH View Post
Something similar (but not related to iDCT) happened between DivX 5.02 and 5.03 too, probably a change in the rounding (Oct. 2003). That resulted in what I called "orange-blue bug". But quite soon after I reported it (March 2004), ffdshow was able to adapt its decoding.
Since you apparently need histogram("luma") to even see the differences MatLz is speaking of, it does not appear to be an issue of anywhere near that magnitude.
TheFluff is offline   Reply With Quote
Old 1st November 2011, 16:24   #1359  |  Link
MatLz
I often say "maybe"...
 
MatLz's Avatar
 
Join Date: Jul 2009
Location: France
Posts: 583
The real visible things without enhancing the luma differences are mostly in flat areas.

Thx for the explanations.

Btw, could be possible to easily implement the ffms2 idct algorithm in mpeg2source ?
Using 1 thread (to be fair) does show it is 20% faster.
MatLz is offline   Reply With Quote
Old 2nd November 2011, 08:02   #1360  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by MatLz View Post
The real visible things without enhancing the luma differences are mostly in flat areas.

Thx for the explanations.

Btw, could be possible to easily implement the ffms2 idct algorithm in mpeg2source ?
Using 1 thread (to be fair) does show it is 20% faster.
DGDecode or Mpeg2Dec3 are the recommended methods for sourcing mpeg2 material. For VOBS and transport streams I would trust them more than ffms2.

You get something like 6-7 different iDCT options with both DGDecode and Mpeg2Dec3. They both default to whatever is found in the d2v file but can be overridden in avisynth. DGIndex defaults to Skal SSE and DVD2AVI defaults to SSE2MMX. The one most like libav's default is probably Simple iDCT although that's just a guess based on libav's naming of a constant.

Mpeg2Dec3 is normally faster than DGDecode especially when seeking. I prefer DVD2AVI's default choice (with mpeg2dec3) for all things mpeg2. YMMV and opinions also.
TheRyuu is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.