FFmpegSource

[Myrsloik]

@kemuri-_9:

Any thoughts on slllooooowww indexing performance?

Derek

Did you try only indexing audio/only indexing video to see if it is caused by only one of them?

Overall I think a bit of threading really wouldn't hurt in the indexing since it's completely single threaded and having many audio tracks can slow it down quite a bit.

[Blue_MiSfit]

Hi Myrsloik,

I will do that. Almost all of my samples contain only a single audio track - either a stereo MP2 or a 2ch or 5.1ch AC3. No fancy pants stuff ;)

Threading would be much appreciated :D

Derek

[TheRyuu]

ffms2-mt-r318.7z (http://ffmpegsource.googlecode.com/files/ffms2-mt-r318.7z)

Can probably be considered just a little less experimental now (in regards to the auto-initializing pthreads). :)

Included the ffmpeg-mt libs and some other stuff (opencore) all of which is probably useless (can be used to build x264 with ffmpeg support, suppose that's good if you can't build ffmpeg for whatever reason).

[Wojszwillo]

I have this error message, when try to open avs file in VirtualDub: "FFVideoSource: Can't create decompressor: Unsuported Compression method". Problem, i think is in mkv file, that has been created with mkvmerge 4.2.0. (nevest one). Index file is'nt created nor in automated script mode, nor with ffmsindex.exe (error message - the same - "Indexing error: Can't create decompressor: Unsuported Compression method"). With files, made by older mkvmerge versions, ffms2 works fine.
What You think about that?

[kemuri-_9]

I have this error message, when try to open avs file in VirtualDub: "FFVideoSource: Can't create decompressor: Unsuported Compression method". Problem, i think is in mkv file, that has been created with mkvmerge 4.2.0. (nevest one). Index file is'nt created nor in automated script mode, nor with ffmsindex.exe (error message - the same - "Indexing error: Can't create decompressor: Unsuported Compression method"). With files, made by older mkvmerge versions, ffms2 works fine.
What You think about that?

the error is generated when the mkv file has some kind of content compression within the mkv that is not ZLIB/LIBZ.
so your new mkvmerge is generating mkv files differently than it has before.

[Wojszwillo]

It seem's that "enhancement":
* mkvmerge: enhancement: Header removal compression has been enabled by default for MPEG-4 part 10 (AVC/h.264) video tracks with a NALU size field length of four bytes"

[tebasuna51]

Remux the mkv with:
"mkvmerge" -o "output.mkv" --compression -1:none "input.mkv"

[TheFluff]

http://www.mod16.org/ffms2/ffms-r322pre-avs.7z

Now with 100% more MKV header removal compression support. Go test it for me.

[TheRyuu]

ffms2-r325.7z (http://ffmpegsource.googlecode.com/files/ffms2-r325.7z)

Contains both vanilla ffmpeg and ffmpeg-mt ffms2 builds.

[stax76]

suggestion for thread title:

ffms2 source filter plugin (FFVideoSource/FFAudioSource)

[AMED]

There seems to be a problem with the reported frame rate.

v3.25 reports 23.8091908091908 FPS
http://i36.tinypic.com/34ew3md.png

v3.18 reports 23.976 FPS (which is correct)
http://i34.tinypic.com/18zmli.png

[TheFluff]

That's not really a bug. MKV's do not have a framerate, so FFMS2 makes one up by looking at the average frame duration. There is some code that tries to set it to a familiar fractional value (i.e. 24000/1001) if it appears to be close enough, but nothing with that has changed between r318 and r325, so I dunno why you'd get different results. Maybe it's libavformat being cute.

[minda76]

i get 23,809 fps too with r325. with r3.18 is all ok. the index file created with r318 is 1.152 kb, with r325 only 65 kb.

[don_h]

i get 23,809 fps too with r325.

I can confirm this. 23,809 instead of 23,976 fps.

[cretindesalpes]

Same here.
Subtitle (String (FrameRateNumerator ()) + "/" + String (FrameRateDenominator ()))
now gives me 23833/1001 instead of 24000/1001 previously.

[TheFluff]

I assume y'all are using TheRyuu's r325 build? Is it the same with both the -mt and the standard version?

[minda76]

I am using standard r325 build and it is the same with -mt version

[Sharktooth]

there is indeed something wrong with FPS detection...

[TheFluff]

there is indeed something wrong with FPS detection...

You don't say, only five people reported it before you!

ffms2-r327.7z (http://www.mod16.org/ffms2/ffms2-r327.7z) should fix the issue.
For the curious it was indeed related to libavformat being cute. FFMS2's detection of when to use the native matroska parser and when to use libavformat is pretty dumb and based on inspecting what name libavformat claims the format has. Recently some enterprising individual changed "matroska" to "matroska,webm" which broke the detection so libavformat got used instead of the native matroska parser, and I'm not that surprised to note that libavformat still kinda sucks at matroska.

[Sharktooth]

thanks for the fix :)
any chance for MT and 64bit builds?

[Blue_MiSfit]

I get similarly odd frame rates when decoding DVCProHD wrapped in an MOV container. AssumeFPS fixes it, but still :p

I don't imagine the same issue is responsible for this particular bit of weirdness :devil:

Derek

[TheRyuu]

thanks for the fix :)
any chance for MT and 64bit builds?

Can't help you with the 64bit part but I can with the former.

ffms2-r327.7z (http://ffmpegsource.googlecode.com/files/ffms2-r327.7z)

Contains both vanilla ffmpeg and ffmpeg-mt. Anyone using the r325 build should upgrade to this one to fix the parser bug.

[Sharktooth]

Thanks again for your help. Ill try to get the sources and compile a 64 bit version.
however im scared of compiling all dependencies...

[kemuri-_9]

the avs C/x64 plugin branch now has HEAD merged to it - making it become r328.
I just built it with generic ffmpeg and can be found here (http://kemuri9.net/dev/avs/ffms2/ffms2-x64.zip)

If you desire otherwise, you can compile it yourself.

[Sharktooth]

thanks

[livetolove92]

Can anyone show me the command line to use ffmsindex.exe ? I can't find it.
If I use the ffindex in meGUI, i can't index some mkv and files larger than 10GB.

[kemuri-_9]

Can anyone show me the command line to use ffmsindex.exe ? I can't find it.
If I use the ffindex in meGUI, i can't index some mkv and files larger than 10GB.

using ffmsindex without any options prints the available command line options.

[livetolove92]

Can you show me the command line? I can't find it.

[buzzqw]

C:\>ffmsindex.exe -f Test.m2ts test.index
Writing index... done... 100%

BHH

[livetolove92]

Thanks very much :D.

[rack04]

Can someone please upload a zip file of the latest svn checkout? For some reason I keep getting the following message:

svn checkout http://ffmpegsource.googlecode.com/svn/trunk/ ffmpegsource-src
svn: Server sent unexpected return value (405 Invalid Method) in reponse to PROPFIND request for '/svn/trunk'

[kemuri-_9]

checkout works fine here.

[rack04]

checkout works fine here.

Must be something with the connections settings at work. I can access it on my home pc.

[Underground78]

@rack04: try https, it has already worked for me when I had this error.

[stax76]

It's seems there is a memory leak with all transport streams.

[kemuri-_9]

It's seems there is a memory leak with all transport streams.

which demuxer is being used? (libavformat or haali)
does a leak similarly occur with ffmpeg?

[stax76]

I've looked in Process Explorer which shows Haali DLLs so yes I think Haali is used.

I'm not sure what's the easiest method to check it with ffmpeg.

[stax76]

Some more info.

MediaInfo source file:

General
ID : 1
Complete name : D:\Video\Samples\M2TS\1080p - AVC - DTS-HD - Die Another Day.m2ts
Format : BDAV
Format/Info : Blu-ray Video
File size : 404 MiB
Duration : 2mn 11s
Overall bit rate : 25.8 Mbps
Maximum Overall bit rate : 35.5 Mbps

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Duration : 2mn 11s
Bit rate mode : Variable
Maximum bit rate : 37.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Audio
ID : 4352 (0x1100)
Menu ID : 1 (0x1)
Format : DTS
Format/Info : Digital Theater Systems
Format profile : MA
Muxing mode : Stream extension
Duration : 2mn 11s
Bit rate mode : Variable
Channel(s) : 6 channels
Channel positions : Front: L C R, Side: L R, LFE
Sampling rate : 48.0 KHz
Bit depth : 24 bits

Text
ID : 4608 (0x1200)
Menu ID : 1 (0x1)
Format : PGS

avs script:

SetMemoryMax(128)
LoadPlugin("C:\...\ffms2.dll")
FFVideoSource("D:\Video\Samples\M2TS\1080p - AVC - DTS-HD - Die Another Day.m2ts")

When I open the script with VD and press Space to run the preview,
at the end of the 2 minutes clip, task manager shows 600 MB,
when I close the script, the memory is released. It seems to be
reproducible with all DVB or Blu-ray HD AVC TS. With ffplay only
constant 40 MB are used so no problem there.

[TheFluff]

I can't say I'm surprised. The entire Haali parser stuff is full of COM/ATL magic that I have absolutely no idea about how it works. Myrsloik knows more than I do but he's disappeared off the face of the earth.

[LoRd_MuldeR]

I never managed to get FFmpegSource2 decode my H.264 TS files (captured from DVB-S2) properly.

It always crashed randomly during the encoding process, sooner or later, although a quick preview of my AVS script looked okay.

And yes, I have Haali's Splitter installed ;)

The solution for me: Demuxing the original TS file with tsMuxR and then muxing the raw H.264/MP3 streams into MKV with MKVToolnix.

FFmpegSource2 handles the resulting MKV file perfectly fine and I can live with that procedure...

[kemuri-_9]

does the leak occur only with h.264 in TS or other video codecs as well?
(asking because i saw some special handling of h.264 in the haali demuxer code)

[stax76]

@LoRd_MuldeR

Looks like the same problem.

@kemuri-_9

IIRC it didn't happen with VC-1 and MPEG-2 but I'm not 100% sure, I can make some more tests if it helps, I've got a pretty complete sample collection. :)

[TheFluff]

If the same problem occurs with MKV's then it's something else than the COM magic and then it should definitely be fixed. Send me the sample file.

[LoRd_MuldeR]

If the same problem occurs with MKV's then it's something else than the COM magic and then it should definitely be fixed. Send me the sample file.

No, doesn't happen with MKV. That's the point! At east for me with MKV everything works perfectly fine :)

(Being able to access the TS file directly would still be an important advantage though)

Uploading a sample will be difficult, as it happens randomly after 1-2 hours of encoding. And the TS files usually are ~10 GB in size. A short snippet won't reproduce the issue.

In case you have access to a DVB-S2 card, you can produce a sample file yourself easily though...


Myrsloik knows more than I do but he's disappeared off the face of the earth.

Obviously he isn't hiding too deep in the ground ;)

Last Activity: Yesterday 15:18

[kemuri-_9]

@kemuri-_9

IIRC it didn't happen with VC-1 and MPEG-2 but I'm not 100% sure, I can make some more tests if it helps, I've got a pretty complete sample collection. :)

it would help us nail down exactly what section of code is exactly being problematic,
e.g. the entire haali demuxer or just the h.264 special handling within it

[TheFluff]

In case you have access to a DVB-S2 card, you can produce a sample file yourself easily though...

As a matter of fact, I do. I guess I'll cap some stuff.
I'm not that interested in maintaining the Haali dshow parser stuff though, since it's not crossplatform. I'd rather write a better TS parser from scratch myself, but that takes time, which I don't have too much of right now.

[LoRd_MuldeR]

I just want to confirm that the crash I was experiencing really is related to the memory leak:
* http://mulder.brhack.net/temp/ffms2_memleak_1.png
* http://mulder.brhack.net/temp/ffms2_memleak_2.png

The script I use is really simplistic:
FFmpegSource2("ArteHD.ts", atrack=1)

[stax76]

@TheFluff

Some more tests showed it's AVC specific. Is raw AVC supported? That would possibly be more useful then AVC in TS and easier to add, I'm guessing here. :)

@LoRd_MuldeR

Even if there wouldn't be the memory issue there would possibly be sync issues, somebody posted in the German board that tsmuxer helps with sync issues. I tested it with a sample capture that has a error and tsmuxer fixed that when remuxing ts, I don't know if the error correction is applied when demuxing.

[LoRd_MuldeR]

Having to pipe the TS file trough tsMuxR once (TS to TS), before processing it with FFmpegSource, still would be one step less than my current work-flow (TS to "raw" to MKV).

[stax76]

What is your current work-flow? In case it's tsmuxer > mkvmerge > ffms2 you can try DGDecNV instead if you capture much and have a Nvidia card.