FFmpegSource

[jase99]

Thanks for taking the time you already have to look into it. I'll keep an eye out tho as this will be such a huge benefit if it's ever resolved. Cheers.

[SpAwN_gUy]

OK. i'm moving here..

i have a demuxed from MKV 6ch .aac sound file..
as far as bassAudio is not working.. with 6ch.. i was advised to use this plugin.

with FFmpegSource with .aac as input creates cashe more than 700mb, and then i ran out of diskSpace.
with reMUXed .mp4 as input does the same.

FFaudioSource plays(tried with MPC) both streams (demuxed .aac and remuxed .mp4) with some glitches.. like skipping something.

how can i fix this? or.. any Help???

mpc plays .mp4 file just fine.. with haali's splitter and ffdshow audio decoder

[Myrsloik]

Don't expect perfect results without using the cache. This plugin is not for you.

[dimzon]

how can i fix this? or.. any Help???

mpc plays .mp4 file just fine.. with haali's splitter and ffdshow audio decoder
Just use DirecShowSource ;)

[DeathTheSheep]

dssource is busted with alpha 2 (latest). Is there a fixed version somewhere?

[TheFluff]

dssource is busted with alpha 2 (latest). Is there a fixed version somewhere?
I don't know but if there is you probably won't find it in the ffmpegsource thread.

[SpAwN_gUy]

Just use DirecShowSource ;)went to graphedit(i just forgot about that method) and transcoded to AC3@640kbps.. using FFdshow Audio decoder.. and ac3 output...

but the thing is.. that sound was Stereo :( ... well there were 6 channels, but... only L and R were busy...

[TheFluff]

You didn't happen to have the ffdshow downmixing filter active, did you?

[SpAwN_gUy]

You didn't happen to have the ffdshow downmixing filter active, did you?

nope... all checks - UNcheked..
Input: 48KHz, 6 channels, 117kbps SBR AAC (libfaad2)
Output (preview): 48KHz, 6 channels 16-bit integer..
current volume levels - only L and R are Jumping.. all other - 0

if connected through Nero Filtes.. ffdshow processor (connected after the decoder) shows moving also only in L and R
i guess.. it's the file...

'kay, quiting offtop...

[Shinta]

Hi,

I don't know if this or something equal was asked before (maybe I missed it or I'm just too lazy to search properly), anyway I'll just give it a try...

I have a mkv-video(h264) that seems to have a variable framerate (I don't know if this is possible in the container or the codec, but using the "Video Renderer" stats in MediaPlayerClassic I got something like 59.94 fps on the first 1:20 min and after that it drops 23.97 fps).

The problem should be obvious. The overall framerate gets calculated using playtime and framecount (28.53 fps). Audio and video are completely out-of-synch.

I've looked into the FFmpegSource sources (GetFrame to be precise), and saw that the seeking is done using the pure framenumbers.

Is it possible to change that to a timebased-nearest-frame something based on a virtual framerate?
(If you don't know what I mean:
When opening a file the filter searches for the two (time-)closest frame or divides framenumber and playtime to calculate the (virtual) framerate or gets a hint be the user (which I prefer).
Then it uses this framerate and the playtime to calculate a new (virtual) number of frames.
If the filter gets asked for a frame (GetFrame-Methode), it calcs the timestamp for that virtual frame using the virtual framerate, seeks to the time-closest-frame and returns it. Voila.)

Would be great if this could be implemented, I would try it myself, but still having trouble compiling ffmpeg... (though, a precompiled ffmpeg and a VS2005 project for ffmpegsource could help ;-)

[Myrsloik]

WRITE TIMECODES TO A FILE. Fix it yourself using that. This was also very harshly answered earlier in the thread.

For the <edit mod: removed profanity> of you trying to compile ffmpeg source this is what you need and what will happen:
A recent gcc compile like 4.2.3. Have fun compiling it yourself with mingw.
A recent ffmpeg. Straightforward IF it compiles.
Uneventfully compile zlib but forget --prefix=/mingw and confuse yourself.
Make an empty dll project in vs2005 and add all files. Also add a list of obscure gcc include libraries to appease ffmpeg.
Compile.
You are now likely to be the proud owner of a miscompiled ffmpegsource. Order a pizza and watch tv instead.

[TheFluff]

For the fools of you trying to compile ffmpeg source you need and what will happen:
A recent gcc compile like 4.2.3. Have fun compiling it yourself with mingw.
A recent ffmpeg. Straightforward IF it compiles.
Uneventfully compile zlib but forget --prefix=/mingw and confuse yourself.
Make an empty dll project in vs2005 and add all files. Also add a list of obscure gcc include libraries to appease ffmpeg.
Compile.
You are now likely to be the proud owner of a miscompiled ffmpegsource. Order a pizza and watch tv instead.
After successfully compiling Aegisub with ffmpeg in VS2005 (with your help) I wrote some slightly more detailed instructions (http://malakith.net/aegiwiki/Build_Instructions#ffmpeg) on how to statically link ffmpeg with something compiled in MSVC. The configure flags are not completely identical (remember to add libpostproc and its corresponding .a) and you'll need zlib as well but maybe it helps somewhat.

[Shinta]

@Myrsloik: Thanks for the loud and extremly confusing answer.
@TheFluff: Thx for the slightly more helpful one.

After fixing a small problem running sh-scripts with exe-extension in msys (in ./configure) and adding a missing #define to rtsp.c, I was finally able (about 3 hours) to compile ffmpeg. The compiling of ffmpegsource (I tried this using precompiled ffmpeg libs, but this results in a very small ffmpegsource.dll (292 kBytes), which was obviously crap) was just (re-)linking.

So, if someone is interessted in a VS2005 project (including ffmpegsource and zlib sources and the ffmpeg libs compiled by me), you may send me a PM.

.oO(Now, lets see, what can I do to make the filter returning frames with cfr on vfr videos)°

[Shinta]

Is the source at https://spaceboyz.net/svn/aegisub/trunk/FFmpegSource the current version (1.16)?
The file ffmpegsource.html holds only changelog entries until 1.14...

Also, in matroskacodecs.cpp lines 137++:

case MAKEFOURCC('Z', 'L', 'I', 'B'):
return CODEC_ID_FLV1;
case MAKEFOURCC('F', 'L', 'V', '1'):
return CODEC_ID_ZLIB;


Shouldn't this be...?:

case MAKEFOURCC('Z', 'L', 'I', 'B'):
return CODEC_ID_ZLIB;
case MAKEFOURCC('F', 'L', 'V', '1'):
return CODEC_ID_FLV1;

[Myrsloik]

That's a really obscure bug. Why were you reading the whole codec list anyway?

Version 1.16 + that fix is now in the svn.

[Shinta]

That's a really obscure bug. Why were you reading the whole codec list anyway?
I was just speed-reading the code and it caught my eye.

But another thing. I really don't understand the whole video decoding stuff completely yet, so don't be mad if I understood it wrong.

In ffmpegsource.cpp & ffmatroskasource.cpp:

// Flush the last frame
if (CurrentFrame == VI.num_frames - 1 && VideoCodecContext->has_b_frames)
Ret = avcodec_decode_video(VideoCodecContext, AFrame, &FrameFinished, NULL, 0);

Is it really only the last frame that isn't decoded at this moment? Aren't it as much as frames as the number of (deferred?) b-frames since the last seek?


// Flush the last frames
if (CurrentFrame <= VI.num_frames - 1 && VideoCodecContext->has_b_frames)
Ret = avcodec_decode_video(VideoCodecContext, AFrame, &FrameFinished, NULL, 0);


Just one thing more: In FFBase::SaveTimecodesToFile you sort the timecodes before writing them to the file. Shouldn't the frames (or their DTSs) be sorted for seeking too?

[TheFluff]

In ffmpegsource.cpp & ffmatroskasource.cpp:

// Flush the last frame
if (CurrentFrame == VI.num_frames - 1 && VideoCodecContext->has_b_frames)
Ret = avcodec_decode_video(VideoCodecContext, AFrame, &FrameFinished, NULL, 0);

Is it really only the last frame that isn't decoded at this moment? Aren't it as much as frames as the number of (deferred?) b-frames since the last seek?


// Flush the last frames
if (CurrentFrame <= VI.num_frames - 1 && VideoCodecContext->has_b_frames)
Ret = avcodec_decode_video(VideoCodecContext, AFrame, &FrameFinished, NULL, 0);

This makes no sense. Even if there were more than one frame, which I highly doubt (read the loop above the mentioned code again), your fix wouldn't output more than one frame because avcodec_decode_video() only outputs one frame at a time. By the way the frame that is "decoded" is just NULL, so... (see avcodec.h)

[Myrsloik]

Due to b-frame reordering decodign will never lag more than one frame. Example:

Shown: IBPBPI
Stored: IPBPBI

Shown: IBBPBBPI
Stored: IPBBPBBI

Meaning that you will never have more than a one frame lag (indicated by has_b_frames). Thus there are no more frames to flush. As thefluff pointed out that part is also only triggered if all the other logic failed which is for the last frame with decoding lag OR for very unusual decoding errors and then only for the last frame.



FFBase::SaveTimecodesToFile - The whole DTS relation is a bit special. That function only sorts them because mkvmerge expects it as input (historical note: old versions used unsorted).

The reason ffmpegsource doesn't care about it being sorted or not is that the position of keyframes is independent of the reordering. As the only value extracted from it is "what frame number does this DTS correspond to?" and we know that seeking always goes to keyframes (minus bugs and that stuff). So the only thing the other DTS values in between are really used for are placeholders.

[Shinta]

For videos with only one b-frame your right, but I found a video that seems to have 2 frames left to decode after reaching EOF.

Using this code...

// Flush the last frames
if (CurrentFrame <= VI.num_frames - 1 && VideoCodecContext->has_b_frames)
Ret = avcodec_decode_video(VideoCodecContext, AFrame, &FrameFinished, NULL, 0);

FILE* err=fopen("err.txt", "a");
if(err)
{
fprintf(err, "%d %d %d %d %d\r\n", CurrentFrame, VI.num_frames-1, Ret, FrameFinished, VideoCodecContext->has_b_frames);
fclose(err);
}


I got this err.txt:

90286 90287 0 200 2
90287 90287 0 200 2


I don't know what the 200s in the 4th column means exactly, but they're not zero and thus the frame were decoded.

[Myrsloik]

To debug these things it's usually much easier to encode a clip with similar settings and clearly number every frame. Does your clip by any chance have open GOPs?

[Shinta]

Maybe it has something to do with this:

(From the german wikipedia on h264)

Long-Term Prediction: P- und B-Frames können nicht nur Referenzen auf den jeweils letzten I- oder P-Frame enthalten, sondern auch zu theoretisch beliebig vielen vorhergehenden, um eine effizientere Kodierung von periodischen Bewegungen zu ermöglichen. In den Profiles und Levels ist die Anzahl von Referenzbildern allerdings auf maximal fünf begrenzt.


My translation (hope it is good enough):

Long-Term Prediction: P- and B-frames doesn't have to reference only the last I- or P-frame, but also to theoreticaly any number of previous (I- or P-)frames, to increase encoding performance on recurring movements. The profiles and levels limits the number of referenced frame to five.



To debug these things it's usually much easier to encode a clip with similar settings and clearly number every frame. Does your clip by any chance have open GOPs?


I don't know, and don't know how to find that out. But I've splitted the video at the last keyframe and can send you this 74 kByte mkv.

[Shinta]

Lets try an attachment: (it is that 74kb mkv)

[Myrsloik]

Seeking in h264 should be limited to IDR frames which means the assumption is still true (I is just another letter in between). Find some place to upload the file to and I will look at it.

[Shinta]

Or, http://www.megaupload.com/?d=Y9QNAHQB

[Manao]

The one-frame lag implied by bframes is valid for all codecs prior to H264. For h264, the possibility to create b-reference (and thus pyramidal b-gops) may create longer lags (one level of byramid is commonly used with x264 and creates a two frame lag)

[Myrsloik]

Bug fixed and put in the svn. As Manao hinted it only applied to h264 with pyramid stuff and would usually result in the last frame being dropped and the now second to last frame duplicated.

No new compile today though, ffmpeg svn is having one of those broken weeks it seems.

[thetoof]

Feature request: handling of the anamorphic flags

[Myrsloik]

So I've read the AR from the stream, now what? Return it in a variable with a user specified name? Rescale the video and kill kittens?

[jase99]

I regularly use the filter with anamorphic sources, what additional support is needed?

[Myrsloik]

How aspect ratio reporting is done now:
FFSAR_NUM, FFSAR_DEN and FFSAR are set to the container values for mkv and the video decoder values for all other filetypes.
The problem:
Several mp4 files report a SAR of 0/1 which isn't right or very helpful. Should the cases with 0 be converted to 1/1 or left as it is?

[IanB]

You are probably better reporting exactly what is in the container. People can then devise what ever is best for them. If you implemet any scheme then you hide information, i.e. is it really 1/1 or did Myrsloik convert 0/1 to 1/1.

[Myrsloik]

New version posted (1.17) which fixes decoding of the last frames in h264 and adds reporting of the SAR for the currently decoded clip.

[buzzqw]

seems that last version break vc1 support
with previus versione (1.16) i was able to open m2ts file using this string
FFmpegSource("part0022_vc1.m2ts",vtrack=-1,atrack=-2,seekmode=-1,timecodes="Test.timecodes",vcache=true,vcachefile="Test.ffcache")

but now i got a failure on opening script with virtualdub

here is a sample http://www.mediafire.com/?crgnxmvzpzy of m2ts (60mb)

thanks!

BHH

[Myrsloik]

Fixed in 1.18. Obviously lavc returns lots of errors when decoding that kind of clip. All errors are once again ignored.

[buzzqw]

yes, working!

:thanks:

BHH

[Nagisa_chan]

I have one question

Is FFmpegSource better than DirectShowSource?

I am novice

the people "not here" says :directshowsource is shit

[IanB]

What ever works is better :)

DirectShowSource use Micro$ofts DirectShow media player API to attempt to strip Video and Audio from media for processing. The DirectShow API's goals are to support DRM, make writing Media Player easy and playback media smoothly, none of which Avisynth particularly needs. The API does not mandate sample exact positioning or maximum speed and quality decoding, both of which are highly desirable for Avisynth usage.

FFMpeg provides source decoders for many formats to the DirectShow interface. FFMpegSource accesses those source decoders before DirectShow gets it's grubby little paws in the mix, so hopefully can provide better control over the media data being extracted. But remember as the target audience is DirectShow some source decoders do not implement the features like sample exact positioning that Avisynth requires, so decoding some formats can still be problematic.

When decoding media in Avisynth you should look for a native source decoder first, like AviSource, MPeg2Source, QTSource, Nic*Source, etc. If none are available then look for the better generic media decoders like FFMpegSource. If still none are available then you may have to use DirectShowSource, for many formats it is the only choice available and for some formats it even does a very excelent job.

As I said "What ever works is better".DirectShowSource is shitIn many case yes, but please remember and explain to people that it is always doing the best that it can with an API that is sometime antagonistic about striping media.

[vlada]

Hi,

I'm working on an automated transcoding application. One of the tasks I need to do when opening a video is cropping detection. If I open a DVD disc (VOB file), the indexing takes a very long time. To detect letterbox, I don't need frame accurate seeking (I guess this is what the indexing is good for). Is there any way to disable the indexing?

I could use DirectShowSource, but I would like to use FFmpegSource, because it is independant on what the user has installed.

[buzzqw]

@vlada

dont' want to bother.. but mplayer.exe has a good detect crop rountines... you can use it for determining crop values
mplayer.exe -vf cropdetect -nosound -vo null -frames 15 -sstep XXX inputfile.vob

where xxx is the totaly duration of movie in sec / 15

BHH

[Myrsloik]

Alternatively compile ffmpeg with only the mpeg2/vob parts and link it into your program. Using ffmpeg directly isn't that hard and will be a lot easier than convincing me to add that "feature".

[vlada]

Thanks for the suggestions.

I will stay with Avisynth and ffmpeg source for now. I need to index the files for conversion anyway. But after completing the first half of my project (build a universal anything -> DVD/AVI/MP4/MKV convertor) I plan to transform my application into a NLE.

One of the basic features I plan to implement is "Direct Stream Copy". So I will need mencoder or ffmpeg for this anyway. But I'm still very far away from this. Also my programming knowledge is still very low.

[Nagisa_chan]

Thanks for the reply:)

[CruNcher]

@Myrsloik

Somehow the Framerate of .WMV files are wrong detected i allways have to use Assumefps to correct that :(
going over directshowsource works via ffdshow and microsofts decoder both detect the framerate correctly.
Tough that is only a small problem :) keep on the source filter is great.

Would it be possible to parse the H.264 bitstream (for all container) and look if Pulldown has been used (HD-DVD) and according to that set the correct fps :) (would make transcoding life another step easier currently ffmpegsource detects it allways as 29.97 fps

[Myrsloik]

Framerate for all containers except mkv is determined by the time delta between the first and the second frame and nothing more. For cfr files it works great, for vfr it works not so great.

Summary: it's vfr so you should be writing out the timecodes anyway because avisynth doesn't support vfr.

Pulldown support is extremely unlikely to be added because of how it would have to be implemented. There is no way to retrieve that information from the libavformat parsers and instead the video has to be fully decoded which would make the indexing ridiculously slow. Finding out whether pulldown flags exist at all would also most likely require the whole video to be decoded.

[vlada]

Hello,

I have another couple of questions. The first problem is nubering of audio tracks. I want to transcode audio track(s) from a DVD-Video using FFmpegSource and SoundOut. Using MediaInfo I dtected that the first audio track has ID 80. But if I used atrack=80, FFmpegSource returned an error that this not an audio track. Using atrack=1 works fine. So how are the audio tracks IDs treated? Right now it seems to me that the first track (0) is video and nex tracks (starting from 1) are audio. Is this correct?

Second question - what is the audio caching good for? It takes quite a long time to complete. Why isn't the audio decoded "on-the-fly"?

As I already wrote I'm working on a simple to use program based on Avisynth. It should be able to convert between most popular formats. FFmpegSource is really great for my needs. It doesn't rely on installed codecs or filters and I can specify which audio track to process.

The problem is that during indexing or caching of audio the program seems to be freezed, because I have no idea how to check and display the progress of indexing. Is there any possibility to do it?

[Myrsloik]

This is all you will ever know about track numbering because there is no real system to it: (coincidentally it's also in the manual)
atrack & vtrack: Track number as seen by the relevant demuxer, starts from 0, -1 means it will pick the first suitable track and -2 means it's disabled.

Find out for yourself why audio caching is a good idea:
FFAudioSource(string source, int atrack = -1, string acachefile, string ademuxedfile)

Experimental, may or may not be accurate enough for real usage.

And no, there is no sane way for avisynth filters to show creation progress.

[CruNcher]

@Myrsloik

http://forum.doom9.org/showthread.php?p=1127362#post1127362

[Myrsloik]

A link for me? you really shouldn't have...

It says mplayer parsers work but libavformat fails? do I do something now? should I panic? what's the mbaff stuff about? is that what makes it fail and not the parsers? was the point that mplayer patches ffmpeg until it can no longer be recognized as usual?

Don't panic

[CruNcher]

I'm not sure why ffmpegsource fails with those files it just doesn't do anything anymore it seems i see no cache created nothing seems to happen, the application just freezes and almost the whole Windows does when i execute the .avs (it looks like a buffer overflow), but without the mbaff patch it should react like videolan i guess and show these decoding errors and not crash immediately or hang in a loop.

It seems only when MPC-HC is trying to open that .avs the whole system almost freezes when x264.exe does i see io activity seems like indexing is in progress for the 2 gb file now

jep it works now from x264.exe as it should with the .ts one

avis [info]: 1280x720 @ 23.97 fps (15016 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
[h264 @ 01968E20]warning: first frame is no keyframe
encoded frames: 2899/15016 (19.3%), 15.68 fps, eta 0:12:52

ok i updated it everything works except MPG (the same case as for Mplayer which isn't suprising since both use libavformat and are up2date) :)

@ Myrsloik
another problem occurred I created from a TS file 23.976 FPS a .MKV and .MP4 with Haalis Dsmux and both seem either being wrong detected or Haalis muxer does something wrong :(

Input TS information


General #0
ID : 0
Complete name : C:\x264hardware\test.ts
Format : MPEG-4 Transport
Format/Family : MPEG-4
File size : 1.98 GiB
PlayTime : 10mn 26s
Bit rate : 27.1 Mbps

Video #0
ID : 4113 (0x1011)
MenuID/String : 1 (0x1)
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec profile : High@L4.1
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 2
PlayTime : 10mn 26s
Bit rate : 26.1 Mbps
Width : 1920 pixels
Height : 1080 pixels
Display Aspect ratio : 16/9
Frame rate : 23.976 fps
Chroma : 4:2:0
Interlacement : Progressive


the initial TS is exactly 23.976 Film (all applications detect it as such)


MKV:


General #0
Complete name : C:\x264hardware\test.mkv
Format : Matroska
File size : 1.93 GiB
PlayTime : 10mn 27s
Bit rate : 26.5 Mbps
Encoded date : UTC 2008-04-21 13:04:09
Writing application : dsmux
Writing library : Haali DirectShow Matroska Muxer 1.8.122.18

Video #0
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec profile : High@L4.1
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 2
PlayTime : 10mn 26s
Bit rate : 25.4 Mbps
Width : 1920 pixels
Height : 1080 pixels
Display Aspect ratio : 16/9
Frame rate : 23.976 fps
Chroma : 4:2:0
Interlacement : Progressive



MP4:

General #0
Complete name : C:\x264hardware\test.mp4
Format : QuickTime
Format/Info : Original Apple specifications
Format/Family : MPEG-4
File size : 1.93 GiB
PlayTime : 10mn 27s
Bit rate : 26.5 Mbps
Encoded date : UTC 2008-04-21 13:33:00
Tagged date : UTC 2008-04-21 13:33:00

Video #0
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec profile : High@L4.1
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 2
PlayTime : 10mn 27s
Bit rate : 43.9 Mbps
Maximum bit rate : 26.5 Mbps
Width : 1920 pixels
Height : 1080 pixels
Display Aspect ratio : 16/9
Frame rate mode : VFR
Frame rate : 23.939 fps
Minimum frame rate : 0.999 fps
Maximum frame rate : 23.976 fps
Chroma : 4:2:0
Interlacement : Progressive
Bits/(Pixel*Frame) : 0.885
Encoded date : UTC 2008-04-21 13:33:00
Tagged date : UTC 2008-04-21 13:33:00




FFMPEGsource parsing results

TS = avis [info]: 1920x1080 @ 23.97 fps (15016 frames) - the first frames are non key frames tough ffmpegsource doesn't skip them
MKV = avis [info]: 1920x1080 @ 23.94 fps (15014 frames) - starts with the keyframe (correct)
MP4 = avis [info]: 1920x1080 @ 23.98 fps (15014 frames) - starts with the keyframe (correct)

how can this be ???

[vlada]

I have an MKV file with 2 audio tracks: MP3 and Vorbis. It is a short sample I made for testing my application.

FFmpegSource has problems reading audio from this file. I use command FFAudioSource("video.mkv", ademuxedfile = "demuxed.ac3", atrack=1, acachefile="temp"). For the first audio track (MP3) no demuxed file is created, but the audio is handled to the output. For the second track (Vorbis) the audio is not even passed to the output. It is probably normal for Vorbis tracks that they can't be demuxed.

I tried to demux the file with ffmpeg. It demuxes the MP3 correctly but fails with the Vorbis track. It is a general problem with MKV/Vorbis files (I tested it on other videos I have). Reencoding the Vorbis audio track to other format works correctly.

I'm wondering why FFmpegSource won't read the Vorbis track at all and why it won't demux the MP3 file. Do you have any idea?