Cross-Platform D2V Source for VapourSynth

[Daemon404]

So, I was bored and wrote a cross-platform D2V parser and decoder for VapourSynth.

Current Version: 1.2

Windows Binary: https://github.com/dwbuiten/d2vsource/releases/download/v1.2/d2vsource-1.2-windows.zip - Contains 32-bit and 64-bit binaries.
Source: https://github.com/dwbuiten/d2vsource/releases/download/v1.2/d2vsource-1.2.tar.xz

Known Limitations & Bugs

Does not support user specified cropping. Would be easy to add, but I disagree with the premise. Use VapourSynth's cropping.
Needs the full path to the D2V file.
Probably more I'm forgetting!


Example usage:
import vapoursynth as vs
core = vs.Core()

core.std.LoadPlugin(path=r'C:\Path\To\d2vsource.dll')

ret = core.d2v.Source(input=r'C:\Path\To\my.d2v')

last = ret

Parameters:
input - Full path to input D2V file.
nocrop - Always use direct-rendered buffer, which may need cropping. Provides a speedup when you know you need to crop your image anyway, by avoiding extra memcpy calls.
rff - Invoke ApplyRFF (True by default)
threads - Number of threads FFmpeg should use. Default is 0 (auto).

If you want to contribute, feel free to send a pull request on GitHub.

[Chirico]

I can look into what would be needed to port DGIndex to *nix. The last time I looked at the code it was a bit menacing, though. It also relies on a bunch of ASM that would probably be a nightmare to port so we'd have to stick to plain C++ until/unless someone wants to take on that task.

I'll also test building this on OS X when I get home tonight.

[jackoneill]

I can look into what would be needed to port DGIndex to *nix. The last time I looked at the code it was a bit menacing, though. It also relies on a bunch of ASM that would probably be a nightmare to port so we'd have to stick to plain C++ until/unless someone wants to take on that task.

I'll also test building this on OS X when I get home tonight.

There is a "trimmed" dgindex here: http://fraps.servegame.com/movie/. IIRC it did compile in Linux when I discovered it (ages ago).

[Chirico]

Okay. Though thinking about it more: Does DGIndex actually give us any benefits over ffms2 with a generated index file these days? I can really only see for possibly cutting streams and demuxing audio, but I'm not sure those features alone are worth myself taking on the effort. Though I can definitely provide you with feedback on building on OS X.

[JEEB]

Okay. Though thinking about it more: Does DGIndex actually give us any benefits over ffms2 with a generated index file these days?
The LAVF mpeg-ts/ps demuxer doesn't really seem to be too stable in order to give frame-exact seeking. It seems to work for some things, and fail at others. Basically the idea of indexing the packet borders for the needed IDR packets from which lavf parsing/decoding can be started seems to work rather well.

Of course, if possible it would be nice for this kind of stuff to be implemented in ffms2 itself, but this is a basic first step towards having cross-platform solutions for various files.

[Daemon404]

Okay. Though thinking about it more: Does DGIndex actually give us any benefits over ffms2 with a generated index file these days? I can really only see for possibly cutting streams and demuxing audio, but I'm not sure those features alone are worth myself taking on the effort. Though I can definitely provide you with feedback on building on OS X.

I haven't had much luck at all using FFMS2 to seek frame-accurately in MPEG-PS or MPEG-TS.

[Daemon404]

Of course, if possible it would be nice for this kind of stuff to be implemented in ffms2 itself, but this is a basic first step towards having cross-platform solutions for various files.

This would mean that FFMS2 has two types of indexing, which is... kinda ~meta~, and somewhat ugly.

[Chirico]

I haven't had much luck at all using FFMS2 to seek frame-accurately in MPEG-PS or MPEG-TS.

Okay, so that is definitely a good reason. Then I'll definitely look into it. Especially from the starting point of that already ported and trimmed-down DGIndex.

[Chirico]

So I got it to compile in OS X with the removal of "-Wl,-Bsymbolic" but I just get:


File "vapoursynth.pyx", line 1016, in vapoursynth.Function.__call__ (build/src/cython/vapoursynth.c:15627)
vapoursynth.Error: 'Cannot initialize decoder.'

Trying to trace the issue now.

[Daemon404]

Trying to trace the issue now.

Yes, I should really add proper error reporting instead of dumping to stdout...

It's probably failing to open the file(s) referenced in the D2V file. I assumed, since this is on OS X, you are using a relative path?

EDIT: As of b7a0ada5, error messages are properly propagated.

[Chirico]

Actually I just figured out that it was something stupid on my part. I forgot to change the path in the d2v file from when I generated it on Windows. Works fine now. For anyone who wants it, here (http://www.sendspace.com/file/olb5yp) is the compiled version for OS X.

[Chirico]

There is a "trimmed" dgindex here: http://fraps.servegame.com/movie/. IIRC it did compile in Linux when I discovered it (ages ago).

The only issue with compiling that is on OS X is the lack of the non-standard fcloseall that it uses which I'm trying to fix now.

[Daemon404]

The only issue with compiling that is on OS X is the lack of the non-standard fcloseall that it uses which I'm trying to fix now.

I compiled it on Windows, and it's not that useful. It fails on MPEG-PS and elementary streams, and only works on some MPEG-TS streams...

[Chirico]

Okay. Then I'll look into the full DGIndex instead, then.

[Revgen]

I have a large MPEG2-TS file. And it won't seek past 40,000 frames in VirtualDub with VPY support.

[Daemon404]

I have a large MPEG2-TS file. And it won't seek past 40,000 frames in VirtualDub with VPY support.

It's definitely possible that I messed up large file support. I don't have any very large MPEG-TS on meat this moment, but I'll either generate one or try and test with a large MPEG-PS stream.

Do you know if it gets stuck while decoding linearly too, or if it's solely a seeking problem?

In any case, here is a test build with a possible "fix": <Linked Removed>

The binary is a fair bit bigger due to being built with MinGW this time. If this binary works, I'll work on integrating an MSVC-compatible fix as well.

[Revgen]

It doesn't get stuck decoding linearly. But if I seek past 40,000 frames or stop the playback of the video once it passes that mark, Virtual Dub freezes and I have to shut it down with the task manager.

I tried your "fixed" version. The problem is still there.

[Daemon404]

It doesn't get stuck decoding linearly. But if I seek past 40,000 frames or stop the playback of the video once it passes that mark, Virtual Dub freezes and I have to shut it down with the task manager.

Hmm, so the problem is not what I thought it was. I'm going to have to try and generate a large MPEG-TS file then and try to reproduce.

[Daemon404]

Hmm, so the problem is not what I thought it was. I'm going to have to try and generate a large MPEG-TS file then and try to reproduce.

OK, as of f16e83a2, it is fixed. I've made a few other fixes, and put a new binary in the original post.

[Revgen]

The fix works. Seeking works now. Thank you.

[lil'jerry]

IJust wanted to say thanks for providing this. After seeing the announcement for AvxSynth I was thinking of porting DGIndex to something cross-platform but got discouraged by the fact that it's a huge mass of spaghetti code. But after seeing this post, I've started up again a few days ago this time starting from scratch and just re-implementing core features (generating d2v files, demuxing video/audio, d2v fixer, etc.) and leaving out all the other cruft (AC3 decoding, audio resampling, cropping filter, luminance filter, etc). Right now, I have d2v generation and demuxing working and I'm getting identical results to DGIndex in my limited tests. I'm trying to finish up the last few features I want supported and trying to test it as much as possible. Should have something by mid next week for others to test and I'll throw the code up on github. It's written for Qt and uses ffmpeg libs for parsing/demuxing of the streams.

[Guest]

IJust wanted to say thanks for providing this. After seeing the announcement for AvxSynth I was thinking of porting DGIndex to something cross-platform but got discouraged by the fact that it's a huge mass of spaghetti code. But after seeing this post, I've started up again a few days ago this time starting from scratch and just re-implementing core features (generating d2v files, demuxing video/audio, d2v fixer, etc.) and leaving out all the other cruft (AC3 decoding, audio resampling, cropping filter, luminance filter, etc). Right now, I have d2v generation and demuxing working and I'm getting identical results to DGIndex in my limited tests. I'm trying to finish up the last few features I want supported and trying to test it as much as possible. Should have something by mid next week for others to test and I'll throw the code up on github. It's written for Qt and uses ffmpeg libs for parsing/demuxing of the streams. Please rename the tool as I prefer to keep the DG imprimatur for my stuff. Thank you.

[lil'jerry]

Yes, it will be called something different such as D2VCreator since it shares no code but only shares a similar-looking GUI and a subset of functionality of the DGIndex/DVD2AVI programs.

[Guest]

Great, thanks very much and good luck with the project!

[lil'jerry]

Hey Daemon in looking at your code it seems you ignore the "field operation" flag in the D2V file. Are you planning to support this?

[Daemon404]

Hey Daemon in looking at your code it seems you ignore the "field operation" flag in the D2V file. Are you planning to support this?

I'm not sure. Its use seems rather dubious... it doesn't do anything that can't be done elsewhere, as far as I can see?

As a side note, new beta is posted, this time with a MinGW build as well, which has inline assembly and should be faster.

Changlog here, because I'm lazy: https://github.com/dwbuiten/d2vsource/compare/beta2...beta3

[Keiyakusha]

Daemon404
Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?
Anyway I want to make feature request for it - please don't forget some kind of indication where vobs,cells start/end so we can easily cut exactly at these points. Of course if cutting will be part of the subset at all.

[Daemon404]

Daemon404
Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?
Anyway I want to make feature request for it - please don't forget some kind of indication where vobs,cells start/end so we can easily cut exactly at these points. Of course if cutting will be part of the subset at all.

I didn't. I think you meant lil'jerry. :)

[lil'jerry]

I'm not sure. Its use seems rather dubious... it doesn't do anything that can't be done elsewhere, as far as I can see?

How is it dubious? The force film operation is quite useful for soft telecined input files so you don't have to IVTC. Unless your just skipping all the field repeats in decoding anyway.

Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?


I'm working on a cloned DGIndex/DVD2AVI right now as I announced above.

[Daemon404]

How is it dubious? The force film operation is quite useful for soft telecined input files so you don't have to IVTC. Unless your just skipping all the field repeats in decoding anyway.

As it is right now, neither VapourSynth nor d2vsource has a concept of a 'field'. I do not feel it is worth the complexity of adding this to d2vsource to achieve something you can easily do outside of it, in the same 'perfect' way. There's no value-add for this feature, as far as I can tell...

[lil'jerry]

So then how are you decoding mpeg-2 files with RFF flags set? Are you just always ignoring the repeats or what? ffms2 has the rffmode option that still works under Vapoursynth just as it does under Avisynth. That Vapoursynth has no concept of a field seems irrelevant.

[Daemon404]

So then how are you decoding mpeg-2 files with RFF flags set? Are you just always ignoring the repeats or what?

FFmpeg's stuff handles this, AFAIK.

[lil'jerry]

FFmpeg's stuff handles this, AFAIK.

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

[Keiyakusha]

I didn't. I think you meant lil'jerry. :)

woops, sorry! yes, lil'jerry! :scared:

[Daemon404]

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do.

Again, this is EASILY worked with outside of d2vsource, and I do not want to fill it with unrelated cruft. Why are people so obsessed with ~everything done in the source filter~? I have very purposely not implemented several of DGDecode's bloat features. It sounds to me like all this feature accomplishes is adding a slight convenience, in exchange for a lot more complexity.

EDIT: It is feasible, however, to return rff flags as part of the frame metadata for other filters. Like hinting in AVS.

[Myrsloik]

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

HAHAHA!

Got ya!

I deleted the rff code from the vs side of ffms2 because I was lazy. NANANA:p

[jackoneill]

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

OR, you could just open a progressive DVD with d2vsource and see that it ignores those flags.

[TheFluff]

So then you are pushing through 29.97 with frame repeats even with soft telecine?

The other way around. ffmpeg ignores repeat fields when decoding, so you get only the coded frames. If you want to add the "missing" fields in the correct places you can do so by examining the frame PTS's, I believe. Or if you want to make your life easy just put the repeat_pict number in a per-frame property.

[Guest]

I find this discussion amusing. The source filter is the last one to know about the RFF flags, so if repeated fields are going to be honored, that is the right place to do it. And to coordinate with random access by frame number...it has to take into account the repeats. This stuff is hardly "cruft", if you want to do things in a half-assed way, fine, it's your choice, but don't try to blame someone that has done the right thing.

There are lots of streams with irregular pulldown and ones with mixtures of hard and soft pulldown. If you want to have a hope of processing those correctly, you are going to need to honor repeat flags.

[Daemon404]

I find this discussion amusing. The source filter is the last one to know about the RFF flags, so if repeated fields are going to be honored, that is the right place to do it.

I respectfully disagree. It adds needless complexity to a source filter. The better idea is add it to the frame metadata in VapourSynth, and have a separate filter handle it (IVTC filter, perhaps). There is absolutely no reason to burden the source filter with this, whose job is not to mess with the frames.

And to coordinate with random access by frame number...it has to take into account the repeats.

Um, no. There's is no reason you need RFF flags for "proper" random access. In fact, you shouldn't even have to 'expand' them at all. I'm curious how you define "proper" here.

This stuff is hardly "cruft", if you want to do things in a half-assed way, fine, it's your choice, but don't try to blame someone that has done the right thing.

"Right thing" is subjective here, and you also seem to be bogged down in AviSynth's and its filters' limitations. Your way is merely one way to do it. And for what it's worth, DGDecode has plenty of cruft. It's like a bunch of stuff that just happens to have a source filter in it. How many people actually think BlindPP(), Deblock(), LumaYV12(), or the pp, showQ, cpu, cpu2, etc params of MPEG2Source() are even useful in 2012? Were they ever useful? It's cruft-level is libpostproc-like (which actually does something similar).

There are lots of streams with irregular pulldown and ones with mixtures of hard and soft pulldown. If you want to have a hope of processing those correctly, you are going to need to honor repeat flags.

I've been working with hybrid material (in YATTA, anime obviously) with all sorts of crap like that for about about a decade now. As I said, there is absolutely no reason to handle this in the source filter when you have standard metadata for frames.

P.S. You should take a look at how I'm actually using FFmpeg before you make such statements in the future.

[Guest]

I've had my say, you've had yours. When you have something working along the lines you describe, I suppose you'll let us all know. I won't be holding my breath. :)

And BTW, please stop the profanity per forum rule 4. Thank you. I have edited your post.

[Daemon404]

When you have something working along the lines you describe, I suppose you'll let us all know. I won't be holding my breath. :)

I'll write something this week or next, and maybe add support to VIVTC (we'll see... scary tritical code port), and publish it along with a few other things. I never did say I wouldn't write something to handle it separately (but perhaps bundled).

Anyway, there's no compelling reason to couple the two, since you usually do want the unprocessed video, and VapourSynth has this nice frame metadata (as opposed to *bleck* AVS's hinting).

If anyone has any particularly bad samples, feel free to post them, and I will test them while I write it. I'd appreciate it.

I'm rather surprised people have only been mentioning this sort of thing, instead of the much larger and obvious problem of leading B frames, which I can't seem to stab libavcodec to handle in a nice way no matter what I do, seemingly. Yes, I've tried implementing something similar to what DGDecode does, but libavcodec is having none of that. Help or suggestions are VERY welcome with regards to this.

BTW, perhaps you could be a little less condescending with your posts in the future.

[Guest]

BTW, perhaps you could be a little less condescending with your posts in the future. Maybe when you stop referring to my work as 'cruft'. :)

I'll write something this week or next, and maybe add support to VIVTC (we'll see... scary tritical code port), and publish it along with a few other things. I never did say I wouldn't write something to handle it separately (but perhaps bundled).

Anyway, there's no compelling reason to couple the two, since you usually do want the unprocessed video, and VapourSynth has this nice frame metadata (as opposed to *bleck* AVS's hinting). You can't move pulldown handling out of the source filter, because the source filter must randomly access and display a numbered frame in the pulled-down stream. I'd be interested to hear if you have a solution for that, maybe you haven't thought it through?

I'm rather surprised people have only been mentioning this sort of thing, instead of the much larger and obvious problem of leading B frames, which I can't seem to stab libavcodec to handle in a nice way no matter what I do, seemingly. Yes, I've tried implementing something similar to what DGDecode does, but libavcodec is having none of that. Help or suggestions are VERY welcome with regards to this. You can accomplish the job with libavcodec; I have done that several times. What problem are you running into with libavcodec? There are some streams it may struggle with, but for most stuff it works just fine. You just have to know what it does when you feed it nondecodable frames.

[06_taro]

A souce filter is supposed to preserve whatever exists in the souce and convert it into a structure that the downstream filter can handle. If d2vsouce does put those flags as per-frame properties without loosing anything useful or even normally useless (you never know what weird souces may be and what weird things others may want to do, so a normally useless things may accidentally be useful in some cases), and there are filters knowing how to handle them and convert the clips into different forms according to different purposes in a user-friendly way , then everything is fine. Otherwise we cannot leave those flags to a condition in which it is a pain to deal with.

[Revgen]

1) I've had TS captures in the past that contained TFF and BFF frames. Why? I don't know, but that's how it was captured through the Firewire port. DGIndex and DGMPEGDec's ability to tell which frames were TFF and BFF by reading the RFF flags helped me tremendously to deinterlace the video. I've yet to capture such a file for awhile now, but I never know when it might happen again.

2)Also, I've had to encode silent movie captures that contained different framerates. Parts of the silent movie were 20fps, some were 22fps, and some were 24fps. These movies were originally broadcasted hard-telecined to handle the different framerates. And for good reason. It's a hellava lot easier than using RFF flags. I had to IVTC and use RFF flags for each section (via DGPulldown) and join them together in order for the whole movie to play progressively while keeping the same audio soundtrack. If I ever decided to rip these movies from my DVD-R's again and back them up to a different format, I'd need to have those multiple RFF flags read in order to process the video correctly.

I hope the metadata solution works for these issues.

[Daemon404]

You can't move pulldown handling out of the source filter, because the source filter must randomly access and display a numbered frame in the pulled-down stream. I'd be interested to hear if you have a solution for that, maybe you haven't thought it through?

It's not strictly needed in the source filter. You can frame accurately seek within the coded picture, which is indeed what you way want.

Per-frame metadata mat not be entirely sufficient in this case, actually, so what I'm thinking it something like this:

import vapoursynth as vs

core = vs.Core()

ret = core.d2v.Source(input=r'C:\somefile.d2v', nocrop=True)
ret = core.d2v.ApplyRFF(ret, d2v=r'C:\somefile.d2v')

last = ret

This would nicely decouple the two, while allowing you to apply a d2v's RFF flags to an arbitrary clip (useful if you mess up or something). The other option is to use whole-clip metadata, when it is added to VapourSynth (Myrsloik says it's coming!).

Also, to clarify, ApplyRFF would be akin to honoring the RFF flags. I still do not intend to implement Force FILM.

You can accomplish the job with libavcodec; I have done that several times. What problem are you running into with libavcodec? There are some streams it may struggle with, but for most stuff it works just fine. You just have to know what it does when you feed it nondecodable frames.

In my case, when I tried returning the first decodable frame in place of all the leading non-decodable frames, the entire clip ended up being shorter than it was supposed to be, since it was missing 2-3 frames still, somehow. My general method was to still start at the beginning of the first GOP, and decode until frame N, where N is the first decodable frame, and return that in place of the first N frames.

Can you perhaps elaborate on your strategy?

1) I've had TS captures in the past that contained TFF and BFF frames. Why? I don't know, but that's how it was captured through the Firewire port. DGIndex and DGMPEGDec's ability to tell which frames were TFF and BFF by reading the RFF flags helped me tremendously to deinterlace the video. I've yet to capture such a file for awhile now, but I never know when it might happen again.

The d2v file stashes TFF/BFF info, so that is easy to expose.

2)Also, I've had to encode silent movie captures that contained different framerates. Parts of the silent movie were 20fps, some were 22fps, and some were 24fps. These movies were originally broadcasted hard-telecined to handle the different framerates. And for good reason. It's a hellava lot easier than using RFF flags. I had to IVTC and use RFF flags for each section (via DGPulldown) and join them together in order for the whole movie to play progressively while keeping the same audio soundtrack. If I ever decided to rip these movies from my DVD-R's again and back them up to a different format, I'd need to have those multiple RFF flags read in order to process the video correctly.

I'd be grateful if you could supply a sample for testing when I implement this. :)

[Guest]

ret = core.d2v.Source(input=r'C:\somefile.d2v', nocrop=True)
ret = core.d2v.ApplyRFF(ret, d2v=r'C:\somefile.d2v')

last = ret I don't see what you gain from "nicely decoupling" the two, but hey go for it if it pleases you. Having to load a separate plugin, coordinate the D2V filename between them, etc., appears pointless to me and introduces unnecessary complications. The fact is you need that code somewhere, and it is definitely not cruft.

I still do not intend to implement Force FILM. Why not? It's proven to be quite useful for several types of streams.

In my case, when I tried returning the first decodable frame in place of all the leading non-decodable frames, the entire clip ended up being shorter than it was supposed to be, since it was missing 2-3 frames still, somehow. My general method was to still start at the beginning of the first GOP, and decode until frame N, where N is the first decodable frame, and return that in place of the first N frames.

Can you perhaps elaborate on your strategy? It's no different from what you described. Have you determined where the missing frames are? You may be missing some at the end.

[Daemon404]

I don't see what you gain from "nicely decoupling" the two, but hey go for it if it pleases you. Having to load a separate plugin, etc., appears pointless to me. The fact is you need that code somewhere, and it is definitely not cruft.

Maintainability. And the idea is to have it in the same plugin (dll/so), and in the same namespace (core.d2v), but a different function name. It's the "VapourSynth Way" to do things. Here's a lazy wrapper:

core.std.LoadPlugin(r'C:\d2vsource.dll')

def betterD2VSource(path):
ret = core.d2v.Source(input=path)
ret = core.d2v.ApplyRFF(ret, d2v=path)
return ret

Maybe I'll even bundle it!

Why not? It's proven to be quite useful for several types of streams.

It doesn't seem like anything but a convenience thing to me. Even the most basic IVTC plugin could do this given proper metadata. And that what this is: IVTC. IIRC, TIVTC even supports doing this in AviSynth (by pointing it at a d2v).

It's no different from what you described. Have you determined where the missing frames are? You may be missing some at the end.

They're missing at the beginning on the stream I tested, which DGDecode handles correctly. I might try and read avconv.c/ffmpeg.c to see what they do, but they're also kind of weird on these cases, I think. I haven't tested FFMS2.

[Daemon404]

Just posted Beta4.

Quick summary of things:

4:2:2 Support
MPEG Transport Stream PID Support (Multi-stream)
RFF Support (See README)
Bug Fixes!


Full changelog here (https://github.com/dwbuiten/d2vsource/compare/beta3...beta4).

At some point I plan to enable sliced threading for the MPEG1/2 decoding, and frame threading for RFF application.

URLs in OP.

[Revgen]

Good news. The RFF flags seem to be working. I tried one of my silent movies with 20fps and 24fps sections and the pulldown flags are being properly honored.

Bad news. The GCC Beta4 build seems to be broken. It just hangs when I try to load it. MSVC Beta4 build works fine.