Cross-Platform D2V Source for VapourSynth

IJust wanted to say thanks for providing this. After seeing the announcement for AvxSynth I was thinking of porting DGIndex to something cross-platform but got discouraged by the fact that it's a huge mass of spaghetti code. But after seeing this post, I've started up again a few days ago this time starting from scratch and just re-implementing core features (generating d2v files, demuxing video/audio, d2v fixer, etc.) and leaving out all the other cruft (AC3 decoding, audio resampling, cropping filter, luminance filter, etc). Right now, I have d2v generation and demuxing working and I'm getting identical results to DGIndex in my limited tests. I'm trying to finish up the last few features I want supported and trying to test it as much as possible. Should have something by mid next week for others to test and I'll throw the code up on github. It's written for Qt and uses ffmpeg libs for parsing/demuxing of the streams.

[Guest]

Originally Posted by lil'jerry

IJust wanted to say thanks for providing this. After seeing the announcement for AvxSynth I was thinking of porting DGIndex to something cross-platform but got discouraged by the fact that it's a huge mass of spaghetti code. But after seeing this post, I've started up again a few days ago this time starting from scratch and just re-implementing core features (generating d2v files, demuxing video/audio, d2v fixer, etc.) and leaving out all the other cruft (AC3 decoding, audio resampling, cropping filter, luminance filter, etc). Right now, I have d2v generation and demuxing working and I'm getting identical results to DGIndex in my limited tests. I'm trying to finish up the last few features I want supported and trying to test it as much as possible. Should have something by mid next week for others to test and I'll throw the code up on github. It's written for Qt and uses ffmpeg libs for parsing/demuxing of the streams.

Please rename the tool as I prefer to keep the DG imprimatur for my stuff. Thank you.

Yes, it will be called something different such as D2VCreator since it shares no code but only shares a similar-looking GUI and a subset of functionality of the DGIndex/DVD2AVI programs.

[Guest]

Great, thanks very much and good luck with the project!

Hey Daemon in looking at your code it seems you ignore the "field operation" flag in the D2V file. Are you planning to support this?

[Daemon404]

Originally Posted by lil'jerry

Hey Daemon in looking at your code it seems you ignore the "field operation" flag in the D2V file. Are you planning to support this?

I'm not sure. Its use seems rather dubious... it doesn't do anything that can't be done elsewhere, as far as I can see?

As a side note, new beta is posted, this time with a MinGW build as well, which has inline assembly and should be faster.

Changlog here, because I'm lazy: https://github.com/dwbuiten/d2vsourc.../beta2...beta3

[Keiyakusha]

Daemon404
Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?
Anyway I want to make feature request for it - please don't forget some kind of indication where vobs,cells start/end so we can easily cut exactly at these points. Of course if cutting will be part of the subset at all.

[Daemon404]

Originally Posted by Keiyakusha

Daemon404
Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?
Anyway I want to make feature request for it - please don't forget some kind of indication where vobs,cells start/end so we can easily cut exactly at these points. Of course if cutting will be part of the subset at all.

I didn't. I think you meant lil'jerry.

Originally Posted by Daemon404

I'm not sure. Its use seems rather dubious... it doesn't do anything that can't be done elsewhere, as far as I can see?

How is it dubious? The force film operation is quite useful for soft telecined input files so you don't have to IVTC. Unless your just skipping all the field repeats in decoding anyway.

Originally Posted by Keiyakusha

Hi. You mentioned gui before, but I don't see it. It is not yet ready, right?

I'm working on a cloned DGIndex/DVD2AVI right now as I announced above.

[Daemon404]

Originally Posted by lil'jerry

How is it dubious? The force film operation is quite useful for soft telecined input files so you don't have to IVTC. Unless your just skipping all the field repeats in decoding anyway.

As it is right now, neither VapourSynth nor d2vsource has a concept of a 'field'. I do not feel it is worth the complexity of adding this to d2vsource to achieve something you can easily do outside of it, in the same 'perfect' way. There's no value-add for this feature, as far as I can tell...

So then how are you decoding mpeg-2 files with RFF flags set? Are you just always ignoring the repeats or what? ffms2 has the rffmode option that still works under Vapoursynth just as it does under Avisynth. That Vapoursynth has no concept of a field seems irrelevant.

[Daemon404]

Originally Posted by lil'jerry

So then how are you decoding mpeg-2 files with RFF flags set? Are you just always ignoring the repeats or what?

FFmpeg's stuff handles this, AFAIK.

Originally Posted by Daemon404

FFmpeg's stuff handles this, AFAIK.

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

[Keiyakusha]

Originally Posted by Daemon404

I didn't. I think you meant lil'jerry.

woops, sorry! yes, lil'jerry!

[Daemon404]

Originally Posted by lil'jerry

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do.

Again, this is EASILY worked with outside of d2vsource, and I do not want to fill it with unrelated cruft. Why are people so obsessed with ~everything done in the source filter~? I have very purposely not implemented several of DGDecode's bloat features. It sounds to me like all this feature accomplishes is adding a slight convenience, in exchange for a lot more complexity.

EDIT: It is feasible, however, to return rff flags as part of the frame metadata for other filters. Like hinting in AVS.

[Myrsloik]

Originally Posted by lil'jerry

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

HAHAHA!

Got ya!

I deleted the rff code from the vs side of ffms2 because I was lazy. NANANA

[jackoneill]

Originally Posted by lil'jerry

So then you are pushing through 29.97 with frame repeats even with soft telecine? That seems silly to do. As I mention above, ffms2's rffmode works even under Vapoursynth so that these field repeats can be ignored so IVTC isn't needed.

OR, you could just open a progressive DVD with d2vsource and see that it ignores those flags.

[TheFluff]

Originally Posted by lil'jerry

So then you are pushing through 29.97 with frame repeats even with soft telecine?

The other way around. ffmpeg ignores repeat fields when decoding, so you get only the coded frames. If you want to add the "missing" fields in the correct places you can do so by examining the frame PTS's, I believe. Or if you want to make your life easy just put the repeat_pict number in a per-frame property.

[Guest]

I find this discussion amusing. The source filter is the last one to know about the RFF flags, so if repeated fields are going to be honored, that is the right place to do it. And to coordinate with random access by frame number...it has to take into account the repeats. This stuff is hardly "cruft", if you want to do things in a half-assed way, fine, it's your choice, but don't try to blame someone that has done the right thing.

There are lots of streams with irregular pulldown and ones with mixtures of hard and soft pulldown. If you want to have a hope of processing those correctly, you are going to need to honor repeat flags.

[Daemon404]

Originally Posted by neuron2

I find this discussion amusing. The source filter is the last one to know about the RFF flags, so if repeated fields are going to be honored, that is the right place to do it.

I respectfully disagree. It adds needless complexity to a source filter. The better idea is add it to the frame metadata in VapourSynth, and have a separate filter handle it (IVTC filter, perhaps). There is absolutely no reason to burden the source filter with this, whose job is not to mess with the frames.

Originally Posted by neuron2

And to coordinate with random access by frame number...it has to take into account the repeats.

Um, no. There's is no reason you need RFF flags for "proper" random access. In fact, you shouldn't even have to 'expand' them at all. I'm curious how you define "proper" here.

Originally Posted by neuron2

This stuff is hardly "cruft", if you want to do things in a half-assed way, fine, it's your choice, but don't try to blame someone that has done the right thing.

"Right thing" is subjective here, and you also seem to be bogged down in AviSynth's and its filters' limitations. Your way is merely one way to do it. And for what it's worth, DGDecode has plenty of cruft. It's like a bunch of stuff that just happens to have a source filter in it. How many people actually think BlindPP(), Deblock(), LumaYV12(), or the pp, showQ, cpu, cpu2, etc params of MPEG2Source() are even useful in 2012? Were they ever useful? It's cruft-level is libpostproc-like (which actually does something similar).

Originally Posted by neuron2

There are lots of streams with irregular pulldown and ones with mixtures of hard and soft pulldown. If you want to have a hope of processing those correctly, you are going to need to honor repeat flags.

I've been working with hybrid material (in YATTA, anime obviously) with all sorts of crap like that for about about a decade now. As I said, there is absolutely no reason to handle this in the source filter when you have standard metadata for frames.

P.S. You should take a look at how I'm actually using FFmpeg before you make such statements in the future.