What is ReAuthorist?

[aquabubble]

Some of you might be thinking what this ReAuthorist tool is that's being talked about. Well, here I am to fill you in on a few details.

Well, ReAuthorist is the new name for doItFaster4Me and is designed to take the effort out of CCE/Scenarist projects. It combines the functionality of several tools that are currently needed into one neat package, and adds a whole raft of new features that are available nowhere else. It's come on a very long way since the early alpha that was released in January. It still has some more to go before I am ready to release it, but the basic functionality is now there.

Following a rip from DoItFast4U, ReAuthorist performs all of the manual steps that you'd normally undertake to put all the pieces back together again. Not only this, but the first public release will see an interactive interface to let you adjust the amount of compression to use on your different VTSs, showing you in real time what your new DVD size will be - kiss goodbye to your bitrate heartaches! In addition, it will also let you choose which audio and subpicture streams to keep, tracking your new DVD size as it goes.

This first public release will be able to be used with a great many titles, but restricted to PGC-rips only. However, that should be the only major initial restriction. With enough positive feedback I will move on to vobid and cellid ripped projects.

For more information for where I am up to, please feel free to look at my changelog.

[fourtyfour]

servus...

Hy aquabubble.


This first public release will be able to be used with a great many titles, but restricted to PGC-rips only.

Release Date? :)

I´ve read about two weeks, that´s true?

[aquabubble]

I hope so, but as ever it depends on workload/free time. I'm putting in virtually every free minute I can spare to develop it though. Keep your eyes open in this forum for updates.

[fourtyfour]

servus...

Hy aquabubble.

I´m very interested with your tool and how it works.

I can make Audio,subtitle, *.avs with DIF4U.

So how do you use these to get ready in Scenarist?

I mean, the audio have a small delay.
Do you fix that delay?

In your changelog is something to read about ecl for CCE.
Do you use EClCCE for support newer versions of CCE?

Can ReAuthorist handle with Button over Video like Matrix "follow the rabbit"?

How do you handle these all together to import in Scenarist?
Do you use "new from script" for import everything?

BTW: if you need more beta-tester for your tool, let me know. ;)

[aquabubble]

Hi fortyfour,

It analyses all of the demuxed files, AC3, DTS, D2V/M2V etc to identify their attributes, such as duration, bitrate and number of channels - basically, all of the information that Scenarist needs to know about your different assets. If there is any information missing (as in the case of missing audio streams), then it fills in the blanks.

Audio delays should, for the most part, be handled automatically by Scenarist so no alteration of the AC3 is needed. However, I am aware of a problem with some PAL titles, but until a proper workaround can be found I will not implement this into ReAuthorist. Personally, I think there may be a bug in DVD2AVI/mpeg2dec that is causing some frames to go missing. You see if everything worked as expected, no alteration of any of the demuxed files should be needed.

At the moment my ECL only supports version 2.5 of CCE. If someone could give me an example of an ECL from a later version then I will look into implementing this. As an alternative - and this may be the preferred way of doing things - the soon to be released version of ReAuthorist will also create a file compatible with DoCCE4U.

ReAuthorist will recreate your subpicture streams for things like button over video, but in order to get this working IFOUpdate has to do its job properly.

Yes, the file generated by ReAuthorist is a SCP script file so it is imported by using the "New from script" feature - thanks Sonic! :)

If you would like to help beta test, pm me your email address and I'll include you on the distribution list.

[Mikel]

To my knowledge, there is no fully working trial version of CCE above 2.5 available. That's why most people are still sticking to that old version. All versions 2.6 and above have no saving code included.


To automate things, ecl files are really neat. That's the only reason I still use version 2.5. I use an advanced editor (like UltraEdit) to change all the settings i need in the batch-file. This saves quite some time.

Aquabubble: I am really looking forward for your tool. I was thinking about coding something like this myself, but not in C.

Anyway I hope you can release your first beta soon.

Cheers

Mikel

[fourtyfour]

servus...

PM is out.

2.64 Versions works very well.

For 2.66 and 2.67 there is a solution called EclCCE.

It works also very well.

Maybe we can find a way to include EclCCE to import Ecl in CCE.

But first I have to test the beta :D

[Mikel]

ECLCEE, is that some kind of tool? Never heard of before.

What is PM?

Thanks for your help

Cheers

Mikel

[fourtyfour]

servus...

PM= private message (for aquabubble)

For EclCCE use the search button in this forum.

[Mikel]

Thanks for your help regarding ELCCee. I just downloaded it and it was just what i was looking for.

Great

Cheers

Mikel

[mikeathome]

aquabubble, fourtyfour you got PN.
I would like to share CCESP 2.66 automation and missing frames correction with you...

mike

[Shamanis]

Aquabubble,

As per subject, I've also noticed this. Adding up all the frames in a multi-pgc title and then comparing to the number of frames in CCE seems to drop about 15 or so. This causes the video to be about 1 seccond shorter, and leads to duration mismatch in IFOupdate. No serious problem as long as there wasn't a chapter point on that end (which sometimes there is). I'm going to try and manually add those 15 (or whatever) frames to CCE ECL file and see if it encodes correctly, or dumps the CCE Error message into tresulting MPV file.

EDIT: Just tried a file that had dropped 4 frames. Added the 4 frames to the ECL and can't see any error message etc. anyone else wanna try and confirm?

[aquabubble]

shamanis: it's a really annoying problem! I'm not sure that adding frames to the ECL is the way to avoid it, since the mpeg2dec plugin reads the d2v file to find out where the frames are in the m2v. I know from my testing that adding a few frames does not result in error, but adding say 100, does. Also, my tests happened to be with video clips that had frames of black at the end, so couldn't tell whether any "real" frames were being added - what did your test reveal?

Besides, my belief is that frames may be being lost from the beginning as well as the end (and possibly mid-way through as well).

[mikeathome]

Originally posted by Shamanis
Aquabubble,

As per subject, I've also noticed this. Adding up all the frames in a multi-pgc title and then comparing to the number of frames in CCE seems to drop about 15 or so. This causes the video to be about 1 seccond shorter, and leads to duration mismatch in IFOupdate. No serious problem as long as there wasn't a chapter point on that end (which sometimes there is). I'm going to try and manually add those 15 (or whatever) frames to CCE ECL file and see if it encodes correctly, or dumps the CCE Error message into tresulting MPV file.

EDIT: Just tried a file that had dropped 4 frames. Added the 4 frames to the ECL and can't see any error message etc. anyone else wanna try and confirm?

Hi,

you need to patch (add the frames in the .d2v) Add a '2' for every missed frame. Don't forget the '9' at the end. I automated this process incl. the count of the missed frame. Mail me, I'll send you the solution.

mike

[hakko504]

Originally posted by aquabubble
shamanis: it's a really annoying problem! I'm not sure that adding frames to the ECL is the way to avoid it, since the mpeg2dec plugin reads the d2v file to find out where the frames are in the m2v. I know from my testing that adding a few frames does not result in error, but adding say 100, does. Also, my tests happened to be with video clips that had frames of black at the end, so couldn't tell whether any "real" frames were being added - what did your test reveal?

Besides, my belief is that frames may be being lost from the beginning as well as the end (and possibly mid-way through as well). Frames are indeed lost from the beginning if the vob doesn't start with an I-frame.
Unless you have an faulty stream, frames should not be lost in the middle.
And as have been pointed out, sometimes frames are lost from the end. I'm not sure why, but I have a feeling it might be that the last frames are marked as dropped in order to close a GOP, and that DVD2AVI doesn't recognize this, but simply skips those frames when it creates the .d2v, making it a little shorter than it should be.

[mikeathome]

Originally posted by hakko504
Frames are indeed lost from the beginning if the vob doesn't start with an I-frame.
Unless you have an faulty stream, frames should not be lost in the middle.
And as have been pointed out, sometimes frames are lost from the end. I'm not sure why, but I have a feeling it might be that the last frames are marked as dropped in order to close a GOP, and that DVD2AVI doesn't recognize this, but simply skips those frames when it creates the .d2v, making it a little shorter than it should be.

Hi,

this matches exactly my investigations. I did an indeep framecount at the beginning at the end of a vob (choosed a small from a split by VobID) and found, that in 99% of the cases when you split at VobID you'll loose frames (1-5) at the end. Indeed, this might be due to 'artificial' closed GOPs which are necessary at VobID/CellID border. Since a VobID needs to start with an I-Frame I found this not to be an issue.

mike

[aquabubble]

Originally posted by hakko504
Frames are indeed lost from the beginning if the vob doesn't start with an I-frame.

Agreed. In what circumstances would a VOB not begin with an I-frame? Remember that we are dealing with demuxed M2Vs (one big one normally), not that it should make any difference.


And as have been pointed out, sometimes frames are lost from the end. I'm not sure why, but I have a feeling it might be that the last frames are marked as dropped in order to close a GOP, and that DVD2AVI doesn't recognize this, but simply skips those frames when it creates the .d2v, making it a little shorter than it should be.

Interesting... what do you mean by "marked as dropped"? I wish I had the time to delve deep into DVD2AVI to look into correcting this!

[aquabubble]

Originally posted by mikeathome
you need to patch (add the frames in the .d2v) Add a '2' for every missed frame. Don't forget the '9' at the end. I automated this process incl. the count of the missed frame. Mail me, I'll send you the solution.

Hi mikeathome,

I had a look at the solution you sent me - thanks for that, it looks interesting. However, one thing that I find most intriguing in all of this is that the virtualdubmod for mpeg2 is based on DVD2AVI, yet it gets the frame count right! Now if we can work out what it's doing differently...

[hakko504]

Originally posted by aquabubble
Interesting... what do you mean by "marked as dropped"? I wish I had the time to delve deep into DVD2AVI to look into correcting this! Instead of encoding a black frame, It is possible to set frametype to Z, instead of IPB. This is equivalent to a dropped frame - it's a frame of size Zero.

And if you use FAT(32) there is a lot of reasons why a vob wouldn't start with an I-frame. ;) But you are right - when dealing with vobs there is very few occasions where the vob won't start with an I-frame. I can't really think of any, as long as all cuts are made at cell boundaries.
Some versions of DVD2AVI can handle transportstreams as well, and those almost never start with an I-frame, but that's a different thing.

[aquabubble]

Originally posted by hakko504
Instead of encoding a black frame, It is possible to set frametype to Z, instead of IPB. This is equivalent to a dropped frame - it's a frame of size Zero.

Just been looking into this to see if it would be possible to find out where/how many dropped frames there would be. According to my documentation (dating back to 1995), the picture_coding_type is a three bit value:

000 - forbidden
001 - intra-coded (I)
010 - predictive-coded (P)
011 - bidirectionally-predictive-coded (B)
100 - shall not be used (dc intra-coded (D) in ISO/IEC11172-2)
101 - reserved
110 - reserved
111 - reserved

Hmmm... which one would be Z?

(we are seriously off-topic here - should we take this to the DVD2AVI forum?)

[Eyes`Only]

Everyone,

We're constantly on the lookout for a viable solution to add to our apps to fix the vobid demux. I'm also aware that the subtitle vobid demux doesn't work right and I'm working on resolving that too.

In addition, I've talked with Light_UK, to see if the problem can be resolved by having him make DVD Decrypter write the .m2vs differently. If anyone has suggestions on how he could do this, he's ready to listen. We realize this is a big issue and as soon as a viable solution can be found it will be implemented. Personally, I'm looking at fixing it from the DVDDEC side, if possible, since that seems to be where the problem stems from.

[hakko504]

@aquabubble

I've spent the evening reading the DVD2AVI code, trying to find exactly what happens at the end of a file but I haven't been able to draw any certain conclusions - except one: if you don't have a closed gop at the end of a vobID this could lead to the last two B-frames ending up in the next .m2v! There are some mystic flags being parsed also, but I'm not sure what all of them does, and if that could be interpreted as bad frames.

And feel free to move (or copy) this thread to the DVD2AVI forum. I also think we should ask Belgabor if he knows anything about this as VirtualDubMod does deliver those last missing frames.

[trbarry]

I've never understood this either but would love to have someone explain it to me. I'm making a couple small changes to the save-oe version of DVD2AVI now anyway.

As far as dropped frames on the front ... I notice that some dshow filters appear to use duplicates of the first I-frame to fill in the positions until the GOP with that I-frame really starts. But I don't know if that's a good idea, or how to implement it.

And I'm not sure of the implications for syncing audio. Audio processing especially is a part of DVD2AVI that I have never understood.

- Tom

[int 21h]

I think the thing that many people assume is that if their audio and video durations are not exactly the same.. then they will have desynchronized playback, original files, demuxed from the original source (not using DVD2AVI), often show that the durations are off by small amounts. This leads me to believe that the 'fix' may already exist in the standard, and that we are simply not interpeting a flag correctly in the software.

Anyways, I believe the dropped frame flag that hakko is referring to is referenced here: (@mpucoder's site) (http://members.aol.com/mpucoder/DVD/mpeghdrs.html#gop)