MeGUI development

[Sharktooth]

i wouldnt touch that code (the first rule in programming is "dont fix it if aint broke").
we're going toward a stable version so such changes should be in the next versions.

[berrinam]

Yes, certainly. Only bugfixes should be in the current versions, I was just thinking about the future (the good thing about CVS is that you can work on multiple things at once).

[berrinam]

@Sharktooth: Have you tried the SVN migration again? Is a nightly tarball actually necessary, because these are apparently deprecated, so waiting for them may mean we never migrate. Could you make me an admin in MeGUI, so I can have a look round that stuff as well please?

[berrinam]

0.2.3.2170
Commit by berrinam:
- Catch DirectShow exceptions
- Force Source Detection to only display results _after_ it has finished

[berrinam]

Should we divide the MeGUI source into folders? I think we are getting too many files in the one folder -- too disorganised.

[ChronoCross]

if you can come up with a decent folder structure.....I've seen some projects where the folder says NOTHING about the content. That sucks big time.

[berrinam]

As I see it, the source code in MeGUI can be divided into the folowing, functionality-wise:

Core -- the classes that are essential to MeGUI
\
\
-Queue -- manages the queue
-Profiles -- manages the profiles
-Plugin-manager -- although we don't actually support runtime plugins, we're heading towards that
-Interfaces (for plugins) -- VideoReader, IJobProcessor, ISettingsProvider, possibly also a IJobCreator (see later)
-Job creation -- manages the plugins, etc to create jobs
Core-GUI -- the bits of the GUI that are essential to MeGUI
\
\
-The main form
-Settings form
-Updater
Utils -- Dunno how useful this is
\
\
-VideoUtil.cs, basically, but without the job creation tools
Packages: if we get a very dynamic structure, most of the tools will end up being here
\
\
-Codecs, implementing ISettingsProvider, for both Audio and Video, which includes also their CodecSettings and SettingsForms
-Job creators (AutoEncode, One-Click, D2V, etc), implementing IJobCreator
-Job processors, implementing IJobProcessor
That makes it a pretty extensible mechanism, if possible. Then, we could group the files into folders by the top-level groupings I outlined.

IJobCreator: Basically, the AutoEncodeWindow and the One Click Encoder are both just interfaces to the same encoding backend, and there is no real reason to integrate them strongly into MeGUI. So instead, we could make them implement IJobCreator, so extra tools like them can be dynamically added to the Tools menu.

[Doom9]

I wanted to bring up another thing for quite a while now: Should the preview window be subclassed for the various purposes? I bring this up because after the next stable release there's going to be one more mode of operation: cutting.

Also, what do you think about the workflow when it comes to cutting? I feel right now it's already pretty complex with where you set your start/end credits and where you set your zones... cutting will make it even more complex. Any suggestions for that?

[berrinam]

We need a way to store info about the videos that we are dealing with. This info will mean zones, cuts and compressibility check info. I'm against the idea of having 'project files' because I like the idea that you can simply load a file in MeGUI and encode it immediately without having to mess around with projects. I think what could work is saving cut-files (and compressibility-check files and possibly zones-files) in the same directory as the input AVS file, but with a different extension. Then, MeGUI could auto-detect these when loading the AVS file, and load the cuts/etc as well. This is clear to the user, because it would show 'using cutpoints from ___ file'

As to the workflow, cutting comes in the AviSynth stage, because that's what it is -- editting the video (as opposed to credits/zones, which are editing the encode). This avoids confusion between cutting and zones/credits. Adding a 'cut-file' field in the audio encoding section would also allow for integration with audio cutting.

We could also introduce a reference in the AviSynth file to the location of the cut-file, so that when deleting intermediate files, we can delete the cut-file as well as the avisynth file.

[Sharktooth]

@Sharktooth: Have you tried the SVN migration again? Is a nightly tarball actually necessary, because these are apparently deprecated, so waiting for them may mean we never migrate. Could you make me an admin in MeGUI, so I can have a look round that stuff as well please?
Automatic SVN migration just fails and as you said nightly tarballs are gone...
The migration can be done with rsync but i havent time to read the "how to..." etc.
however it seems i cant update your status to admin (maybe doom9 can though).

We need a way to store info about the videos that we are dealing with. This info will mean zones, cuts and compressibility check info. I'm against the idea of having 'project files' because I like the idea that you can simply load a file in MeGUI and encode it immediately without having to mess around with projects. I think what could work is saving cut-files (and compressibility-check files and possibly zones-files) in the same directory as the input AVS file, but with a different extension. Then, MeGUI could auto-detect these when loading the AVS file, and load the cuts/etc as well. This is clear to the user, because it would show 'using cutpoints from ___ file'

As to the workflow, cutting comes in the AviSynth stage, because that's what it is -- editting the video (as opposed to credits/zones, which are editing the encode). This avoids confusion between cutting and zones/credits. Adding a 'cut-file' field in the audio encoding section would also allow for integration with audio cutting.

We could also introduce a reference in the AviSynth file to the location of the cut-file, so that when deleting intermediate files, we can delete the cut-file as well as the avisynth file.
I like it :)

@all: sorry but nero aac encoder had to go from the auto-update until we manage to work this license thing around...

[berrinam]

0.2.3.2171
Commit by berrinam:
- Fix a bug which caused some mux paths not to be found

[Doom9]

Shouldn't the Cutlist directly go into the script when it's being generated? After all it's the script that will be used for the encoding configuration (credits, zones) and if it's not directly applied, setting zones will be a heck of a mess (you have to recalculate every start and end frame with respect to the cuts).

[berrinam]

I meant that, but we also need to be able to remember what the cuts were so that identical ones can be done for audio.

[Doom9]

Of course.. shouldn't we keep cuts in memory as well though so as not to have to write and re-open files all the time?

[berrinam]

Yes.... not that it makes much difference:
1. write the avisynthscript+ cuts-file
2. Open your audio files
3. Read the cuts file to encode the audio.

It's only one extra read....

[Sharktooth]

we can store cuts in the avisynth script as MeGUI macros too...

[berrinam]

0.2.3.2172
Commit by berrinam:
- Fix 'Queue analysis pass'

[berrinam]

I declare 0.2.3.2172 a stable version, because it's had many bugfixes and I now have some new features prepared which I am going to commit. Hopefully, someone can put this version up on SF, so it is 'official'. I don't know if we want a numbering change, so I'm going to continue with the current numbers, and someone else can suggest a new system if they want.

Autoupdate will continue to be updated to the latest versions of MeGUI, so if people want to stay on a stable build, they should set core updates to be ignored. A better system for dealing with this will come eventually.

[berrinam]

Back to development builds now:
0.2.3.2173
Commit by berrinam:
- Add support for MediaInfo

[berrinam]

Any comments on my post about folders and plugins (http://forum.doom9.org/showthread.php?p=842136#post842136)?

[Sharktooth]

Uhm.... 2172 should be put on SF. However there are still bugs to fix :(
I think i can work a bit on MeGUI on this weekend.
About folders, yes a reorganization is strongly needed.

[Sharktooth]

0.2.3.2174
Commit by Sharx1976:
- Fixed MediaInfoWrapper.dll resource dependancy in the project file
- Added a confirmation MessageBox when clearing the queue
- Changed a menu item title to "Avisynth Script Generator" to keep consistency with the form title

0.2.3.2175
Commit by Sharx1976:
- Restored the menu item title to "Avisynth Script Creator" and changed the title of the form instead

[berrinam]

If I have this for video:
interface IVideoReader
{
Bitmap readBitmap(int frame);
}
then what should I have for audio?

[dimzon]

If I have this for video:
interface IVideoReader
{
Bitmap readBitmap(int frame);
}
then what should I have for audio?



// fast method, returns how much bytes are readen
long ReadAudioSamples(long nStart, int nAmount, IntPtr buf)

// slow method
byte[] ReadAudioSamples(long nStart, int nAmount)

[berrinam]

Thanks

[dimzon]

Thanks
talking about fast/slow implementation I propose

// write RGB data directly into memory buffer
int ReadVideoFrames(long nFirstFrameNumber, int nFramesCount, IntPtr buf)

[berrinam]

(We might as well do it for completeness, but) Why does MeGUI need fast video reading?

[berrinam]

// fast method, returns how much bytes are readen
long ReadAudioSamples(long nStart, int nAmount, IntPtr buf)

// slow method
byte[] ReadAudioSamples(long nStart, int nAmount)


How is this supposed to behave with non-byte-sized samples?

[dimzon]

(We might as well do it for completeness, but) Why does MeGUI need fast video reading?
Current implementation via readBitmap is really slow - it creates/destroys multiple GDI objects when You use PREVIEW ability...

How is this supposed to behave with non-byte-sized samples?
16 bit Stereo = 2*2 = 4 bytes ;)

[berrinam]

Ok, I'll write the prototypes as you described and you can write the implementations ;)

[berrinam]

I see this code at the moment for VideoReader:
public abstract class IVideoReader : IDisposable
{
public abstract void Close();

public abstract Bitmap ReadFrameBitmap(int framenumber);

#region IDisposable Members

void IDisposable.Dispose()
{
Close();
}

#endregion
}

and in d2vReader:
public override void Close()
{
closeD2V();
GC.SuppressFinalize(this);
}
Why do we need to call GC.SuppressFinalize(this); in d2vReader, and is it necessary to implement IDisposing directly in the base VideoReader interface? Shouldn't that be implemented by an abstract class which also implements IDisposing?

[dimzon]

Why do we need to call GC.SuppressFinalize(this) in d2vReader
To avoid unnececary Finalizer call to speedup garbage collector a lot
Actually (according MSDN (http://msdn2.microsoft.com/en-us/library/system.idisposable.aspx) guide) if your class contains unmanaged resources itself you must

implement finalizer, cleanUp all umnamaged resources in it
implement IDisposable.Dispose(), cleanUp all unmanaged resources in it, call IDisposable.Dispose to all disposable members, disable futher finalization via GC.SuppressFinalize(this)

using System;
using System.ComponentModel;

// The following example demonstrates how to create
// a resource class that implements the IDisposable interface
// and the IDisposable.Dispose method.

public class DisposeExample
{
// A base class that implements IDisposable.
// By implementing IDisposable, you are announcing that
// instances of this type allocate scarce resources.
public class MyResource: IDisposable
{
// Pointer to an external unmanaged resource.
private IntPtr handle;
// Other managed resource this class uses.
private Component component = new Component();
// Track whether Dispose has been called.
private bool disposed = false;

// The class constructor.
public MyResource(IntPtr handle)
{
this.handle = handle;
}

// Implement IDisposable.
// Do not make this method virtual.
// A derived class should not be able to override this method.
public void Dispose()
{
Dispose(true);
// This object will be cleaned up by the Dispose method.
// Therefore, you should call GC.SupressFinalize to
// take this object off the finalization queue
// and prevent finalization code for this object
// from executing a second time.
GC.SuppressFinalize(this);
}

// Dispose(bool disposing) executes in two distinct scenarios.
// If disposing equals true, the method has been called directly
// or indirectly by a user's code. Managed and unmanaged resources
// can be disposed.
// If disposing equals false, the method has been called by the
// runtime from inside the finalizer and you should not reference
// other objects. Only unmanaged resources can be disposed.
private void Dispose(bool disposing)
{
// Check to see if Dispose has already been called.
if(!this.disposed)
{
// If disposing equals true, dispose all managed
// and unmanaged resources.
if(disposing)
{
// Dispose managed resources.
component.Dispose();
}

// Call the appropriate methods to clean up
// unmanaged resources here.
// If disposing is false,
// only the following code is executed.
CloseHandle(handle);
handle = IntPtr.Zero;

// Note disposing has been done.
disposed = true;

}
}

// Use interop to call the method necessary
// to clean up the unmanaged resource.
[System.Runtime.InteropServices.DllImport("Kernel32")]
private extern static Boolean CloseHandle(IntPtr handle);

// Use C# destructor syntax for finalization code.
// This destructor will run only if the Dispose method
// does not get called.
// It gives your base class the opportunity to finalize.
// Do not provide destructors in types derived from this class.
~MyResource()
{
// Do not re-create Dispose clean-up code here.
// Calling Dispose(false) is optimal in terms of
// readability and maintainability.
Dispose(false);
}
}
public static void Main()
{
// Insert code here to create
// and use the MyResource object.
}
}

is it necessary to implement IDisposing directly in the base VideoReader interface?
I just like to write something like

using(IVideoReader vr = new BlaBlaBlaReader(...))
{
// bla bla bla
}

in this case IVideoReader MUST implement IDisposable

[berrinam]

I just like to write something like

using(IVideoReader vr = new BlaBlaBlaReader(...))
{
// bla bla bla
}

in this case IVideoReader MUST implement IDisposable
But IVideoReader should simply inherit IDisposable, NOT have an implementation of it:


THIS IS WRONG:
public abstract class IVideoReader : IDisposable
{
public abstract void Close();

public abstract Bitmap ReadFrameBitmap(int framenumber);

#region IDisposable Members

void IDisposable.Dispose()
{
Close();
}

THIS IS RIGHT:
public interface IVideoReader : IDisposable
{
void IDisposable.Dispose();
Bitmap ReadFrameBitmap(int framenumber);
}

Do you agree?

[berrinam]

Uhm.... 2172 should be put on SF. However there are still bugs to fix :(I think we've stopped development for long enough to remove most of the bugs. While there certainly are bugs (like the safe profile alteration one) MeGUI is still nowhere near a complete release (the updater really needs some work to become a proper solution), so continued development on real features, not bugs, is IMHO important.

[berrinam]

0.2.3.2176
Commit by berrinam:
- Add '-threads' option to xvid config
- Fix 'safe profile alteration' feature. Does anyone actually need this?

[berrinam]

0.2.3.2177
Commit by berrinam:
- Fix profiles not being saved when closing bug
I know I said earlier it's better to be safe than sorry regarding the CVS->SVN conversion and history, but I really would like some more advanced stuff like SVN and trac, so I'm interested in that. I'm no longer convinced we actually need the history. What does everyone else think?

PS. Who would host trac if we moved to that?

[dimzon]

I'm no longer convinced we actually need the history. What does everyone else think?
No, please, keep history

[buzzqw]

noooo please !!! i love to read changelog !

BHH

[berrinam]

noooo please !!! i love to read changelog !

BHH
History, as in CVS history, not the changelog. What I mean is that the CVS history IMHO needn't all be transferred to SVN.

@dimzon: Why do you think we should keep the CVS history?

[Doom9]

The history Berrinam is speaking of is the CVS history.. the changelog (4th tab) will remain. Either way, I have many many old versions still archived. You can check out every revision so that when needed, a diff between two revisions can always be made. But rare are the ocurrences when you actually have to go back to something that was. The only time I ever needed to go back I ended up using a hard copy and manually merging files to get back to an usable state.

[Sharktooth]

@berrinam: i managed to upgrade your status to admin, so now you have the possibility to have a look at subversion migration.
However if automatic cvs->svn fails there are very few chances we can keep history since cvs tarball are deprecated.

[Sharktooth]

megui SVN has been resynched (without history though).

[Doom9]

about catching stdout/stderr: recently I adopted a slightly different strategy at reading stdout/stderr output from a process. When you activate events, there's an event that's triggered whenever something is written to a redirected output.. I'm wondering whether that might rid us of the annoying "megui doesn't get the error message that x264 shows before crapping out" thing (where people consistently forget to think just a little bit before posting the log and realize that a second pass that only lasts for a few seconds obviously is no real second pass)

[daverc]

about catching stdout/stderr: recently I adopted a slightly different strategy at reading stdout/stderr output from a process. When you activate events, there's an event that's triggered whenever something is written to a redirected output.. I'm wondering whether that might rid us of the annoying "megui doesn't get the error message that x264 shows before crapping out" thing (where people consistently forget to think just a little bit before posting the log and realize that a second pass that only lasts for a few seconds obviously is no real second pass)


I used this approach for xAnime, it works well but it is not perfect either. Some exe don't like it. I don't remember why but for for X264.exe i had to use a helper class called ProcessCaller, which was actually my 1st choice. However for mp4box and mkvmerge watching stderr/stdout with events was working well. If you ever want to see what i mean, you can find the sources here (http://detritus.sobanet.com/files/xanime/xAnime.0.1.5.full.7z).

The classes for command line processing are CLIEncoder for management with events, and inherited X264CLIencoder with overriden methods using ProcessCaller helper. I also use some kind of notifier to report progress of CLI outputs.

[Sirber]

I used this approach for xAnime, it works well but it is not perfect either. Some exe don't like it. I don't remember why but for for X264.exe i had to use a helper class called ProcessCaller, which was actually my 1st choice. However for mp4box and mkvmerge watching stderr/stdout with events was working well. If you ever want to see what i mean, you can find the sources here (http://detritus.sobanet.com/files/xanime/xAnime.0.1.5.full.7z).

The classes for command line processing are CLIEncoder for management with events, and inherited X264CLIencoder with overriden methods using ProcessCaller helper. I also use some kind of notifier to report progress of CLI outputs.That's why I gave up on .NET. Not programmer friendly.

[Doom9]

Hmm.. basically your Processcaller seems to be doing something very similar than my reader.. I also have two threads reading the outputs and firing events when something is being received. I even added something on top of that which keeps those reading threads running up until there's really nothing left in both readers anymore. I have no idea how x264 manages to bypass my code.

[daverc]

That's why I gave up on .NET. Not programmer friendly.
Right, and that's exactly on these lines of code where you left.
And where i started.
I spent countless hours to do useless experiments to figure out a working solution.


Doom9, I'm pretty sure you already tried this one. . If not, It seems to be .Net 2 standard approach. Notice that the trigger to report completion of the process is somehow hidden : EnableRaisingEvent. This time there's only one thread. Its main flaw is issues to deal with the order of the events when stderr and stdout outputs simultaneously. At least this time you can be sure not to stop reading before every message has been processed.

One more thing you can do is read the exitcode of the process.
Let's hope x264.exe return something far from -1 on error.

Last workaround to losing error notification can be to store every output string. Ie for debug purpose, store up to 128 kb, for regular use, store only last 10 lines.




Process calcProc;
List <string> stdout=new List <string> ;
List <string> stderr=new List <string> ;

private void AsyncExec(string CmdLine, string CmdParams)
{


calcProc = new Process();
ProcessStartInfo i = new ProcessStartInfo();

// Start info
i.FileName = CmdLine;
i.Arguments = CmdParams;

i.RedirectStandardOutput = true; // makes MP4Box fail
i.RedirectStandardError = true; // makes x264 fail
i.CreateNoWindow = true;
i.UseShellExecute = false;

calcProc.StartInfo = i;


//Redirects the output of the console
calcProc.OutputDataReceived += new DataReceivedEventHandler(calcProc_OutputDataReceived);
calcProc.ErrorDataReceived += new DataReceivedEventHandler(calcProc_ErrorDataReceived);

// in case event is not catched by any eventhandler
//new MethodInvoker(this.calcProc_OutputDataReceived).BeginInvoke(null, null);
//new MethodInvoker(this.calcProc_ErrorDataReceived).BeginInvoke(null, null);

//Enables the redirection of the event fired at completion of the process
calcProc.EnableRaisingEvents = true;
calcProc.Exited += new EventHandler(calcProc_Exited);
// Start
calcProc.Start();
//Starts the redirection of the output

calcProc.BeginOutputReadLine();
calcProc.BeginErrorReadLine();

}

//On end of the process write the time and launch next step
void calcProc_Exited(object sender, EventArgs e)
{if (calcProc.ExitCode!=-1) DoSomeThingWithExitCode(calcProc.ExitCode);

DoSomeThingOnCompletion();
}


//for X264 the progress is displayed on std error
void calcProc_ErrorDataReceived(object sender, System.Diagnostics.DataReceivedEventArgs e)
{stderr.Add(e.data);
DoSomeThingToReportStdErr(e.data);
}

void calcProc_OutputDataReceived(object sender, System.Diagnostics.DataReceivedEventArgs e)
{stdout.Add(e.data);
DoSomeThingToReportStdOut(e.data);
}

[daverc]

Btw, processCaller is borowed from CodeProject (http://www.codeproject.com/csharp/LaunchProcess.asp)
As you noticed, it uses threads to handle the readings.
It has been written in 2003, and is obviously .net 1.1.
I just added a kill method, maybe it would need an exitcode as well. I tried to get rid of it with above code, but i had no success with x264.exe.
However MeGui already has everything required to deal with command lines.

[bob0r]

megui SVN has been resynched (without history though).

Will megui updates be in the sf SVN now?
(auto update script is ready to roll!)

[berrinam]

Yes, they should.