MeGUI development

[Emp3r0r]

I updated GUI so it is resizable... mainly for the Queue tab.

Also, I've found the problem sillKotscha described about video input for mpg; it also affects the click once screen as they both call openVideoSource in the VideoUtil.cs which has the following code:486 string infoFile = VideoUtil.getInfoFileName(fileName);
487 bool putDummyTracks = true;
488 ar = AspectRatio.CUSTOM;
489 if (!infoFile.Equals(""))where infoFile always equals empty string since mpg, ts, m2v, etc normally don't have a "Stream Information" file.

[TheBashar]

Yes.. ugly workaround that you don't need anymore.

You might not, but I do. At least until x264 zones change. But then, you already read my plight with that in a different thread.

The infamous "stupid user" error

Yup, that's me! I live in the blissful world where usually a Ctrl-Z can get me out of my messes. Alas, sometimes you just don't get a second chance.

well then, I think I'm just going to add an above normal priority and be done with it. It's not like normal settings make any sense.. if you have cpu cycle consuming crapware running on a pc that isn't to be used during encoding, the difference in encoding time won't be noticeable. If it is.. your setup is to be blamed.

Were you addressing my comments? If so, I have NO idea what you are talking about! ;) I simply mean that sometimes I like to do something else so I crank the priority down to "Low". Then when I'm done, I like to put it back. Unfortunately, you have a dropdown box where the "Normal" entry partially obscures the "Abort" button. Miss clicking on the "Normal" drop-down item by a few pixels and you can accidentally get the Abort button.

I think there was a misunderstanding because I don't know how a super-duper high priority choice would help. Even on a poor setup like mine running cpu consuming crapware.....

Cheers!

[Doom9]

umm.. it looks like you've been working with an old version.
And I already knew where to look for the info file thing... I simply never bothered to make the code conditional on the input ;)

@berrinam: how's the work on the main form coming along? I haven't done anything since the last release so far.

[berrinam]

Ok, I have completed and tested both the conditional compiling and matroska output. The project is too large (by only 9KB) to attach here, so I posted it on rapidshare.de (http://rapidshare.de/files/3274780/2.1.4_conditional___matroska.zip.html). The three compiled versions (full, x264, snow) I have attached here.

Notes:
-The way I have managed the conditional compiling is just to have three instances of the initializecomponent method within the megui class and the calculator class.
-Matroska muxing is done through mkvmerge.exe. It takes avi, mp4 and mkv input (no raw input) and it supports ac3, aac, mp3 audio formats.
-Both the oneclickwindow class and the autoencodewindow class are aware of mkv muxing.
-The oneclickwindow class now has the option "don't encode audio" which keeps the original audio tracks and muxes them directly into the output mkv (only allowed for mkv output, and it assumes that mkvmerge will accept any audio input set in the oneclickwindow).
-A mkvMuxWindow class has been added which is almost identical to the Muxwindow used for mp4. This should only be used for muxing files created by megui (and associated apps like dgindex, etc), because of the problem with audio muxing (it assumes that the audio track is 0, same for video).
-Languages for mkv muxing are identical to mp4 muxing as they both follow the same iso standard.

[Doom9]

great.. I'll give it a test drive, add the other patches and fixes that are due and try to have a new release this week-end. Oh, and I will have to look into matroska overhead, you didn't list that as one of the things done and I'd like to have a precise calculator.

[berrinam]

I searched a little for matroska overhead before writing it, and the only real consensus I found was that it was hard to estimate because of EBML lacing schemes.

[Doom9]

then what kind of overhead do you use for ac3 and mp3 in mkv? using the avi values would obviously not be correct (way too high), and an approximation will be fine. You cannot exactly determine the mp4 overhead either (I forgot why but bond told me once and it's somewhere in this very forum), but the approximation I used (courtesy of ateme) seems to work out.

[berrinam]

Well, I can't remember exactly what I do with mp3 and ac3, but I have a feeling I use the same overhead values as for aac. I know it's dodgy, but I don't have anything else to work with. I did find a page explaining how matroska overhead works (from avimux gui) here (http://www-user.tu-chemnitz.de/~noe/Video-Zeug/AVIMux%20GUI/en_estimate_overhead.html#overhead_mkv). Unfortunately, I do not know how up-to-date this is, in particular, it does not refer to avc muxing. I found that this thread (http://forum.doom9.org/showthread.php?t=63628) has several methods for estimating mkv overhead.

[Doom9]

well.. aac audio has no overhead ;) Or rather, if you have a video-in-mp4 and add an audio-in-mp4, you won't incurr any additional overhead. So basically MeGUI calculates video overhead only in mp4 mode.

[The Link]

-Matroska muxing is done through mkvmerge.exe. It takes avi, mp4 and mkv input (no raw input) and it supports ac3, aac, mp3 audio formats.

Is adding of vorbis audio also considered in the future? IMHO this would especially make sense for mkv output (besides aac which is a bit behind tuned vorbis versions at some bitrates afaik).

[Doom9]

who knows.. maybe one day.

I am more and more thinking this is the wrong direction though.. all those options make MeGUI more and more inaccessible and harder to use. But there's so much groundwork to be done for a Recode clone that does only x264 and xvid that I'm ages away of even determining the feasibility.

[sillKotscha]

is it possible to add a 'mono' encoding option to the mp3 section... I've "DVB-T streamed" Tarantula (great b-movie :) ) and they broadcastet it with mono sound. Via Wavelab I've made a pseudo stereo-file (mono-mix) and encoded it sepreatly. But it would be nice to have mono support in MeGUI as well. I know, your aim was a kind of full DVD-backup solution and up2date DVDs got very rarly mono sound but...

thanks, Sill

[Doom9]

that doesn't even fall under 90/10.

I'm currently working on the mkv bitrate calculation.. it's based on a lot of more or less educated guesses, but one thing I can say right now: using the mp4 calculations will be rather inaccurate.. mkv has higher overhead. And the calculations get more and more complex with each container. I love the simplicity of MP4 when it comes to that.

[sillKotscha]

that doesn't even fall under 90/10.

= 9 :)

but I'm lost here... what do you mean, your "todo-list"?

mkv bitrate calculus is horror, I guess. Overhead calculations by GKnot are often false when files get muxed by mkvmergeGUI -> undersized files.

Try your best!! :)

[Doom9]

90/10 = features that 90% of the user will use versus those that 10% will use. In software, you normally are happy with 80 / 20, and mono is such a special case, I'm sure not even 10% of what's backed up with MeGUI would be mono sound.

My matroska audio overhead (except vorbis which seems to be rather random) will be accurate, but when it comes to video, you need to know the number of i, p and b-frames and since you cannot know that before encoding, calculations will never be accurate.

[Doom9]

@berrinam: I've been looking over your patch and I have a couple of questions

1) why is MP4 and MKV interchanged in the dropdown on the main window?

2) why is there no call to calc.generateMKVCommandline anywhere in jobutil?

3) where is the mkv muxjob created from the autoencodewindow? (that may depend on number two.. the proper commandline definitely isn't there)

4) what is the workflow to get an mkv with audio/subs? I cannot find any output type override in the video job generation. does that mean that only x264.exe mkv output + muxing audio/subs with mkvmerge is supported. What happens if mencoder is used and / or another codec?

also, please increase the version number, and pm me links to sources and binaries.. I want to check out everything before it is made available to the general public.

[berrinam]

1) MP4 and MKV are interchanged because mp4 is sometimes removed and added to the list (for snow), and I didn't want that stuffing up the order. As mkv should be compatible with every codec, I moved it further up.

2) I forgot to add it when I ported the mkvmerge code from 2.1.3 to 2.1.4 :( I have put the change at the end of this post (it is only two extra lines).

3) Autoencodewindow sets the muxtype variable and then sends that to videoutil in the generateJobSeries function. VideoUtil in turn calls jobutil, hence the problem.

4) I'm not sure what you mean by an output type override ... I just mimicked the way mp4 and avi jobs are handled. For mencoder encoding, it will be saved as avi, then muxed into mkv (copying the style of mp4 muxing, through raw files). This is probably sub-optimal, so I will change it in a moment to go through raw->mp4->mkv.

JobUtil change: in generateMuxJob(...), add if (type == MUXTYPE.MKV)
job.Commandline = gen.generateMkvmergeCommandline(mainForm.Settings.MkvmergePath, job.Settings,
job.Input, job.Output);
just before the return statement.

[berrinam]

Opps, it's still missing a section. I have justed posted the updated mp4muxer class, which also handles mkv muxing (and avi muxing).

[Doom9]

so I will change it in a moment to go through raw->mp4->mkv.That's just what I don't want.. two subsequent muxjobs.

[berrinam]

I understand the problems with matroska, however I don't like being limited to AAC audio when I have x264 video. This is why I implemented the matroska muxing. Maybe it should be limited to x264cli, though, and the other codecs only muxed through avi, with a big warning saying that this isn't using mkv native storage.

However, what IS wrong with having two subsequent mux jobs? We have multiple jobs for multiple video passes; it's similar conceptually. And I don't see (except for the extra job generation code it requires) how it can become an implementation problem. Have I missed something?

[Doom9]

it adds another layer that was never foreseen, and many places where changes have to be made. I've continued to work on the matroska code this morning and there's a bunch of things still left out, which just shows how hard it is to add support for something that the framework wasn't meant for. I'm at a point where I know that if Vorbis audio is to be supported, besides the trouble of not really being able to calculate the bitrate properly (even for the other codecs it's not going to be terribly accurate because there's no way of knowing the frame type distribution before encoding), I will have to rewrite the whole audio part yet again - the software is really meant for just AAC, and the MP3 addition is an ugly hack that now starts to interfere (e.g... you cannot configure two mp3 jobs even if you have matroska output, and I have a lingering suspicion I'll find more in the videoutil or jobutil class).

I'm not sure how mkvmerge handles other codecs in AVI, I presume it can store them natively so that won't be a problem.

[berrinam]

I'm not sure how mkvmerge handles other codecs in AVI, I presume it can store them natively so that won't be a problem.
I think mosu cleared that up just after your post:
AVI is quite ok for MPEG-4 part 2 (aka DivX etc). For AVC it has to be MP4 though.
And since that is the case, it seems that two muxing steps are not needed, except for mencoder-x264 output, so I guess that that should be disallowed.

[Doom9]

so I guess that that should be disallowed.Yup. I'll disable that on my way back home tonight. I think I'm pretty close on finishing the calculator now, just have to check, recheck and reckeck again all the numbers. And then I guess I have to add a lot of stats on overhead somehow because the calculations are more "putting your finger in the wind" rather than accurate.

[berrinam]

@Doom9: a few things that still need to be changed:
-DGIndex needs to be made minimized when it runs. I turned this off so that I didn't have to index the entire film. This is done by uncommenting line 617 of VideoUtil class (VideoUtil.createDGIndexProject()
-The mp4 (and mkv) muxing class is sending WAY too many updates to the log. Fix this by removing log.Append("MKVMerge update: ");
log.Append(frameNumber);
log.Append("\r\n");
from MP4Muxer.getMkvPercent (~line 494-6)
-OneClickWindow shows a popup (I used it for debugging) which it shouldn't. This is in line 1074 of OneClickWindow (the second last line of OneClickWindow.setUpJobs())

[Doom9]

more comments:
x264_only only includes mp4 output (videoOutputOpenButton_Click), it should also offer raw and mkv
theres a lot of conditional code in the calculator that I'm not too sure of. Basically the oly thing that can be disabled is anything that involves NeroAACSettings in snow mode, and anything that involves mp3settings in x264 mode.

the muxer also seems to "run away" at some point. I've just been debugging, and got a 0% message, but never got any higher values.

A bunch of shortcuts in the menu also seem to have mysteriously disappeared. And you can't have the same shortcut for two menu items ;) And to make them work, they have to be set.. using & to underline the subsequent char won't do.. (that paragraph has been taken care off)

I also had to work on the muxer.. if you mux an mp4 audio file, it's track 1 you're interested in, not 0..

[berrinam]

the muxer also seems to "run away" at some point. I've just been debugging, and got a 0% message, but never got any higher values.That could well be linked to a faulty commandline. I have used it, and it has always shown the progress correctly except when it couldn't handle the input files.

A bunch of shortcuts in the menu also seem to have mysteriously disappeared. And you can't have the same shortcut for two menu items ;) And to make them work, they have to be set.. using & to underline the subsequent char won't do.. (that paragraph has been taken care off)

I also had to work on the muxer.. if you mux an mp4 audio file, it's track 1 you're interested in, not 0..
Thanks for both -- didn't know about either.

[Doom9]

@TheBashar: I have found an approach of deleting done jobs that works for me: a setting in the options "remove successfully completed jobs". When a job is completed successfully, and there's no subsequent job linked, it will be removed. If the job in question is part of a series of jobs, then all previous jobs will be removed as well. Since I'm aborting a series of jobs at an error, basically you should only get in the position where all jobs of a series will be deleted if they have all been successfully completed. Naturally, I'd never activate this feature as I tend to redo parts of a series quite often as part of the development process, but you can activate it if you like. How does that sound?

[MeteorRain]

Doom9:
on MeGUI - x264 Codec Configuration - More - Quantizer Matrices, there's a mistake in the filter sentense?
the file-open dialog opened, it filter out all .txt file while the text below says "Quantizer Matrix File (*.cfg)"
i wonder if you write
filter = "Quantizer Matrix File (*.cfg)|*.txt" :o

[Doom9]

filter = "Quantizer Matrix File (*.cfg)|*.txt" No, it's a little different but almost ;)

[TheBashar]

a setting in the options "remove successfully completed jobs". When a job is completed successfully, and there's no subsequent job linked, it will be removed. If the job in question is part of a series of jobs, then all previous jobs will be removed as well.

Thanks Doom9. That sounds like a reasonable way to handle it. Any chance the setting can be made to alter the behaviour of the "Clear" button as opposed to being an automatic removal?

Oh, and thank you for the idiot-proofing. I read about the close during encoding prevention on your frontpage. Believe it or not, I've managed to do that a couple times now, so this idiot will be very glad to have that feature!

Thanks!

[Doom9]

Any chance the setting can be made to alter the behaviour of the "Clear" button as opposed to being an automatic removal?No, clear is really there to start with a clean slate and get around all the question boxes you get when trying to delete a job of a series of jobs. What you want to clear button to do will no longer be necessary as you'll never be in the situation of having properly finished jobs still in the queue when using the new setting. The only ones you might still have would be part of an aborted or errored series of jobs and I strongly believe that those should be kept until such time as the entire series has been successfully completed. You may think that's not necessary but there will be a time when you will be grateful that I didn't change this behavior.

[TheBashar]

You may think that's not necessary but there will be a time when you will be grateful that I didn't change this behavior.

Oh, I agree with you 100% there. That's a much safer way to go. In fact, by suggesting the "remove all successful" function as a user activated button instead of an automagic thing, I was leaning towards even safer yet.

[Doom9]

I think there's been a misunderstanding on what the calculator should do in the minimized mode. The only difference between codec specific and full mode is the codec selection and a limited container selection. But I'm already working on changing that, that way I can get my hands dirty with the conditional clauses as well. And I'm not so sure having two initializecomponent methods is such a great idea.. I've added a button, moved around some others, and there's a new checkbox in the settings.. all that needs to be in both versions.

[berrinam]

And I'm not so sure having two initializecomponent methods is such a great idea.. I've added a button, moved around some others, and there's a new checkbox in the settings.. all that needs to be in both versions. As far as I can see, there is no way around this. It seems certain that conditional code cannot be put inside the InitializeComponent method, as any changes to the gui in the designer will rewrite the method, losing the conditional code.

Mind you, I seem to be stuck with this conditional compiling. It seems there is no way to elegantly manage all the code. I haven't tried the visual inheritance idea you mentioned a while ago, because I'm not sure how this fits with the GUI designer. I thought that the GUI designer only managed forms in which all the elements are defined and handled within the class. However, I may be wrong. Is there a tutorial or reference on visual inheritance you can point me to?

[Doom9]

The only thing I have is in a study book for the MCAD certification. But, you can have a base class with certain base functionality, inherit from that and handle events from the base class as well. Basically any form you create is an instance of a visual inheritance.. it inherits from a base form that has a certain look and functionality (like maximize/minimize, closing the form, etc). You can override the events the base form generates (override OnClosing is one example.. I'm using that to prevent the program from closing when the X button in the main form is pressed and encoding is still under way.

[MeteorRain]

doom9:
sometimes, when x264cli encoder terminated in an irregular situation, the meGUI will still think the file has sucessfully encoded.
those things such as memory overflow will give a done result instead of error.
meGUI should check the final filesize before set the done state. if the file is 0byte, there must occurs errors.

regards
MeteorRain

[stax76]

As far as I can see, there is no way around this. It seems certain that conditional code cannot be put inside the InitializeComponent method, as any changes to the gui in the designer will rewrite the method, losing the conditional code.

you should never edit the designer generated code because either your changes will be overwritten by the code serializer or you break the deserializer.


I haven't tried the visual inheritance idea you mentioned a while ago, because I'm not sure how this fits with the GUI designer.

visual inheritance commonly means that you design a base class including child controls with the designer and use the designer as well for derived classes. Maybe it's a good solution for what you want to achieve. I found it rather troublesome in VS 2003 and since then never used it again and do fine without it.

[Doom9]

I've added the latest sources. I've tried compiling with csc, which works but the resulting binary crashes upon startup. I think it's because of the additional icon that I'm compiling as a resource in VS. Does anybody manage to compile that icon using csc? If so please let me know.. I'd like to include batch files for commandline compiling (there's already one but it produces an executable with the problem described).

[stax76]

I've added the latest sources. I've tried compiling with csc, which works but the resulting binary crashes upon startup. I think it's because of the additional icon that I'm compiling as a resource in VS. Does anybody manage to compile that icon using csc? If so please let me know.. I'd like to include batch files for commandline compiling (there's already one but it produces an executable with the problem described).


when you compile in VS 2003, doesn't show the output pane the exact compiler command-line? In VS 2005 btw project files are MSBuild based which is part of the .NET redistributable meaning project files can be compiled easily with the .NET redistributable.

[Doom9]

when you compile in VS 2003, doesn't show the output pane the exact compiler command-line? In VS 2005 btw project files are MSBuild based which is part of the .NET redistributable meaning project files can be compiled easily with the .NET redistributable.that is new to me.. I've never seen that, and compiling files as resources is very new... but you can dl the sources and if yuo have a working batch file, please share............

[Doom9]

grumpf.. somehow VS managed to kill the beautiful conditional GUIs.. it doesn't show half the GUI classes at all and thinks they're just code classes. I guess I shouldn't have moved a project around in conditional mode. Now I have to go back to the 0.2.1.7 release and port all changes as not being able to use the GUI editor is simply unacceptable.

[Doom9]

I've restored the source. Thank god the last release was still fully functional so it was a simple metter of replacing a couple of files.

[berrinam]

What is the situation with the todo-list? I presume auto deinterlacing is still wanted through the avisynthwindow? Does this mean automatically using Decomb's fielddeinterlace in all NTSC sources, as mezzanine said?

Is there anything else to work on?

[Doom9]

Does this mean automatically using Decomb's fielddeinterlace in all NTSC sources, as mezzanine said?Is that really good enough? I'm not sure

I've updated the todo list with one item.. besides that nothing comes to mind right now.

[berrinam]

Three bugs I've found:

1. MeGUI always deleted completed jobs, irrelevant of whether the checkbox was ticked in the settingsform. This can be fixed by changing line 3623 (the if statement at the end of MeGUI.markJobDone(StatusUpdate) to if (job != null && job.Status == (int)Job.JobStatus.DONE && job.Next == null && settings.DeleteCompletedJobs) The check for settings.DeleteCompletedJobs was missing.

2. If x264 reported an error with the [error] output, MeGUI ignored it, and flagged it as done, even though it was a broken encode. I can produce this sort of error by doing a second pass with more input frames than the first pass. This problem can be fixed by changing this if statement: if (log.ToString().ToLower().IndexOf("Syntax:") != -1
|| log.ToString().ToLower().IndexOf("unknown") != -1)from line 391-2, in VideoEncoder.x264Encoding to if (log.ToString().ToLower().IndexOf("Syntax:") != -1
|| log.ToString().ToLower().IndexOf("unknown") != -1
|| log.ToString().ToLower().IndexOf("[error]") != -1)

3. When opening an avs file, the output name will be auto-selected even if there is a first-pass profile selected, so no output wanted. A fix can be done by adding updateIOConfig(); to MeGUI.inputOpenButton_Click at the end of the if block.

[DigitalDivide]

Just a quick question for a noobie. Is it possible to use meGui to encode movies to .mkv and keep my AC3 stream intact without encoding it to AAC? It's really important to me that I have my AC3 :)

Thanks in advance!

[berrinam]

@DigitalDivide: Yes, it is possible. You have several options for doing this:

1) The OneClickWindow has the option 'Don't encode audio' which will skip the audio encoding stage and mux the original audio straight into mkv.
2) You can set up your video to encode, select mkv filetype, then click autoencode and check 'add additional content (audio, subs, chapters)' which will let you select the audio.
3) You can encode the video yourself (possibly using bitrate calculation from the MeGUI calculator), then mux it with either mkvmerge gui or the megui mkv muxer

[berrinam]

@Doom9: Is video cutting still out of the question? In the HDTV forum, GaveUp has written a small AC3 Cutter which he claims has no more than 48ms sync issues. Could this be combined with AviSynth cutting for video to allow for cutting?

and.... what is audio stream->pid mapping?

[Doom9]

@Doom9: Is video cutting still out of the question? In the HDTV forum, GaveUp has written a small AC3 Cutter which he claims has no more than 48ms sync issues.50ms is considered by many to be the start of visible a/v asynch so that's not really good enough. And in Europe, most digital TV broadcasts use MP2 audio, not AC3, so that would have to be handled as well (and in Japan they have a lot of AAC). Imho, the most appropriate way to handle cutting is an AviSynth AC3 source that decodes all available channels to PCM, and a BeSweet that accepts AviSynth as input (I've never tried). With that, you can have frame accurate cutting and no audible asynch.

what is audio stream->pid mapping?In a transport stream, a PID identifies a stream (audio or video), just like you have substream IDs in a VOB. So, if you have your TS, you need to know which PIDs are used for your audio and video in order to select them. It gets even more interesting if you've been recording the data from an entire transponder.. then you have multiple TV channels that you need to differentiate in between.

[berrinam]

2. If x264 reported an error with the [error] output, MeGUI ignored it, and flagged it as done, even though it was a broken encode.My bad. Although this sometimes means a broken encode, it is also used to signal non-fatal errors. A better error detection method is then needed.