MeGUI Feature Request Thread

[dimzon]

uhm... an alternative would be:
1 - always use the "new" behaviour
2 - add a "optimize for multithreaded decoding" checkbox that will disable "1" and use the 2 (or more - autoset) slices for encoding
And don't forget - there are multiple depended jobs sometime... :rolleyes:

[Sharktooth]

yes, sure. child jobs should be detected and non parallelized with parent jobs.
also stat files should be named with job names to avoid conflicts.

[dimzon]

yes, sure. linked jobs should be detected and avoided by "1".
Prerendering Job reques extensive HDD space - we can't run any Prerendering Job until previous Job output will not be removed from HDD

[Sharktooth]

edited previous post
Prerendering Job reques extensive HDD space - we can't run any Prerendering Job until previous Job output will not be removed from HDD
or ensure there is enaugh free space.

[dimzon]

@Sharktooth
By the way - i'm trying to connect directly with MeGUI developers and you via ICQ (sometimes it's better than forum or PM) but can't.
My ICQ UIN is 107321832

[Sharktooth]

i lost my icq password some time ago and im not able to retrieve it coz the mail address i had specified is no longer existing.
however i'm always on MSN (tbcebola at hotmail dot com) and irc (#x264 on freenode).

[shon3i]

Request:
Add support for Audio encoding via Coding Tehnologies beacouse nero has problems with multichanel audio.

[Doom9]

@dimzon: I don't use ICQ.. never liked that bloatware.

E.g. launch one single-threaded job per core, instead of launching one job and hoping it will use multiple coresMy haircut instantly turned to spike-style when I read that. The interdependencies nightmare is just too frightening. We can have intersecting logfiles, temporary output files and job interdependencies that you don't know about. E.g. it must not be possible to launch an audio dependent video job prior to completion of the audio job. Any interdependent job needs to run in the proper order (but you can break that by moving jobs around). Furthermore, what about progress during simultaneous encoding? What about picking the next ready job to be processed? Now suddenly you add another layer of dependency that is called "don't touch that series because it's being encoded by another "me"".
And then suddenly a job starter wouldn't just have to check for the usual, but check each input and output file for the proper read/write permission to ensure that encoding can proceed.. and then of course it can always happen that the check succeeds, and due to the multithreaded nature, the other job processor starts just in that instance, is a bit faster, goes through the checks as well, everything passes, and then once encoding starts, you have a big explosion because two processes try to write to the same file or one tries to write to a file the other is trying to read from.

[Sharktooth]

lol... well... it's a bit tricky... :D

[Richard Berg]

On my multi-cpu machines, I just run 2 copies of MeGUI (from separate folders, of course, so the xml files don't conflict). It's not that hard, and won't be very common right now.

If we want to support the scenario within MeGUI, the best way would be to have multiple completely independent queues. Each would have its own tab in the UI and its own progress window.

[Sharktooth]

uhm... that way it's ugly.
well, i think this idea should be dropped at this point. we can already default megui to use always 2 threads for x264 even on single core non hyperthreaded cpus (for the reasons explained above).
and we can detect if there are more than 2 execution units and rise the number of threads to 4 for quad core or dual core hyperthreaded cpus...

[Morte66]

On my multi-cpu machines, I just run 2 copies of MeGUI (from separate folders, of course, so the xml files don't conflict). It's not that hard, and won't be very common right now.

Me too, it gets me from about 4.5 to 6fps on the stuff I've been doing lately. That's what made me put the request in. Oh well never mind, and thanks for looking at it guys.

[dimzon]

uhm... that way it's ugly.
well, i think this idea should be dropped at this point. we can already default megui to use always 2 threads for x264 even on single core non hyperthreaded cpus (for the reasons explained above).
And suspicions can be checked up:
We (rissain comminity ) take 2 clips coded with identical adjustments, in 1 and in 2 slices and we measure speed of their decoding on 2 processors nero the decoder (which supports SMP).

1 slice- 169 fps (both of the processor are loaded)
2 slice - 167 fps (both of the processor are loaded)

[Sharktooth]

interesting. maybe nero found a way to multithread the decoder using something different than slices. never looked into it...
however qt uses slices for that purpouse (but as we all know QT = coded like crap...).

[dimzon]

interesting. maybe nero found a way to multithread the decoder using something different than slices. never looked into it...
however qt uses slices for that purpouse (but as we all know QT = coded like crap...).
Elecard h264 encoders perform multithread encoding without slices too
Slices is easyest but not best way for multithreading

[Sharktooth]

indeed... but since i never used the nero decoder i thought it was using slices (like quicktime).
well, if all the mess is for accelerating quicktime playback we can drop even the 2 slices idea for single/non-multithreaded cpus.

[foxyshadis]

Well, Doom9, what about integrating Elder as a possibility? It isn't finished, but it already deals with the multiple encodings, segments, stats files, merging, and all that rubbish for you. And it works for xvid and x264. Obviously it wouldn't be bundled by default, since it's perl, but it'd be really nice to see it officially supported. Could just flip a switch and use its brand of massively multithreaded encoding rather than threads. (And since xvid and x264 support mt encoding anyway, once xvid makes it configurable you could move #threads to a global #cores option.)

[dimzon]

i believe it's easy to port Elder from rerl to C#
i'm interested in it

[Richard Berg]

Integrating Elder looks even more difficult than trying to sort out the dependency difficulties discussed earlier.

The more I think about it, the more I like having independent queues. The existing OO design should make it reasonably easy.

[Sharktooth]

IMHO idipendent queues is an ugly solution.

[dimzon]

IMHO idipendent queues is an ugly solution.
Lets wait for Doom9 refactoring first

[Mutant_Fruit]

IMHO idipendent queues is an ugly solution.
I agree aswell. I wouldn't like the idea of an app i was running having 4 independant queues because i have a dual-core dual-cpu machine.

How about this to help solve the dependancy issue... If a "job" is created, instead put all the jobs and dependant jobs inside an arraylist (just for an example, there's probably better ways of doing it. So we can go something like...


Arraylist Job1 = new Arraylist();
Job1.Add(job1_pass1);
Job1.Add(job2_pass2);
Job1.Add(job3_pass3);
....

Then...
foreach(Job job in Job1)
{
display information in the queue
}

Then, if someone wants to multithread by running two jobs at once, you just have to check that you aren't running two jobs from the one arraylist at the same time. Also, if someone has 10 "Jobs" in the queue, re-ordering them could be made safer because instead of having 20-30 unlinked jobs, we'd now have 10 Arraylists. When they'd try to move "job7_pass3" upwards, instead we'd move the Job7 arraylist above the Job6 Arraylist, then we'd print out the queue again, thus keeping Job7's jobs intact.

I don't know how it works at the moment, but how does that sound? Stupid or not?

[dimzon]

Actually one job series can contain:
multiple independed audio jobs
2-3 depended video jobs (multipass)
mux job

in this case you can run multiple audio jobs at same time


talking about such architecture we are close to "Processor controlled via Data-Flow" ideology (used in all modern CPU)

what does we need to organize such functionality:
jobs must contain references to related jobs (i.e. 2-nd pass must refer on 1-st pass, mux must refer to 2-nd pass and all audio etc)
in this case we can run any job wich related jobs is already done

[berrinam]

IMHO idipendent queues is an ugly solution.
My main gripe with independant queues is that the user would have to be actively thinking about which queue is likely to take the longest and would have to do his/her best to ensure that the encoding time balances up. This is IMHO too much work for the user who is only interested in getting a good end-result video.


How about this to help solve the dependancy issue... If a "job" is created, instead put all the jobs and dependant jobs inside an arraylist (just for an example, there's probably better ways of doing it. So we can go something like...
....
I don't know how it works at the moment, but how does that sound? Stupid or not?
I like this suggestion because it is simple. The issue about audio jobs not relying on each other, which dimzon raised, is not a huge concern, IMO, because the time taken for encoding audio is so small relative to video encoding.

what does we need to organize such functionality:
jobs must contain references to related jobs (i.e. 2-nd pass must refer on 1-st pass, mux must refer to 2-nd pass and all audio etc)
in this case we can run any job wich related jobs is already done
Didn't you suggest this sort of thing before? I agree that this is the optimum solution in customisability and reliability, but probably the most complex code-wise.

[dimzon]

but probably the most complex code-wise.
I don't think so - proper OO design can make it easy

[Doom9]

in this case you can run multiple audio jobs at same timeNo, you cannot, because at the time you pass from audio to video encoding, that's when a bunch of postprocessing routines kick in that make sure you end up with the proper video bitrate.. instead you create an undeterministic waiting period from the time the first audio job has been processed, to the time the second has and the queue can continue.
Actually, it's probably not even as hard to run two queues.. as the statusupdate contains a reference to the job that triggered the update, and related jobs are already linked (they need be for out-of-order processing and proper between job updates).
what does we need to organize such functionality:A job is a part of a linked list.. always has been for about a year now.. you can always get to the start of a series of jobs and to the end of it.. it takes no special treatment. It wouldn't even take too long to change existing functions to start/end jobs and the whole markJobxyz shebang to permit two jobs being processed in parallel (admittedly you have to place locks everywhere or it'll end up in disaster, but if you're a creative locksmith, you'll manage). What I find very perxplexing is that not a single person here but me is even thinking about the most basic question of how the heck you keep a user appraised of what's going on with two jobs in parallel, and how you need to explain a user that suddenly there's only one job being processed when there are 3 left in the queue. And on top of that, what's wrong with opening explorer, select your megui folder, press control-c, go to another folder, press control-v and you now can run megui two times completely independent without no worries whatsoever and not a single line of code needs to be written? What's wrong with the KISS approach for the 0.00001% of the users who cannot live with stock functionality?

[cc979]

just tried the 0.2.3.2097 devbuild works pretty good since the last time i tried, been busy with other things just wondering if there's any plans to show a rendering rate like the same as the video part ?

would be useful for benchmarking and comparing results with others

[Doom9]

just wondering if there's any plans to show a rendering rate like the same as the video part ?And what would a rendering rate be? I understand rendering in terms of POVRay.. essentially an fps rate.

[dimzon]

And what would a rendering rate be? I understand rendering in terms of POVRay.. essentially an fps rate.
encoding_time/audio_duration

[cc979]

And what would a rendering rate be? I understand rendering in terms of POVRay.. essentially an fps rate.

something like kBps per second, processed/rendered sorry about that

[Doom9]

encoding_time/audio_durationBut what does that ratio signify? The only thing I can imagine that would make some sense is indicating a X times realtime indicator. Also, wouldn't it be possible to have a progress indicator even in the first pass? I mean, you should be able to get the track length from avisynth, should you not?

[berrinam]

From the AviSynth documentation for the 'Normalize' filter:
The calculation of the peak value is done the first time the audio is requested, so there will be some seconds until AviSynth continues.So the first pass is done all at one point, so MeGUI can't really get any indication of the progress at any point.

[dimzon]

The only thing I can imagine that would make some sense is indicating a X times realtime indicator.
Yes, i think it's good to have such information

I mean, you should be able to get the track length from avisynth, should you not?
Yes, I can get he track length from avisynth, no problem here (actually I can get samplerate and samplecount -> i can transform it to duration )

[ChronoCross]

MAke it so the source analyser uses "idle" system resources. I have it set to analyse the whole clip. when doing so I noticed that it sucks up all my CPU. Can you make it idle like virtualdub or x264 does while encoding? Thanks

[foxyshadis]

I'd like to have crf mode save stats files. It's possible by custom commandline but it wouldn't hurt encoding at all to just save it by default.

The larger issue, 2-pass based on crf, is quite complex because you can't write the second job until you have the bitrate of the first, so no need for that. I go by "if the crf is good enough keep it, otherwise run a second pass to squeeze more quality out". It's slower but I don't always run the second, plus it's a less arbitrary bitrate target.

[foxyshadis]

2nd req: Can Exhaustive mode be removed from megui? (Whether or not it's removed from normal x264.) I doubt there's any purpose in leaving it in there in megui, when people will see it and think a slower encoding mode will improve quality. (When it's only infintesimally better than umh.) For backups and normal encoding it has no place at all.

[ChronoCross]

2nd req: Can Exhaustive mode be removed from megui? (Whether or not it's removed from normal x264.) I doubt there's any purpose in leaving it in there in megui, when people will see it and think a slower encoding mode will improve quality. (When it's only infintesimally better than umh.) For backups and normal encoding it has no place at all.

The option should still be left available to those who wish to use it. Even if it is a small difference some people are interested in the absolute best quality.

[Richard Berg]

We just need to make sure the default profiles based on it. I have a feeling 90% of users stick to Sharktooth's settings, and the ones who don't are on their own :)

[foxyshadis]

People can copy it into the command line if they need it that badly. I'm not sure why they'd use megui if they need something like esa.

[ChronoCross]

People can copy it into the command line if they need it that badly. I'm not sure why they'd use megui if they need something like esa.

ease of use. I use megui for that reason. I could do everything by hand but I prefer being able to quickly change my settings.

[foxyshadis]

In the other thread I suggested renaming it debug, which could dissuade people who think it provides a noticeable jump in quality, like moving up from dia or hex to umh. How about doing that instead?

[ChronoCross]

In the other thread I suggested renaming it debug, which could dissuade people who think it provides a noticeable jump in quality, like moving up from dia or hex to umh. How about doing that instead?

Yeah but like I said earlier. what is the point. it technically can provide better quality. pengvado said that the others sometimes beat it but not always. it's technically the best one you can use. we should just leave it as is. It's still useable and not really a debug mode.

It was also pointed out that most people just use the profiles as is. Which is the truth. the only people using it are people who know what it does.

[berrinam]

I agree that it is just a debug mode and it would discourage people from using it if it was removed from MeGUI. I agree that there a quite a few people who think they should enable everything, and then complain about how slow it is.

However, MeGUI is the only GUI at the moment which supports all of x264's features, and I don't think that we should give up on that, just for the sake of better user choices.

I think the only bridge between these two problems is some form of warning the user that it is pointless. However, I think the beginning user will just ignore any such warning, so it comes down to the choice between limiting the program and trusting the users, and I prefer to trust the users, as long as they are given a warning.

[berrinam]

MAke it so the source analyser uses "idle" system resources. I have it set to analyse the whole clip. when doing so I noticed that it sucks up all my CPU. Can you make it idle like virtualdub or x264 does while encoding? Thanks
What do other people think about the best way to manage this? I personally don't want mine to run on 'idle', because I want it to have priority over background tasks. I could put a choice somewhere, but where? Settings?

[berrinam]

<Two requests about crf mode>
Added to the list.

[ChronoCross]

What do other people think about the best way to manage this? I personally don't want mine to run on 'idle', because I want it to have priority over background tasks. I could put a choice somewhere, but where? Settings?

The source analyser settings. That would make sense to have it there. I like having it on idle just so my computer doesn't have a cow checking email while analysing a long source.

[berrinam]

Yeah, I think that's probably the best place, but still.... is it really necessary?

Just out of interest, how long does it take to analyse a long source?

[ChronoCross]

Yeah, I think that's probably the best place, but still.... is it really necessary?

Just out of interest, how long does it take to analyse a long source?

29 mins === approx 1 hour per pass.

[berrinam]

So much? I just timed on a whole DVD, and it took me 2 minutes and 11 seconds.

1. Do you have a slow/hi-res input?
2. Have you changed the settings for source detection, so it analyses more than 1% of the film?

[ChronoCross]

I have it analysing 100% of the film. that's why it takes so long. but I'm just being thurough.