language choice for building TS demuxing tool

[stax76]

Can somebody share insight about pros and cons of programming languages suitable to build a TS demuxing tool. I like to learn and understand different languages and language innovations so I follow the development of different languages. Based on this I made a preselection of 3 possible languages I might use for a tool I consider to program:

C++
C#
Rust

I'm not sure which criteria I should use, I'm a experienced VB.NET programmer and know C# quite well, using C# would have the advantage of getting more accustomed to C# and become more productive, if I try I might be able to like some things about the language. Since I already know the language and it's a very easy language and platform I could focus on the application. My second thoughts are performance, I aim for the fastest possible throughput and I don't know if C# is suitable for it, with .NET Native and clever JIT optimization I know it can be very fast, I'm just not sure if it's the best choice for this type of applications, I've never programmed something similar.

Rust could potentially be the most fun, there are many things I like about Rust and there is indication that Rust is gonna be successful, many news sites constantly have news and articles on Rust. I've not programmed with it yet and might as well hate it after using it for a wile but probably I will like it, I've a very good feeling about it like I had with .NET which I use and like since 1.0.

C++ the advantage would be it's a very important language, audio/video programming pretty much everything is C/C++, downside is I never liked C++ and because of this not used it often and for longer time, I find it difficult. On the other side I could gain much in mastering C++ using existing source code and APIs.

Which one of the three would you use or not? Why?

[videoh]

C/C++ is the obvious choice for performance reasons (you can code in C-style using a C++ compiler, BTW, thereby avoiding the overhead of object-oriented coding). That you dislike it is a very bad reason to reject it. I suggest that your "feelings" about a language should play little or no role in your choice.

But beyond that, what is your motivation to write your own demuxer when there are many fine ones already available, many with source code available? Can't you start with one of them and then modify it for your own needs if necessary? I'm curious to know what you plan to do that is not already done by available demuxers? I want to warn you that this is not a simple project like whipping up a GUI in VB.NET. You will need deep familiarity with transport and elementary streams and how to parse them, for instance. That is likely to be much more challenging than mastering C/C++.

Here you will find code for a demuxer called xport written in C-style. This will give you a good idea of what you are getting into. This could be a fine starting point for you to modify for your needs. Scroll down the page to find it.

http://www.w6rz.net/

This code by (IIRC) Ron [drmpeg] is a glittering jewel of C-style programming. You should study it carefully. But hey, where are the header files? ;) You'll quickly see how complicated transport stream handling really is.

The source code for DGIndex is also available and is written in C-style. I'm sure there are many others, but if you tell us what you hope to accomplish we can direct you more specifically.

Don't kid yourself that Rust is any easier to become proficient at than C/C++. It's arguably the other way around, especially for C-style coding. And while it may outperform C/C++ in a few use cases, generally C/C++ is the performance winner (especially for C-style coding). And don't forget that Rust is relatively immature and you may have problems interfacing to libraries written in C/C++ (there are few Rust libraries at this time). There are good reasons that "audio/video programming pretty much everything is C/C++".

http://www.quora.com/Should-I-learn-C-or-Rust-first

Finally, stream parsing involves heavy bit-field manipulation and Rust does not bring much to the table in that regard.

Flame away, everybody!

[stax76]

C/C++ is the obvious choice for performance reasons

I guess C++ is often chosen for performance reason but if you look at this closer isn't ASM what you use if you want to make something really fast? As far as I know Rust is aiming to be high performance as well with zero-cost abstractions, unsafe code and ASM support.

I'm curious to know what you plan to do that is not already done by available demuxers?

There aren't many, I don't want StaxRip users having to install Java only for MPEG-2 demuxing with ProjectX, it's rock solid but I don't like the Java dependency.

DGMPGDec x64 supports only MPEG-2 and does not work on Win10 but there are chances somebody fixes it once Win10 gets more popular.

tools that loose av sync on some of my samples and when you do some cutting/trimming are ffmpeg, eac3to and mkvmerge. I've never reported it because for me it's unlikely it gets fixed because fixing bad TS hasn't great priority in this tools, I could be wrong of course.

dsmux works well but there are samples where it freezes for instance and soon hevc DVB arrives where I then have again 10 years no good solution. What else freeware can I bundle with StaxRip?

The source code for DGIndex is also available and is written in C-style. I'm sure there are many otheers, but if you tell us what you hope to accomplish we can direct you more specifically.

The tool is supposed to demux DVB containing possible errors and keep av sync in a way any tool can work well with the outputted streams afterwards.

Don't kid yourself that Rust is any easier to become proficient at than C/C++.

I hope it's much more fun, C++ has some issues I'm hardly able to tolerate like dealing with headers and decelerations, compile time, memory management and related bugs and probably tons of other things, many things I don't know much in practice but have read about it often so I can't make a good judgement, it has been a couple of years I used it last time, single most biggest pain point for me is headers and declarations.

Finally, stream parsing involves heavy bit-field manipulation and Rust does not bring much to the table in that regard.

I've little to no experience with this type of programming so I'm much interested to learn how the three languages compare for this type of application.

[videoh]

I guess C++ is often chosen for performance reason but if you look at this closer isn't ASM what you use if you want to make something really fast? As far as I know Rust is aiming to be high performance as well with zero-cost abstractions, unsafe code and ASM support. You can do ASM if you like, but it's unlikely to be very useful for a demuxer. And compiler intrinsics are well-developed for C/C++ compilers, while immature for Rust at this point. Anyway, we should compare apples to apples, i.e., C without ASM to Rust without ASM, or C with ASM to Rust with ASM. Once you put all the key code into ASM, the base language choice becomes much less important. However, ASM coding, cycle counting, caches, prefetching, and all the rest is difficult and tedious work, and of course portability between platforms is sacrificed. For a demuxer, simple file I/O and bit-manipulation is all you need. Nothing there is a candidate for ASM coding to improve performance.

There aren't many, I don't want StaxRip users having to install Java only for MPEG-2 demuxing with ProjectX, it's rock solid but I don't like the Java dependency. Again, there you go with your likes and feelings. :) You'll pull in the entire .NET infrastructure but including Java is too much for you?!

DGMPGDec x64 supports only MPEG-2 and does not work on Win10 but there are chances somebody fixes it once Win10 gets more popular. It's highly unlikely it is broken only on Win10, and errors have been reported on Win7/8/8.1. It's not my code so I can't help you to get it fixed. I've always asked people forking my code to replace the DG prefix with their own.

tools that loose av sync on some of my samples I would argue that it's your workflow that loses sync. You can't just mix and match tools in any old way. You have to know what you are doing, i.e., you have to understand what the tools are really doing. If you think that tools must be able to handle any arbitrary stream corruption, then I would argue that you are seeking the programming equivalent of perpetual motion; it isn't ever going to happen. A simple example: In an MPEG2 stream, a RFF flag gets flipped due to corruption. Your audio has just gone async. How do you know if that RFF is intentional or an error? You can resort to heuristics, but heuristics are not provably correct and can break uncorrupted streams. A pit of multiple, disparate but interconnected heuristics soon leads a programmer to despair and insanity. Prove me wrong, though, many will kneel at your feet. ;)

On the other hand, if you think you can improve stream corruption handling compared to existing tools, that would be a fruitful pursuit, and we'd all be excited to hear your ideas.

I've never reported it because for me it's unlikely it gets fixed because fixing bad TS hasn't great priority in this tools, I could be wrong of course. What is the harm in reporting your issues to the developers? They are likely more receptive than you think, especially if your reports affect such basic usability as you claim. For any given "bad stream" that you come up with, you have to be able to suggest how it might be properly handled in some kind of general way; nobody is going to be receptive to you just saying "here is this bad stream that your tool barfs on, fix it, and while you're at it fix any other possible bad streams". If stream corruption is your focus, then instead of trying to put handling in every tool, you should think of creating your own stream preprocessor, that takes in the bad streams and puts out "good streams". Then you'll have no problems with any tools falling short of your expectations in this regard. It would certainly be a challenging project, and you would learn a lot in trying to implement it. Personally, I think most people would prefer to focus on eliminating the root cause of the "bad streams".

dsmux works well but there are samples where it freezes for instance Can you share these samples with us?

The tool is supposed to demux DVB containing possible errors and keep av sync in a way any tool can work well with the outputted streams afterwards. That's unreasonable, and if you knew anything about transport streams and demuxing, you'd understand why. If you want to dispute that, then show us your code, or show us your algorithms (pseudo-code) that implement it.

I hope it's much more fun, C++ has some issues I'm hardly able to tolerate like dealing with headers Why are header files intolerable for you?

and decelerations Did you mean 'declarations'? Why are C/C++ declarations intolerable to you?

You don't *have to* use header files in your own code; you can get by without them. You might only need an include statement as required by your environment or external libraries you use. If you want control over how your data is stored, you definitely need some kind of data declaration, so it's strange to complain about that. These things are not difficult to master, and the best way to master them is to just start trying to create your application. We are here in the Development subforum to happily assist you.

compile time Compile time for C/C++ is very fast in Visual Studio. Do you have some data to back up this claim? I can completely re-compile DGDecNV in less than 20 seconds (including compilation of the CUDA .cu files and the MP4 and Vorbis libraries). A typical single file change is virtually instantaneous.

memory management OK, you have to free memory that you alloc. Are you aware that there is a performance overhead for automatic garbage collection? Or are you looking for some idiot-proof language so that you don't have to be careful?

and related bugs and probably tons of other things What bugs and other things are these? And why do you suppose C/C++ is so dominant in these application areas?

many things I don't know much in practice but have read about it often so I can't make a good judgement, it has been a couple of years I used it last time, single most biggest pain point for me is headers and declarations. Thank you for the frankness, it helps us to understand where you are coming from. Specifically, what is your problem with headers and declarations?

I've little to no experience with this type of programming so I'm much interested to learn how the three languages compare for this type of application. You've heard my opinion; let's see what others have to say.

I don't mean to be insulting, just realistic, so please don't take this wrong. You rebuked me for my lack of .NET knowledge; I accepted the rebuke gracefully. Here you display zero knowledge of the internals of transport streams and demuxing, and even of programming for high performance. Everything you want to automate and make idiot-proof comes with a high performance overhead. It seems you have just googled around and are not basing your opinions on real experience. This stuff is difficult, even for great coders. You're not going to solve that with some magic programming language.

[Groucho2004]

You can't have your cake and eat it too. On one hand you want to write a program that requires knowledge and a programming language operating at levels way beyond "GUIs" and toys like VB.NET but on the other hand you can't tolerate "headers and declarations". This is getting a bit ridiculous.

Edit:
And all that stuff videoh wrote. :)

[videoh]

toys like VB.NET At the risk of appearing to be inflammatory or off-topic, for which my personal history imparts a special sensitivity, and for general amusement and stimulation, I offer this opinion found on the internet byways:

"It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration."

I earnestly hope that the present instance manifests itself as a counter-example.

[Groucho2004]

"It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration."
Ouch.

[stax76]

It's highly unlikely it is broken only on Win10, and errors have been reported on Win7/8/8.1. It's not my code so I can't help you to get it fixed.

It has surprised me but when I ported StaxRip to x64 up to ten bugs did only happen on Win10, if I remember right then DGDecode x64 was working on Win7 but on Win10 the image is completely green.

Why are header files intolerable for you?

Why are C/C++ declarations intolerable to you?

New languages don't need it because compilers are much more advanced today compared to when C/C++ was created. This design with header including is also what causes the slow compile time.

I would argue that it's your workflow that loses sync. You can't just mix and match tools in any old way. You have to know what you are doing, i.e., you have to understand what the tools are really doing.

I wasn't really aware of that there are different ways and incompatibilities.

That seems to have a higher prior probability. What is the harm in reporting your issues?

I probably could do better, trying to get to the bottom of things can take much time.

Compile time for C/C++ is very fast in Visual Studio. Do you have some data to back up this claim?

I'm reading this very often and have experienced it myself, I've also read a few times why it is slow, mostly because of repetitive header inclusion if I remember right. For small and medium size applications it's probably OK.

Can you share these samples with us?

http://jell.yfish.us/media/Jellyfish-Raw-Original.ts

That's unreasonable, and if you knew anything about transport streams and demuxing, you'd understand why. If you want to dispute that, then show us your code, or show us your algorithms (pseudo-code) that implement it.

Why is it unreasonable? I know very little about the technical background but from what I know it don't seem impossible.

OK, you have to free memory that you alloc. Are you aware that there is a performance overhead for automatic garbage collection? Or are you looking for some idiot-proof language so that you don't have to be careful?

I'm know that a GC has disadvantages, it's one of the reasons I don't think the D language will be very successful. Rust memory management looks interesting however, it's not using a GC, I believe it's a lot compile time smarts with some ref counting, I don't know exact but it's efficient using zero-cost abstractions.

What bugs and other things are these? And why do you suppose C/C++ is so dominant in these application areas?

Bug with pointers, memory leaks or generally bugs can happen with manual memory management. Rust makes many of these bugs much harder or impossible. I think the reason why C/C++ very popular is performance, it's very efficient.

I don't mean to be insulting, just realistic, so please don't take this wrong. You rebuked me for my lack of .NET knowledge; I accepted the rebuke gracefully. Here you display zero knowledge of the internals of transport streams and demuxing, and even of programming for high performance.

I know I was too harsh but I didn't even know it was you! I think the first clues that it's you came later. Despite issued from the past you can be sure I would have treated you with more respect, I was very upset.

This stuff is difficult, even for great coders. You're not going to solve that with some magic programming language.

I'll see how difficult it is if and when I start but I believe you when you say it's very difficult.

You can't have your cake and eat it too. On one hand you want to write a program that requires knowledge and a programming language operating at levels way beyond "GUIs" and toys like VB.NET but on the other hand you can't tolerate "headers and declarations". This is getting a bit ridiculous.

I think you are a little bit ignorant here, you cannot imply VB.NET is inferior to C# or C++, the languages have a different age and legacy and different strengths, I clearly favor new languages because they have innovations and lessons learned from old languages but still I see nothing wrong in using old languages, there are advantages too.

edit:

Here you will find code for a demuxer called xport written in C-style. This will give you a good idea of what you are getting into. This could be a fine starting point for you to modify for your needs. Scroll down the page to find it.

Looks pretty promising, it handled one of my broken samples perfectly.

[Groucho2004]

New languages don't need it because compilers are much more advanced today compared to when C/C++ was created. This design with header including is also what causes the slow compile time.
So, you think that declaring variables and assigning them data types is a bad thing? That's exactly the attitude why Basic has such a bad reputation.
As for headers and compile time - look up precompiled headers.

Bug with pointers, memory leaks or generally bugs can happen with manual memory management.
Same as above - discipline is a good thing in programming. Moving all responsibilities to the language/compiler makes bad programmers.

[videoh]

It has surprised me but when I ported StaxRip to x64 up to ten bugs did only happen on Win10, if I remember right then DGDecode x64 was working on Win7 but on Win10 the image is completely green. Win10 is still in development and not yet formally released, so it seems premature to draw any such conclusions. It's possible, however, that Win10 will break a lot of existing applications, but it seems doubtful to me. Anyway, that's all not important for the matters we are discussing.

New languages don't need it because compilers are much more advanced today compared to when C/C++ was created. This design with header including is also what causes the slow compile time. Umm, how do you tell the computer that you want to store this integer in 8 bits? Modern computers can read your mind? Headers make compilations slow? Are you serious? Have you ever compiled a C program? And have you ever heard of precompiled headers (Groucho2004 pointed this out first above), which are generally enabled by default?

I wasn't really aware of that there are different ways and incompatibilities. That's what I meant when I said you have to know what you are doing. :)

I probably could do better, trying to get to the bottom of things can take much time. For sure.

I'm reading this very often and have experienced it myself, I've also read a few times why it is slow, mostly because of repetitive header inclusion if I remember right. For small and medium size applications it's probably OK. See above.

http://jell.yfish.us/media/Jellyfish-Raw-Original.ts Thanks, we'll have a look.

Why is it unreasonable? I know very little about the technical background but from what I know it don't seem impossible. Well, there are just too many complications, or somebody would have done it by now. Suppose you decide "hey, all I have to do is look at PTS timestamps and cut this and that here and there and rewrite the timestamps". But it's not so easy, it's not deterministic, you have to make choices. Do you delete orphaned frames or not, and so on. Either choice could be correct in different use cases. And you can outright lose important things like SPS/PPS. Then you can guess that maybe the previous SPS/PPS can be used, but how do you know? That determines whether you have to discard some big chunk of the stream. And what if the input stream is an ES, or some other container format? And if you start deleting stuff, you break T-STD and VBV models and you can't know if some downstream buffer is going to blow a gasket. Maybe you'll get lucky, but maybe you won't. These complications go on and on and on. As I said, though, if you think you can solve all these things, we'd all be excited to hear about it. Maybe you can develop some limited requirements, implementation of which would be satisfactory for your use case.

I'm know that a GC has disadvantages, it's one of the reasons I don't think the D language will be very successful. Rust memory management looks interesting however, it's not using a GC, I believe it's a lot compile time smarts with some ref counting, I don't know exact but it's efficient using zero-cost abstractions. Perhaps, but a simple demuxer can easily be written without memory allocation, and in C++ you have your constructors and destructors. Believe me, it is not a big deal.

Bug with pointers, memory leaks or generally bugs can happen with manual memory management. Rust makes many of these bugs much harder or impossible. I think the reason why C/C++ very popular is performance, it's very efficient. You mean you think it's easier to make mistakes in C/C++ than Rust? There's no evidence for that but if you are prone to making mistakes, you can surely choose such a language. After Einstein, I would call it a gentle pillow from which you cannot easily be roused, and he would counsel letting you lie there. ;) I don't know if or how Rust handles pointer types, but if they are absent that would be a serious disadvantage for the kind of coding we are talking about.

I think you are a little bit ignorant here, you cannot imply VB.NET is inferior to C# or C++, the languages have a different age and legacy and different strengths, I clearly favor new languages because they have innovations and lessons learned from old languages but still I see nothing wrong in using old languages, there are advantages too. Well, I know that VB.NET and C# both are just different syntaxes for the underlying CLR, so I wouldn't say that VB.NET is necessarily inferior to C#, although I personally would prefer C-like syntax. I would say, however, that both VB.NET and C# are inferior to C for the use case we are talking about, for the reasons previously adduced. I don't see how thinking that makes me ignorant.

Looks pretty promising, it handled one of my broken samples perfectly. Good to hear.

My advice to you if you are serious about this stuff is just go ahead and start trying to do a simple task with Rust. You could, for example, try to make an application that simply counts the video frames in a transport stream. You'll soon find out if the language is getting in your way or not. Philosophical agonizing over languages will only block you from doing anything.

[stax76]

So, you think that declaring variables and assigning them data types is a bad thing?

I can't follow you here.

That's exactly the attitude why Basic has such a bad reputation.

Reputation is one thing, VB.NET isn't really much different then C#, the feature set is almost equal, the Syntax different.

As for headers and compile time - look up precompiled headers.

I have basic C++ knowledge so I know precompiled headers, few examples of what I wrote is shell extension extending the context menu, a DVB application with all basic features like timer, decrypting pay TV (multidec), ts recording, the API was C++ and I made a wrapper with C++/CLR, the GUI was C#. I made a C++ plugin for DVBViewer for my remote control and a few other things, everything mostly small things with a C# GUI on top, when .NET was new I used C# often, I use C++ only when I don't have great choice like there is only a C++ API or .NET wouldn't work well.

Same as above - discipline is a good thing in programming. Moving all responsibilities to the language/compiler makes bad programmers.

I don't know Rust in detail yet but it looks it could be the strictest language so far, only time can tell if the design works.

[stax76]

Umm, how do you tell the computer that you want to store this integer in 8 bits? Modern computers can read your mind? Headers make compilations slow? Are you serious? Have you ever compiled a C program? And have you ever heard of precompiled headers, which are generally enabled by default?

That's how I remember it, most recently I watched a video from a Apple engineer on a draft for modules support for C++, it mentioned this problematic.

I don't know if or how Rust handles pointer types, but if they are absent that would be a serious disadvantage for the kind of coding we are talking about.

It has pointers but there are very strict rules to make them secure.

Well, I know that VB.NET and C# both are just different syntaxes for the underlying CLR, so I wouldn't say that VB.NET is necessarily inferior to C#, although I personally would prefer C-like synatx. I would say, however, that both VB.NET and C# are inferior to C for the use case we are talking about, for the reasons previously adduced. I don't see how thinking that makes me ignorant.

It was a reply to Groucho2004. :)

[videoh]

It was a reply to Groucho2004. :) Groucho2004 is definitely not ignorant. :D

[videoh]

dsmux fails on the sample for me too, and the stream does not appear to be corrupted or otherwise bad. This problem with dsmux has been reported some time ago. If you just want to repackage as MKV, you can extract the ES using xport and then mux to MKV with mkvmerge.exe. With the source code in hand for xport, you'll have good prospects of fixing things if you run into any issues. Some practical problems you will have to solve for using xport are how to determine the correct last three parameters to use (program, video, and audio numbers), how to deal with multiple audio tracks, etc.

The sample stream does not have audio so it's hard to test sync. And how much noise does a jellyfish make? ;)

[stax76]

Do you remember where it was reported?

[videoh]

Here's one:

http://forum.doom9.org/showthread.php?t=140164

[raffriff42]

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.Let's retire this old quote, shall we?
Why did Dijkstra Hate Basic (http://programmingisterrible.com/post/40132515169/dijkstra-basic)? (programmingisterrible.com)The thing people forget is that programming was substantially different in 1975. Dijkstra railed against Dartmouth Basic—a glorified assembler language. It isn’t the BASIC used today–
Variables names were one letter long with an optional digit
Although FOR was present - while loops, break and other niceties were implemented with unrestricted goto.
Whitespace was optional between expressions.
Subroutines were available. GOSUB line number and RETURN.
All parameter passing had to be done with the existing global variables.
26 user defined functions could exist FNA-FNZ, which could contain one line of code.
IF statements were also limited to one line of code.
I could go on about the limitations, but the consequences are pretty obvious: It was hard to write readable and understandable code in. The features we take for granted now just weren’t present. It wasn’t until after the article that BASIC began to grow up, into the language people know today.Just wanted to point out that VB.NET does not have to be terrible, if the programmer uses good practices, such as variable declaration(!)

EDIT Off topic, the rest of Dijkstra's uncomfortable truths (http://www.cs.virginia.edu/~evans/cs655/readings/ewd498.html)are an interesting read if you haven't seen them before - I sure hadn't.

[foxyshadis]

Winding all the way back to the third post, ProjectX is, as you say, rock solid. It's only that way because of years and years of bug-fixing and bad-stream handling. Even with specs in hand there's no way you'll be able to handle a large number of in-the-wild bad streams, without lots of workarounds over a similar length of time.

So I'd suggest rewriting ProjectX in C# or something else you're familiar with. Maybe start with automatic translation like Sharpen (https://github.com/mono/ngit/tree/master/Sharpen). If you want to write it yourself, of course, no one's going to stop you and the best choice is probably always going to be something you're intimately familiar with. It's 2015, speed of any format parser is going to be limited by disk io, not your language choice, so speed should not be a consideration, unlike encoding and decoding. Even Python or Ruby would be fine, I bet.

[raffriff42]

Even with specs in hand there's no way you'll be able to handle a large number of in-the-wild bad streams, without lots of workarounds over a similar length of time. So I'd suggest rewriting ProjectX in C# or something else you're familiar with. I once did a port of VirtualDub's core AVI-parsing code to VB.NET for that very reason (sourceforge) (https://sourceforge.net/p/frafsaviinfo/svn/HEAD/tree/trunk/VDub/CAVIInfo_2.vb).
The final product works great, but creating workarounds for certain untranslatable C++ language idioms was a lot of work.
EDIT point being, "yes" to re-using proven code, "maybe" to translating into your favored language, or maybe retain the original.

[captainadamo]

Winding all the way back to the third post, ProjectX is, as you say, rock solid. It's only that way because of years and years of bug-fixing and bad-stream handling. Even with specs in hand there's no way you'll be able to handle a large number of in-the-wild bad streams, without lots of workarounds over a similar length of time.

Yeah, without a very good reason for doing so a from-scratch rewrite sounds like a really, really bad idea. Especially for what is a somewhat silly reason (no Java dependency). Even well-funded companies with lots of staff have found such things to be brutally hard to do without creating a whole swath of new bugs or without making their new code "ugly" as well once they've had to incorporate all the dirty hacks the old code did to actually work in the real-world not the ideal world of the standards document. And then even after all the effort just to get back to exactly the functionality of what ProjectX already has, one has to weigh if all that effort was really worth it for the marginal benefit.

stax you can obviously do what you like, but I don't think you really understand the momentous task it will be to reimplement the man-years of effort that goes into writing programs like ProjectX.