Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th August 2009, 15:11   #1  |  Link
Chengbin
Registered User
 
Join Date: Oct 2007
Posts: 1,060
Size of programs (like encoders)

I wonder why, for example x264, is only 1MB (or 230KB compressed) when it is an incredibly complicated program that has enough code to need almost 1200 revisions to get to where it is now. Look at the majority of softwares out there. It is quite hard to find a 1MB program, and if you do find one, it is extremely simple. So why is x264 only 230KB?
Chengbin is offline   Reply With Quote
Old 4th August 2009, 15:18   #2  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Because x264 is highly optimized, but not only for speed, also for minimizing code size. Also my x264.exe is more in the ~650 KB range, your 230 KB file may be packed (UPX, kkrunchy, etc).

Furthermore keep in mind that all those revision don't add loads of code. Many replace existing code or even kill old/redundant code.

I think a famous software developer (Edit: Kenneth Thompson, one of the C and Unix pioneers) once said that his most productive day was the day when he deleted 1000 lines of code

Last but not least x264 is a CLI tool. All those GUI applications are much bigger because they contain tons of graphics and/or are linked against bloated GUI libraries...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 4th August 2009 at 15:33.
LoRd_MuldeR is offline   Reply With Quote
Old 4th August 2009, 16:07   #3  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
One reason is there is no need to use huge frameworks like wxWindows, QT, MFC etc. Classes in these libraries have many methods which in return use other classes and all this code gets pulled into the product. EventGhost done with Python and wxWindows is a notorious resource hog for instance. It would have been better to use .NET, ATL/WTL or a combination of both. It's not that there isn't a Python implementation for .NET, in fact there are two. The advantage of .NET is it's part of the OS (3.0 in Vista) or a seperate downoad for the newest version that only have to be downloaded and installed once instead every application installs a runtime/library or statically links to it. The upcoming 4.0 version of .NET will be available as client profile which is a stripped down version, I think it will be at 30 MB which is a lot smaller than the full version.

Furthermore applications typically include resources like images, sound etc. General bloat is also common, useless features, legacy features, code duplication, doing a thing with thousand lines of code that could a good programmer might as well do with a ten lines e.g. by leveraging operating system features, modern systems like Win 7 have much to offer in this regard. A lot programmers simply don't care about the size of their source code and binaries and many of them don't care about the performance of their application or don't want to take the time to optimize or learn how to optimize. They generally don't seem to be concerned about efficiency and efficiency often isn't the only problem such applications suffer. Projects with a lot contributors suffer usually more from such issues. A very sad example for this paragraph is MediaPortal.

Personally I prefer lightweight applications like uTorrent, note that the u stand for nano (I'm too lacy to pick the right character) implying the author is concerned about efficiency. Some might like MPC for being lightweight though I don't buy that it is, it's just the GUI is old and limited (still by far the best player around). I'm also working on a couple of aplications intended to be lightweight but there is still a lot room for improvement in this regard and it's certainly something I'm much concerned about.

Last edited by stax76; 4th August 2009 at 16:34.
stax76 is offline   Reply With Quote
Old 4th August 2009, 17:08   #4  |  Link
nurbs
Registered User
 
Join Date: Dec 2005
Posts: 1,460
Quote:
Originally Posted by stax76 View Post
Personally I prefer lightweight applications like uTorrent, note that the u stand for nano (I'm too lacy to pick the right character) implying the author is concerned about efficiency.
µ generally stands micro. n stands for nano.

By the way µ is AltGr+m on german keyboards.

Last edited by nurbs; 4th August 2009 at 17:11.
nurbs is offline   Reply With Quote
Old 4th August 2009, 17:20   #5  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,480
@ stax: µ = ALT + 0181

Hopefully Windows 8 will NOT run on top of .NET framework
Midzuki is offline   Reply With Quote
Old 4th August 2009, 17:53   #6  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
WinAPI don't has changed since Win 95 in Win 7 nor will it change in Win 8, it is C and COM+ and a major pain to work with. It's because it's fast, small and language agnostic. There is now also .NET and more and more Microsoft applications take advantage of it, for instance in Visual Studio they have replaced major parts of the UI with shiny WPF like the command/tool bar, even the code editor was replaced with WPF (next version) for a colorful and animated presentation. Some story with with Office, I guess the announced online office will be build with .NET, one of the main reasons why .NET and Silverlight was developed after all, I don't think JavaScript cut it though it's also evolving. I'm curious about how google's announced OS will be implemented. The issue with .NET is startup time and memory consumption for huge applications like a IDE or media center especially when heavy API's like WPF are used and WPF is where everything is heading at. Because of this core applications and services like the shell and extensions to it will stay native in particularly to avoid the CLR being loaded into many native processes by shell extensions unnecessarily. Besides that there is a versioning issue since different extensions might need a different CLR version but a process can only host a single CLR instance.
stax76 is offline   Reply With Quote
Old 4th August 2009, 18:25   #7  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by stax76
WinAPI don't has changed since Win 95 in Win 7 nor will it change in Win 8
Why not put a Win32 API compatibility layer on top of .NET ???

But my guess is that Windows 8 will be a pure "online" operating system running on a deciated server. All that's left running on your local machine will be Chrome OS

Quote:
Originally Posted by stax76
I'm curious about how google's announced OS will be implemented
I think the info that is available yet says that Chrome OS will be based on a Linux Kernel, but with a new Window Manager.

Also Chrome OS will serve mainly (solely?) as a platform to run the Chrome browser. And the Chrome browser severs as an access point to Google's web-applications.

Whether it will be possible to install "local" applications under Chrome OS is still speculation...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 4th August 2009 at 18:30.
LoRd_MuldeR is offline   Reply With Quote
Old 4th August 2009, 18:37   #8  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Quote:
Originally Posted by Chengbin View Post
I wonder why, for example x264, is only 1MB (or 230KB compressed) when it is an incredibly complicated program that has enough code to need almost 1200 revisions to get to where it is now. Look at the majority of softwares out there. It is quite hard to find a 1MB program, and if you do find one, it is extremely simple. So why is x264 only 230KB?
x264 is actually only about 650 kilobytes (less with -Os); the rest is from GPAC and similar. Linking external libraries is a great way to make executables huge. It's probably even less if you leave out pthreads.

650KB is about the size of one Lord of the Rings novel. That's a staggering amount of information. The real question is not why x264 is so small, but why everything else is so bloated.

Also, with kkrunchya2, x264 compresses to about 195KB. With PAQ8P, though not an executable compressor, x264 compresses to about 175KB.
Dark Shikari is offline   Reply With Quote
Old 4th August 2009, 22:32   #9  |  Link
vlada
Registered User
 
Join Date: Apr 2003
Location: Czech Republic
Posts: 752
LoRd_MuldeR> Just an OT - I think ChromeOS will be a modification of Chrome (browser). All applications will be Google's online services written in AJAX.
vlada is offline   Reply With Quote
Old 5th August 2009, 11:36   #10  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by vlada View Post
LoRd_MuldeR> Just an OT - I think ChromeOS will be a modification of Chrome (browser). All applications will be Google's online services written in AJAX.
No, it won't. Chrome OS will be a "real" OS, not just an application. They already announced that Chrome OS will be based on a Linux kernel, but with their own window manager.

Whether the Chrome browser will be the only "local" application running under Chrome OS (an all applications are on the web) or not is still speculation...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th August 2009 at 12:35.
LoRd_MuldeR is offline   Reply With Quote
Old 6th August 2009, 17:13   #11  |  Link
Dr.Khron
Registered User
 
Dr.Khron's Avatar
 
Join Date: Oct 2006
Location: Gotham City, USA
Posts: 389
Quote:
Originally Posted by Dark Shikari View Post
650KB is about the size of one Lord of the Rings novel. That's a staggering amount of information. The real question is not why x264 is so small, but why everything else is so bloated.
This is an important point, size is relative. 650KB of text data, ie novels or programming code, is ENORMOUS, whereas 650KB of audio data is pretty small, and 650KB of video data is almost non-existant.

That said, the first awnser off the top of my head was the CLI thing. GUIs do not come cheap.
Dr.Khron is offline   Reply With Quote
Old 7th August 2009, 10:59   #12  |  Link
Doobie
Registered User
 
Join Date: Oct 2001
Posts: 555
Quote:
Originally Posted by Chengbin View Post
It is quite hard to find a 1MB program, and if you do find one, it is extremely simple.
Most of the size of programs come from the interface and bloat. Anything you want to do, you can do in very little space. Consider, the Atari 2600 game system only has 128 bytes of RAM and address space for mass storage of only 4K. 1MB is HUGE. You could never, individually, write a tight executable of that size in your entire life.
Doobie is offline   Reply With Quote
Old 10th August 2009, 18:06   #13  |  Link
von_Runkel
Registered User
 
von_Runkel's Avatar
 
Join Date: Feb 2008
Location: Sweden
Posts: 53
Quote:
Originally Posted by Dark Shikari View Post
The real question is not why x264 is so small, but why everything else is so bloated.
This question is the most important, IMHO. I guess you can fit the complete x264 source, editor, compilers, linkers etc. on a pair of floppies?

Recently, I threw out the CS4 bloatware from my computer at home and at work. I am happier than ever.
von_Runkel is offline   Reply With Quote
Old 10th August 2009, 19:08   #14  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
Quote:
Originally Posted by von_Runkel View Post
Recently, I threw out the CS4 bloatware from my computer at home and at work. I am happier than ever.
i'am still fighting with cs3, whats the alternative?
__________________
certain other member
smok3 is offline   Reply With Quote
Old 10th August 2009, 19:31   #15  |  Link
von_Runkel
Registered User
 
von_Runkel's Avatar
 
Join Date: Feb 2008
Location: Sweden
Posts: 53
Quote:
Originally Posted by smok3 View Post
i'am still fighting with cs3, whats the alternative?
Depends on what parts of the suite you really need. Myself, I needed Premiere and installed an older (lighter) version instead. Same with Photoshop, took out my ancient PS7 disc from the bookshelf and found that it was faster and could do the things that I need. Encore: You have the tools here at Doom9.

Of course, capturing from new tape devices and converting HDRI image sequences can be a challenge w/o the newest from Adobe. But the ffmpeg dev team seem determined to include more and more formats and codecs. For example, from my last visit at the dev list, I could see that an EXR decoder was under construction.
von_Runkel is offline   Reply With Quote
Old 10th August 2009, 20:05   #16  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,815
Quote:
i'am still fighting with cs3, whats the alternative?
I've been using Corel Photo-Paint since version 6.

Here is my simple example what you can do with latest version 14

http://img193.imageshack.us/img193/3...rbuilding2.png

I achieved Comic effect by just using 3 layers.

Last edited by Atak_Snajpera; 10th August 2009 at 20:07.
Atak_Snajpera is offline   Reply With Quote
Old 10th August 2009, 22:18   #17  |  Link
iffybob
Registered User
 
Join Date: Jul 2009
Location: England , South Yorkshire
Posts: 99
I've had the impression for some time now that many things to do with PC software and the Win OS, just need scrapping, and start again.

The whole thing is looking like a proto-type development circuit board ( but for software ), all tangled wires every where, with bits bolted on every where.

At some point you have to right the whole thing off, and start again. One point is you can flattern the security.

The problem is we all like "ower" software, we are familair with it, so we like the backward compatabilty "and" the new functionality.

You have to upgrade the OS to use the new software and hardware, but you do want to use your old programes.

the other problem is the monumental task that it is,
this also brings up the point of "The sooner it starts, the sooner its done" ..... Please
iffybob is offline   Reply With Quote
Old 10th August 2009, 22:37   #18  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
Quote:
Originally Posted by von_Runkel View Post
Depends on what parts of the suite you really need. Myself, I needed Premiere and installed an older (lighter) version instead. Same with Photoshop, took out my ancient PS7 disc from the bookshelf and found that it was faster and could do the things that I need. Encore: You have the tools here at Doom9.
basically it goes like this (production premium cs3 - i don't think you can install parts on multiple machines..?):

01 after effects (becoming unstable the more you run it as it seems, here would mac actually count, because i had no problems with it in the past)
02 premiere mit cineform (lately getting crashes with font tool, probably have to wipe out any non-standard system fonts)
03 photoshop (i could probably almost completely replace that with only using after fx)
04 flash (sketching with wacom)
05 illustrator
06 lightroom sometimes
07 encore rarely

so i'am problematic :/

premiere is partially replaced with sony vegas 9 pro, others don't have good alternatives imho, cs4 is certainly not something i'am looking forward.

the partition is huge, making snapshots is not an option.

p.s. actually if there was good docs on whats happeing with gamma on quicktime and if final cut pro had a decent audio mixing workflow i would probably bite the ugly fruit.
__________________
certain other member

Last edited by smok3; 10th August 2009 at 22:45.
smok3 is offline   Reply With Quote
Old 10th August 2009, 23:20   #19  |  Link
von_Runkel
Registered User
 
von_Runkel's Avatar
 
Join Date: Feb 2008
Location: Sweden
Posts: 53
Quote:
Originally Posted by smok3 View Post
basically it goes like this (production premium cs3 - i don't think you can install parts on multiple machines..?):
Depends on you license. With a "multi-license" (or whatever they call it at Adobe), you can install different parts of the suite on different machines, yes.

Cineform: if the main purpose is to make proxies of RED footage, then try ffmpeg instead (but I guess you already know this).

Otherwise, Flash & Illustrator are hard to replace. Same with AE. I can live without AE but sometimes I miss Flash for debugging problematic VP6 videos with alpha channel.

IMHO, the suite runs better on a Mac. Haven't heard any complaints from my Mac users. The opposite goes for my Windows users - sluggish, bloatware, even spyware!

I don't think the path to FCP is the way to go. You will get frustrated and think that the gamma problem is the least problematic one. Quicktime is far from mature and can't survive a regression test with uncompressed sources.
von_Runkel is offline   Reply With Quote
Old 10th August 2009, 23:59   #20  |  Link
smok3
brontosaurusrex
 
smok3's Avatar
 
Join Date: Oct 2001
Posts: 2,392
at least it would be a different kind of frustration, but yes i know :P
__________________
certain other member
smok3 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:21.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.