What's real programming like?

[Shadow]

Okay, this is a little difficult to express, but I'll do my best. I wanted to ask, what's real programming like? As in, how do they do that? I've always wondered how they, I don't know, make Half-Life 2, or Adobe Photoshop...both of those are programmed in C++, yet with what I've learned so far, I just have no clue how they did it. I feel like there's some major something I'm missing. Also, how do they start? Do they just create a new project in their favorite IDE and start coding? Do they even use IDE's for that matter?

 

I've also been fascinated with game mods, specifically those that significantly alter the gameplay. It's obvious that they had to alter part of the game code somehow. How do they do that?

 

I just have Step >X< as learning C++ the way they teach it on the internet, and then suddenly step >Y<, which is stuff like I described above...and I have no clue what's in between.

 

This is lamely written since I'm writing it in a hurry, but I hope it conveys the gist of what I'm trying to ask. Thanks in advance for any light shed on the matter.

[JillSwift]

The same concepts that apply to smaller programs apply to "real" programs.

 

The larger projects are often broken down into modules - for instance, Photoshop might be broken down into image management, UI management, filter management, settings management, etc.

 

And those modules are often broken up into smaller modules, like image management has routines to handle what's loaded in memory, what's cached, etc.

 

Like any large project in any medium, really. You break it down into manageable parts. It's just a matter of getting to know ways to go about the specific tasks you need to get done.

 

IDEs may or may not be used, depending on thier usefulness to the process. I have a vague memory about in-house IDEs built specifically to aid development of Doom 3 engine games, though.

 

HTH

[Cap'n Refsmmat]

If it helps, here's a link to where you can browse through the source code to Mozilla Firefox online and see how it all fits together. You can see how things are split up into pieces and how generally complicated it all is.

 

http://hg.mozilla.org/mozilla-central/file/50be346999f6

[Shadow]

I knew about the splitting up part...after all, that's basic problem solving. But I just can't imagine how that's the only difference between them and me. But thanks for the link Cap'n.

 

Also, once they're finished, they just hit "Compile" and than doubleclick on run.exe? And what are all those files around? Most of my executables are just one .exe file and that's it. I realize you have to have sound files, and the high-tech 3D equivalent of sprites and stuff, but it still doesn't account for all the files I don't think...does it?

 

And one last question, when looking at an amount of code as large as for example the Firefox code...where do you start? Where's the friggin' main function?

[insane_alien]

well, the main executeable is usually pretty small. maybe a few megabytes. then there are all the .dlls and other modules that don't need to be active all the time etc.

[D H]

The same concepts that apply to smaller programs apply to "real" programs.

I have to disagree, Jill. "Real" programs add a several dimensions that either do not exist or are trivially small in small programs.

 

Requirements: For a small program, there might be but one requirement that can be written in the form of a sentence or two. The hacker's mantra, "we don't need no stinkin' requirements" has sunk many large programs. Keeping track of what is required of a project and ensuring that the project truly satisfies all things required of it takes on ever more importance as the project grows in size.

 

Design: The same goes for design. In a large project, requirements often map to multiple design components, and a design component can (partially) satisfy multiple requirements. What is a simple one-to-one relationship in a small project can easily become a many-to-many nightmare if the requirements and design are not well thought out.

 

Testing: Cap'n talked about Firefox. Something like Firefox needs a specialized team of testers, people who are good at poking holes. Get something really wrong in a small program and you have at worst made the user push the computer's restart button. Get something really wrong in something like Firefox and the user has just leaked state secrets to their enemy. Testers are of a different mindset than developers. In my experience, really good testers are often incredibly lousy developers, and vice-versa.

 

Documentation: Someone needs to say how the big monster program works, and that someone is definitely not the developer.

 

Architecture: Someone needs to see the big picture and ensure that the project moves forward. A small program does not require a system architect. A big project dies without one.

 

Management: Anarchy works great for a team of one. It works fairly good for a team of two or three, OK for four or five. Beyond that, someone needs to be in charge. In real projects, some people find other jobs, some people retire, some people get promoted. How do you keep the team as a whole moving along and properly staffed?

[JillSwift]

I have to disagree, Jill.<snipped>

Well, I was referring to the program itself, not the organization of the project. Lighting up a pixel is the same in the five-line program "Light the pixel" as it is in the 2,000,000 line program "Photoshop".

 

@Shadow: I'm fairly certain the one thing that's different between where you are now as a programmer and the big projects you're looking at is simply knowledge of how to do the given sub-tasks required to meet the overall goal.

 

For behemoth projects like Photoshop, the programming teams - and even individual programmers - specialize in certain programming tasks, like the UI, image manipulation, color separation, etc. So, even the folks working on such projects don't always know all aspects of how to produce the end result.

 

Learning programming after getting the basic concepts down is a matter of pursuing your interests and needs.

[Shadow]

Thanks guys, that helps a lot. It still seems like a mammoth step, but since all of you agreed that there's basically nothing in between, I think it's my perspective that needs adjusting.

[D H]

Did you read my post? There is a huge amount of stuff in between. How to write real programs, as opposed to toy ones, is essentially what software engineering is all about.

[JillSwift]

Nothing in between?

 

I could swear I said there was all manner of things in between. The basic programming fundamentals are the same, but methods and techniques to achieving goals are many and each needs to be learned first.

 

And DH has pointed out that the environment for achieving the overall goals is also dramatically different. The actual production and engineering of large projects of any sort has always been more than a single human brain can properly organize, so we have systems to bring many brains to bear on a problem.

 

Lots of stuff in between.

[Shadow]

Poor choice of words on my side. No fundamental knowledge base in between. Nothing that you can learn and go "Oh I see" and make your own Photoshop. From a purely coding point of view, ie. what Jill was addressing at the beginning, it's a huge step, but there is nothing fundamentally new; just new techniques, new methods, new ideas, new implementations (and of course all the organization and such), but as you said DH, most of that in principle applies on smaller scales, except that it's so trivial that we don't even notice it. I did read your post, and it did help bring things in perspective. But the main question of this post was if I was missing some fundamental part of programming, something that wasn't mentioned in any of the tutorials I've read, and the answer you all gave me was that there isn't; it's not some big something doing all the work, it's lots of little somethings working together. And that's what's in between.

[JillSwift]

Poor choice of words on my side. No fundamental knowledge base in between. Nothing that you can learn and go "Oh I see" and make your own Photoshop. From a purely coding point of view, ie. what Jill was addressing at the beginning, it's a huge step, but there is nothing fundamentally new; just new techniques, new methods, new ideas, new implementations (and of course all the organization and such), but as you said DH, most of that in principle applies on smaller scales, except that it's so trivial that we don't even notice it. I did read your post, and it did help bring things in perspective. But the main question of this post was if I was missing some fundamental part of programming, something that wasn't mentioned in any of the tutorials I've read, and the answer you all gave me was that there isn't; it's not some big something doing all the work, it's lots of little somethings working together. And that's what's in between.

Spot-on! ==

[bascule]

Shadow:

 

Perhaps the thing you're missing the most is how different developing software in teams works versus developing software as an individual. It's not like one person wrote Half-life or Photoshop. Half-life got its start off one of the Quake engines, which was architected by John Carmack but included the contributions of many other people. Subsequent engines (e.g. the Source engine) were also collaboratively developed by multiple people.

[Pangloss]

I think I read somewhere that Grand Theft Auto IV had a team of 1,000 programmers who worked for 5 years on that project. Which sounds like a lot until you hear that it's made something like two billion dollars in sales.

 

I teach programming in a Game Design program at a small local university, and one of the challenges is the fact that it's very hard to squeeze in the building of a complete game inside of a 10-week quarter. We've tackled that a number of ways, including team work, special-topics "project" courses and extracurricular game-building activities. Our game design lab features high-def equipment and 64-bit multiprocessor workstations networked to an XBox 360, and we use C# and XNA studio because it allows us to use rapid application development methods, which allow for the illustration of all parts of the game design pipeline. RAD also allows us to build complete (if simple) games in a single quarter.

 

One of the challenges of this approach is that early on it lulls the programming students into a false sense of what programming is about (all those wizards and shortcuts). We have to correct that later when they take the "serious" programming courses. On the other hand, it draws more students into the program who might otherwise have seen programming as something they couldn't do ("look how easy it is to program!"). This is a good thing, because this country needs not only more programmers, but more network administrators, consultants, etc, with a solid understanding of what programming is about. Since presumably some of my students will be forced to find work in the larger IT industry instead of the gaming industry, this works out well.

 

All of that having been said, the programmers we crank out aren't equipped to write the kind of hard-core coding that drives the most innovative games; the major $60 console releases. They do get a solid core of OOP principles, and they can knock out indie games for the XBox 360 or iPhone. But they need more programming experience and more academic knowledge in order to really get to the level of a typical graduate from a traditional "computer science" program.

 

But that's how the industry has played out in recent years. There's "programming", and then there's programming.

[D H]

I think I read somewhere that Grand Theft Auto IV had a team of 1,000 programmers who worked for 5 years on that project. Which sounds like a lot until you hear that it's made something like two billion dollars in sales.

Oh please, enough with the hyperbole, Pangloss! Five years? It was only 3 1/2.

 

So, 1000 people for 3.5 years, and "over 800,000 lines of code." (Wikipedia reports 900,000 lines of code.) Take the average, more-or-less: Call it 840,000 lines of code in Grand Theft Auto IV.

 

What does that equate to? Let's say that with holidays, sick days, and vacation time an average person works 240 days a year (52 weeks less 4 weeks times 5 days per week). Doing the math,

 

[math]\frac {840,000\,\text{lines of code}}{1000\, \text{people}*3.5\,\text{years}*240\,\text{working days/year}}[/math]

 

Drum roll please:

 

One line of code per person per day.

 

That is typical of big projects, and that, in a nutshell, exemplifies the difference between big projects and small ones.

Edited September 15, 2009 by D H

[ydoaPs]

print('Hello World')

THAT is real programming!

 

 

/me runs away and hides.

[JillSwift]

print('Hello World')

THAT is real programming!

 

 

/me runs away and hides.

 

#!/bin/bash
var='Hello World!'
echo 'Hello world! I have an unreferenced variable!'
exit 0

 

Now, that is real programming