Why is engineering software so hard/different than other engineering pursuits ?
I’ve been pondering this topic a lot lately, and have had several discussions with
colleagues and friends on what steps one might take to improve matters when working
on a project which has issues associated with this topic.
This Dr. Dobbs article brought the topic to
the forefront of my mind and has recently resulted in the following thoughts on the subject.
1) The solution to any given software problem has far more possible solutions than in
other fields of engineering, and tends to be on the bleeding edge of technology. Other engineering fields
don’t tend to put developments out of their R&D labs into customer usable/facing products
nearly as rapidly as in the software industry. Effectively, customers are most commonly using the bleeding
edge instead of a product from tried and true methods, processes and technologies. This
is a fairly sizable contributing factor in the complexity of the creation of software. The fact
that developers are learning new technologies and techniques all the time and applying them
each day is a huge testament to the traditionally less than strict process involved in our industry.
2) The group members in any engineering team are of different quality by any metrics you might measure them by.
How that impacts a project and subsequently the whole team, is where things diverge when comparing software development
to other engineering disciplines. Only in the software industry can one person have as large an impact on the whole ecosystem.
If you’re building a bridge, and one contributor isn’t keeping up, you send them packing and find another in short order. In the
software realm, you could send the person packing, but usually don’t, and whether you do or not there’s a much larger impact.
Replacing someone usually doesn’t happen when it needs to, takes far too long, and is far too difficult to do successfully.
This needs to be dealt with case by case, as every company and environment will be able to address it different ways.
The goals should be to a) have processes in place that make it trivial for people to switch projects on their own, including
capturing their knowledge on current projects. b) allow the team to decide how best to divide the work it’s been given c) force
the team to take responsibility for itself and its project. d) implement current standard practices such as bug tracking and
continuous integration. You might be surprised how regularly such things are either not in place, or not used consistently…
They can help bridge the skill gap to a certain degree by allowing the team to understand which of their colleagues is capable
of what, and can allow them to compensate internally without any intervention. In the software industry
the least capable person has a far greater ability to affect the project than in other industries. Addressing this single
point would go a long long way toward improving the software engineering domain. Sadly, I have no idea what
to suggest, and every attempt I’ve seen to do so has been a failure or marginal success.
3) A view point offered by my manager, Eric Kihn,
is that there’s such a large uncertainty associated with estimating both new features and bug fix efforts, that it’s difficult to plan a software
project to any reasonable degree of certainty when compared to other engineering efforts. Again, if you’re building a bridge it’s far easier
to know if you’re on schedule, and if not what your trouble areas are and how to address them. Too commonly a software developer
will jump into a feature or bug fix and find out that what they thought would take a week will actually be more on the order of months.
Experienced software engineers are obviously better at estimating, but the potential impact of the unknown is still far larger than in
other engineering fields.
4) As the Dr. Dobbs article points out, the human factor is a massive contributor to this phenomenon. I’ll expand
that to point out that because of the human factor, architecture religion/politics/preconceptions all impact a project
way too much, and way more than in any other industry. At least in terms of getting things done. I’m sure bridge
builders have politics too, but it seems to be less about the methods used than the process of doing the work in that case.
If I’m a commuter who’s going to drive over a bridge, I don’t care what brand of tool was used to lay down the asphalt, I
just want the bridge to work reliably day after day. If I buy a house, I don’t care what brand or model of drill was used
to screw in my screws, I just want the house not to fall down, and last year after year. In the software industry we
spend way too much time arguing over what brand of drill we’ll use to create a piece of software. Pick technologies
that make sense for the project, the environment in which it must work, the constraints of the company, and the experience
of the team doing the work. In other words, stop arguing about ultimately pointless things and make sound decisions based on
a project’s goals and constraints.
The goals of a project will determine what makes sense. If the right people understand this and provide
proper guidance, software development can be more like other more stringent engineering pursuits. What
value would we or the product get from that though, and does it matter ? Who knows, but over the course
of my career I’ve heard one question far more than any other. “Why is it so hard to write good software ?”