Monday, January 17, 2005

Ethics at Work

Decision-makers hold key to the ethical side of technology development

By David H. Gleason

Ethical decisions are embedded in technology during design and development. Once a technological system is operating, it repeats those ethics whether or not anyone is watching. The effects of technology - its ethics - are often hard to predict and are usually irreversible.

For example, genetically-modified corn, once released into the environment, would reproduce on its own. Such modifications may strengthen crops and reduce the need for chemical pesticides, but new genetic material may also express in unexpected and damaging ways, for instance, by producing apiary toxins. As new technologies are developed, potentially negative consequences must be considered beforehand to mitigate harmful risks to whatever extent possible.

Subsumption ethics is the process by which decisions become incorporated into the operation of technology and are subsequently forgotten. Technological systems, by nature, repeat operations. Even if those operations have an unethical effect, the systems would continue to execute the operations anyway.

In most technology, small components are developed and tested, and once they are working reliably, they are subsumed into larger systems. The larger systems, in turn, are subsumed into still larger ones. Once components, subsystems and systems are operating, subsumed processes become invisible and unavailable to the user. Each component is forgotten as it is subsumed, and requires no further attention unless it fails. In this way, technological systems are inherently built to eventually become enormous.

To the lay person, technology seems like it should be extremely malleable. For example, people tend to think changes to software should be easy because programming is just a set of instructions, and not a large structure such as a building made of concrete and metal. The principle of subsumption, however, makes it clear that changing base components is much like moving the foundation of a building - most likely it requires changes to entire systems. The year Y2K problem, for example, was a result of subsumed date processing - no-one thought the systems would be around long enough to need four-digit dates.

Subsumption is also a one-way street. Technology is built over time, and components, or subsumed objects, could cause a major problem only much later and under certain circumstances. However, once something goes terribly wrong with technology, it usually can't be undone. In the Challenger and Columbia space shuttle disasters, for example, once subsumed errors manifested, lives had already been lost. Retrospective analysis often shows the causes of problems, and we learn lessons to avoid repeat mistakes. But usually the problems turn out to have been subsumed well before their manifestation. In Challenger's case, the O-rings were designed and manufactured years before they actually failed.

Some failure is inevitable because we are pushing the envelope and dealing with complex, untested technologies and because consequences are hard to predict without experience. But this explanation does not excuse decision-makers from responsibility for harmful outcomes. Indeed, as professionals we are expected to know how technology would amplify our activities.

In the Information Age, business decisions are reaching further than ever before. For example, business systems will flawlessly execute bad decisions indefinitely, because those decisions are codified into software programs. Quick judgments of line managers, programmers, consultants and salespeople can be amplified by technology to monster proportions.

Subsumed decisions are also cumulative. If we want technology to have integrity, reliability and maintainability, then we must consistently make good decisions about how to design and build it. This requires a new orientation on the part of all involved, from executive sponsors to engineers - that is not an easy task.

Outstanding technology projects -those completed on time, within budget and with positive outcomes - are the result of good management first, and technology second. They can be characterized by high-performance teams, rigorous project management techniques and a focus on the human effect the technologies would have. Those people and groups with an interest in the outcomes are the actual stakeholders in any given project.

Project stakeholders can be broadly defined to include those directly and indirectly affected by both the project and its products. They may not have a voice, such as residents of a community in which a particular facility is located. Nevertheless, the consequences of subsumption in technology development should be assessed for each stakeholder.

Technical experts have a job to do, but those experts must also consider the consequences their decisions would have on people and communities. Close attention needs to be paid when creating complex or self-replicating systems as one way to advance the effects - the ethics - of the decisions being embedded into emerging technologies.

David H. Gleason is managing consultant with Boston-based Information Ethics (www.info-ethics.com), an IT consulting firm focused on applied ethics.