An enterprise-wide definition of technical debt is essential because effective technical debt management requires cooperation from almost everyone. Absent a shared definition of technical debt, controversy can develop, especially among those who have previously encountered the concept—namely, among technologists. Policymakers can make invaluable contributions to the design of the cultural transformation that will enable control of technical debt.
A physical expression of shared commitment. Effective management of technical debt requires both a shared understanding of what it is and a shared commitment to do what’s required to get control of it.
Li et al. [Li 2015] found that defining what is and what is not technical debt remains a question at issue in the software engineering literature. Even if we restrict the discussion to software constructed in-house, opinions about what constitutes technical debt do differ. The authors found that in the literature of technical debt, “The term ‘debt’ has been used in different ways by different people, which leads to ambiguous interpretation of the term.”
This finding is perhaps the most significant for policymakers, because it suggests that implementing a technical debt management regime will require forging an organizational consensus about the meaning of the term technical debt. The people of most organizations come from a broad array of different backgrounds. Some have little knowledge of technical debt, and therefore have no preconceptions. But those who are aware of the issue, who are mainly technologists and their managers, probably interpret the term technical debtin a variety of ways. Because some of those who do have awareness of the term are likely to have strong opinions about its meaning, one can anticipate a need to resolve these differences of opinion early in the effort to gain control of technical debt.
Some technical terms, like RAID,byte, compiler, and kilowatt,have standard definitions that are widely accepted. Although the term technical debthas found wide use, there is no standard definition for it. What some people categorize as technical debt, others do not. Those who are accustomed to working with terms that have precise, widely accepted definitions might tend to assume that the term technical debtdoes have (or should have) one as well. This assumption can create some difficulty for people who do not realize that others might not share their views as to the definition of the term.
Policymakers must be aware that there is a lack of consensus about the definition of technical debt. Our definition, crafted specially for the use of policymakers, might seem unusually broad to technologists and engineers. For that reason alone, it’s advisable to become familiar with the various ways the term is used by technologists, because understanding their perspective is essential to formulating sound policy deserving of their respect.
References
[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
Tension between policymakers and technologists, which can manifest as misalignment of their respective priorities, is a significant contributor to uncontrolled growth of technical debt. In this thread I explore sources of this tension and introduce concepts that can assist policymakers and technologists in their efforts to control the growth of technical debt.
Effective, sustainable control of technical debt is the objective of technical debt management policy. In an enterprise that has achieved this objective, technical debt serves as a strategic tool that assists in attaining and maintaining market leadership. In such an organization, technical debt does exist, and some legacy technical debt might remain in place, but technical debt growth is managed strategically, if growth occurs at all. Any technical debt that carries significant MICs, and which compromises productivity and enterprise agility, is addressed and retired with appropriate priority. In short, technical debt is addressed not solely as a technological issue, but as a component of business strategy.
Raindrops on a grapevine. We often think of tension as a negative, destructive force. But tension—in the case of raindrops, surface tension—is what holds a raindrop together. Tension gives the raindrop structure and integrity. The tension between technologists and policymakers can also be constructive. It can ensure that the enterprise manages its technical debt in ways that balance the political imperatives of both technology and strategic health.
This stance is at odds with the historical position most enterprises have adopted vis-à-vis technical debt. Historically, technical debt has been seen as a technical problem, if it has been recognized at all. Most enterprises have left the management of technical debt to the technologists. Frequently, then, the policymaker who enters the discussion about technical debt might be seen by technologists as an interloper, arriving late to the discussion, or as a less-than-knowledgeable invader attempting to seize control of a piece of the technologists’ domain. Tensions can arise between policymakers and technologists. Such tensions complicate the problem of managing technical debt.
One possible source of this tension is revealed in a study of the literature of technical debt, which is evolving so rapidly that it has itself become a focus for research. Li et al. [Li 2015] have produced a review of the software engineering technical debt literature, from which we can extract insights useful to policymakers. Although they studied only the literature relating to technical debt in software engineering, their conclusions are, at least in part, applicable to any field in which the components of the finished product are executed within software tools before being committed to operational form. This covers a wide array of knowledge-intensive endeavors, including mechanical system design, electronic design, framing of legislation, process design, architecture, and even book authorship.
In this thread, I explore the sources of the tension between the modern reality of technical debt as an enterprise issue, and the historical situation of technical debt as a technological issue. This can serve as a guide for policymakers in reframing technical debt from a technological issue dependent for resolution on enterprise resources, to an enterprise issue dependent for resolution on technological resources.
References
[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
Stovepiping can lead to technical debt. Actual stovepipes — the tubes that vent exhaust from stoves — serve as a metaphor for the flow of information in “stovepipe” organizations. In “stovepipe” organizations, information flows predominantly (or only) up or down along the parallel chains of command, and rarely (or never) from a point in one chain of command across to some other chain of command [Waters 2010]. The stovepipe metaphor is imperfect, in the sense that in actual stovepipes, smoke and fumes rarely flow downwards. By contrast, in organizations, some information does flow down the chains of command. But the metaphor does clarify the problem that whatever the organization learns in one metaphorical stovepipe isn’t easily transferred to other metaphorical stovepipes.
A wood-burning stove in a farm museum in Lower Bavaria (German: Niederbayern), which is one of the seven administrative regions of Bavaria, Germany. The stovepipe, which is the black tube running upwards from the stove, channels smoke and fumes out of the kitchen into the chimney, and then, presumably, out of the farmhouse.
Stovepiping can occur in both organizational structures and in engineered systems. These two forms of stovepiping are intimately related, and both can lead to uncontrolled formation of new technical debt, or increased persistence of existing technical debt.
In organizational structures, stovepiping occurs when similar capabilities are implemented in elements of two or more different organizational units that act relatively independently of each other. An example is the dispersal of some elements of the IT function out into IT’s customers. When independent organizations have similar technical needs, they’re at risk of independently implementing technological capabilities that duplicate each other, at least in some respects.
In engineering, stovepiping occurs, for example, when two or more technological assets are managed and maintained independently [McGovern 2003]. In that situation, distinct engineering efforts working on those assets might happen to solve the same problem, possibly in two different ways, with each party either ignorant, or possibly disparaging, of the other’s efforts.
In whichever way duplication of technological capability comes about, it can lead to increasing levels of technical debt, or to increased persistence of technical debt. This happens because the organization might be required to execute future maintenance or enhancement multiple times — once for each instance of the technical artifact. That exposes the organization to additional cost, additional load on its staff, and additional risk of creating defects and incurring liability, compared to a situation in which technical assets are shared among all who need them.
The problem is actually even more worrisome. First, in the case of a defect found in one version of a technological artifact, it’s possible that the people who are aware of the defect might not realize that another version of the artifact exists. If that other version also has an analogous defect, its defect might go unrecognized for some time, with all the usual attendant negative consequences. Second, in the case of a necessary extension of the artifact’s capabilities, the maintainers of one version might recognize the need for an extension, and implement it. Meanwhile the maintainers of other versions might not recognize the need for extension. They might not take action until something bad happens or a possibly urgent need arises. It’s easy to conjure other unfavorable — and costly — scenarios.
In engineering more generally, stovepiping can occur internally in systems, even though only one business unit is involved, and even though the stovepiped artifacts serve purposes invisible to the world outside the system. This can occur whenever communication is weak between the teams designing or maintaining the portions of the system that host the similar artifacts. For those familiar with the Apollo XIII incident, the incompatibility of the two carbon dioxide scrubbers in the command module and the lunar excursion module serves as an example of the risks of technical stovepiping.
When distinct business units or functions operate their own engineering or IT functions, or when they depend on a shared engineering function but require similar work, there is an elevated probability of duplication of technological assets or capabilities. This happens when the organizational structure, or the timing of the work, encourages separation of the engineering efforts. Engineering or IT functions operated separately under the control of distinct business units or functions can clearly produce duplicated capabilities. However, duplication can also occur when the engineering function is shared across distinct business units or functions, but the actual people and teams performing the work differ for different efforts, and when communication is weak between those teams. This can happen whether or not the efforts are conducted contemporaneously.
Because identifying these forms of technical debt after they’re created is notoriously difficult, preventing their formation is preferable to trying to detect them post-implementation. Prevention is possible if the enterprise establishes mechanisms that facilitate consultation and sharing among elements of different, separately operated technology development or maintenance functions. In other words, the organization must “break” the stovepipes — no mean feat, politically speaking.
Another challenge, of course, is providing resources for such sharing mechanisms, because preventing technical debt is rarely recognized as a value generator. If it were so recognized, the resources would likely appear. Changes in cost accounting might make such recognition more likely.
References
[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.
