Resources for policymakers concerned with managing technical debt
Category: Defining technical debt
Of the numerous definitions of technical debt found in the literature, few are suitable for use as a foundation for enterprise policy. In this category you’ll find examples of definitions of technical debt, along with commentary about their strengths and weaknesses as bases for technical debt management policy.
An enterprise-wide definition of technical debt is essential because effective technical debt management requires cooperation from almost everyone. Absent a shared definition of technical debt, controversy can develop, especially among those who have previously encountered the concept—namely, among technologists. Policymakers can make invaluable contributions to the design of the cultural transformation that will enable control of technical debt.
A physical expression of shared commitment. Effective management of technical debt requires both a shared understanding of what it is and a shared commitment to do what’s required to get control of it.
Li et al. [Li 2015] found that defining what is and what is not technical debt remains a question at issue in the software engineering literature. Even if we restrict the discussion to software constructed in-house, opinions about what constitutes technical debt do differ. The authors found that in the literature of technical debt, “The term ‘debt’ has been used in different ways by different people, which leads to ambiguous interpretation of the term.”
This finding is perhaps the most significant for policymakers, because it suggests that implementing a technical debt management regime will require forging an organizational consensus about the meaning of the term technical debt. The people of most organizations come from a broad array of different backgrounds. Some have little knowledge of technical debt, and therefore have no preconceptions. But those who are aware of the issue, who are mainly technologists and their managers, probably interpret the term technical debtin a variety of ways. Because some of those who do have awareness of the term are likely to have strong opinions about its meaning, one can anticipate a need to resolve these differences of opinion early in the effort to gain control of technical debt.
Some technical terms, like RAID,byte, compiler, and kilowatt,have standard definitions that are widely accepted. Although the term technical debthas found wide use, there is no standard definition for it. What some people categorize as technical debt, others do not. Those who are accustomed to working with terms that have precise, widely accepted definitions might tend to assume that the term technical debtdoes have (or should have) one as well. This assumption can create some difficulty for people who do not realize that others might not share their views as to the definition of the term.
Policymakers must be aware that there is a lack of consensus about the definition of technical debt. Our definition, crafted specially for the use of policymakers, might seem unusually broad to technologists and engineers. For that reason alone, it’s advisable to become familiar with the various ways the term is used by technologists, because understanding their perspective is essential to formulating sound policy deserving of their respect.
References
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
Confirmation bias is a cognitive bias [Kahneman 2011]. It’s the human tendency to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. For example, the homogeneity of cable news channel audiences, and the alignment between preconceptions of the audience and the slant of the newscast for that channel, are results of confirmation bias.
Computer-generated image of the third stage ignition, sending the Mars Climate Orbiter (MCO) to Mars in December, 1998. The spacecraft eventually broke up in the Martian atmosphere as a result of what is now often called the “metric mix-up.” The team at Lockheed Martin that constructed the spacecraft and wrote its software used Imperial units for computing thrust data. But the team at JPL that was responsible for flying the spacecraft was using metric units. The mix-up was discovered after the loss of the spacecraft by the investigation panel established by NASA. One of the many operational changes deployed as a result of this loss was increased use of reviews and inspections. While we do not know why reviews and inspections weren’t as thorough before the loss of the MCO as they are now, one possibility is the effects of confirmation bias in assessing the need for reviews and inspections. Image courtesy Engineering Multimedia, Inc., and U.S. National Aeronautics and Space AdministrationConfirmation bias causes technical debt by biasing the information on which decision makers base decisions involving technical debt. Most people in these roles have objectives they regard as having priority over eliminating technical debt. This causes them to bias their searches for information about technical debt in favor of information that would support directly the achievement of those primary objectives. They tend, for example, to discount warnings of technical debt issues, to underfund technical debt assessments, and to set aside advice regarding avoiding debt formation in current projects.
For example, anyone determined to find reasons to be skeptical of the need to manage technical debt need only execute a few Google searches. Searching for there is no such thing as technical debt yields about 300,000 results at this writing; technical debt is a fraud about 1.6 million; and technical debt is a bad metaphor about 3.7 million. Compare this to technical debt which yields only 1.6 million. A skeptic wouldn’t even have to read any of these pages to come away convinced that technical debt is at best a controversial concept. This is, of course, specious reasoning, if it’s reasoning at all. But it does serve to illustrate the potential for confirmation bias to contribute to preventing or limiting rational management of technical debt.
Detecting confirmation bias in oneself is extraordinarily difficult because confirmation bias causes us to (a) decide not to search for data that would reveal confirmation bias; and (b) if data somehow becomes available, to disregard or to seek alternative explanations for it if that data tends to confirm the existence of confirmation bias. Moreover, another cognitive bias known as the bias blind spot causes individuals to see the existence and effects of cognitive biases much more in others than in themselves [Pronin 2002].
Still, the enterprise would benefit from monitoring the possible existence and effects of confirmation bias in decisions with respect to allocating resources to managing technical debt. Whenever decisions are made, we must manage confirmation bias risk.
References
[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
Team composition volatility can interfere with technical debt retirement. In many organizations, project team composition is rarely fixed from beginning to end. In most teams, people who have special knowledge cycle in and out as the work requires. Although these changes in team composition might not interfere with completing a team’s primary objectives, they can affect the team’s ability to retire technical debt that the team incurs over the life of the project. Changes in team composition can also limit the team’s ability to retire specified legacy technical debt that it encounters while working toward its primary objectives.
“Now we know what we should have done.” This is one kind of incremental technical debt. When the composition of a development team changes over the course of project, recognizing how things should have been done can become more difficult.
Changes in team composition can increase the likelihood of incurring non-strategic incremental technical debt, and increase the likelihood of failing to retire all legacy debt specified in the team’s objectives.
Most product development, maintenance, and enhancement is carried out in groups we call teams. In this context, team is usually defined as, “a small group of interdependent individuals who share responsibility for outcomes.” [Hollenbeck 2012] However, as Hollenbeck et al. observe, teams vary widely in both skill differentiation and composition stability. My sense is that both factors can potentially influence a team’s ability to retire incremental technical debt. They also affect its ability to achieve its objectives with respect to retiring legacy technical debt.
For example, consider what Fowler calls the Inadvertent/Prudent class of technical debt — “Now we know how we should have done it.” [Fowler 2009] In a project of significant size, some might recognize that different approaches to all or parts of it would have been more effective than the ones that were chosen. The recognition might come several months, or even years, after the work affected was conceived or even completed.
But for the moment, consider only cases in which the recognition occurs during the project, or shortly after completion. In these cases, the people who performed that work might have moved on to other teams in need of their talents and abilities. The people who now realize “how we should have done it” might not be themselves capable of making the needed changes, even if they have the budget or time to do the work. Or worse, they might not have the knowledge needed to recognize that a different approach would have been more effective. In either case, recognized or not, the work performed by the people no longer on the team comprises incremental technical debt. Because of team composition volatility, recognizing or retiring that incremental technical debt can be difficult.
Team composition volatility can also interfere with retiring legacy technical debt. Some projects are specifically charged with retiring a class or classes of legacy technical debt. But others with different objectives might also be charged with retiring instances of specific kinds of legacy technical debt as they encounter them. When team members with special knowledge required for the team’s primary objectives are reassigned, some legacy technical debt can remain un-retired, if retiring that debt from the context in which it occurs requires their special knowledge, and if the reassignment occurs before they can complete the legacy debt retirement. This mechanism is more likely to occur when the legacy debt retirement objective is viewed as subordinate to other business objectives.
Keeping team membership stable has big advantages relative the technical debt management. Said differently, organizations that must shuffle people from team to team as a consequence of controlling costs by reducing headcount can pay big penalties in terms of increasing loads of technical debt.
References
[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
Technologists must convey what they know about long-term technology trends to enterprise strategists and others. In addition to strategists, the interested parties include internal customers of technology, product owners, product managers, project sponsors, or senior management. Within the enterprise, technologists tend to be among those most knowledgeable regarding the relative alignment between enterprise technological assets and long-term technology trends. Yet technologists frequently fail to communicate this knowledge effectively to those who need it, and that can lead to non-strategic technical debt. I call this phenomenon technological communication risk.
Hear no evil, see no evil, speak no evil — the iconic representation of communication failure. Technical debt can result from communication failures due to unwillingness to inform others of what you know, and unwillingness to receive information from others more knowledgeable.
Technological communication risk is the risk that knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. Policymakers can address this problem by working to define the roles of all involved to specify the need for this communication, and the need to be receptive to it.
A clear understanding of long-term technology trends is important in managing technical debt. Any significant misalignment between enterprise technological assets and long-term technology trends creates a risk of incurring new technical debt. As technologies evolve, enterprise assets must evolve with them. The gap between those assets and the state of the art is a source of lost productivity and depressed organizational agility, which is our definition of technical debt.
Some technologists are better informed about technology trends than are their internal customers, product owners, product managers, project sponsors, or senior management. Technologists often do attempt to communicate what they know on an informal basis, but unless such communication is expected and defined as an official duty, their superiors and internal customers don’t always welcome the information, especially if they haven’t heard it elsewhere, or if it conflicts with what they’ve learned elsewhere, or if its implications conflict with established strategic positions.
Many technologists are aware that their superiors might not welcome their observations about technological trends or technology-based strategic vulnerabilities or opportunities. For example, one might understand why a technologist might be reluctant to alert an unreceptive senior manager to a suddenly revealed cybersecurity risk that would be very expensive to mitigate. This mechanism is especially strong when deploying adequate cyberdefense would compete for resources with other initiatives already underway, or when the negative consequences of the vulnerability are unlikely to materialize. And some tend to question technologists’ credibility when they blame the technologists for the vulnerability itself.
Situations like these can lead to the formation of new non-strategic technical debt in circumstances such as the following:
Management directs the technologists to produce capabilities using approaches known to the technologists to be technological dead ends.
Management directs the technologists to implement capabilities that don’t exploit known approaches that could open new and vital lines of business.
Management directs the technologists to focus resources on initiatives that in the view of the technologists lack sufficient technological imperative.
Policymakers can mitigate technological communication risk by establishing internal standards that encourage knowledgeable technologists to share what they know with internal customers, project sponsors, or senior management. Similarly, those standards can encourage internal customers, project sponsors, product owners, product managers, and senior management to take heed when knowledgeable technologists do speak up.
References
[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
The behavior of internal customers and users of enterprise technological assets can contribute to technical debt formation and persistence. Because of these contributions, introducing effective technical debt management practices requires widespread behavioral changes on the part of those internal customers and users. Accepting these changes, and the initiative and creativity they require, is possible only if people understand the technical debt concept. When they do, they can appreciate the benefits of controlling technical debt, and the consequences of failing to control it. Similarly, when they do not understand or accept the technical debt concept, progress toward effective technical debt management is unlikely. Policymakers can contribute to the planning and execution of the required organizational transformation.
Even when the engineering teams are aware of the technical debt concept, and when they do try to manage technical debt, they cannot make much progress unless they have the support and understanding of their own management, their internal customers, and their customers’ managements. Everyone must understand that controlling technical debt — and retiring it — is a necessary engineering activity that has a business purpose. Everyone must understand that technical debt arises as a result of everyone’s behavior — not just the behavior of technologists.
A tensegrity three-prism. . Read about tensegrity structures. Image (cc) Bob Burkhardt courtesy Wikimedia.Part of the job of Management is to see that engineers have what they need to avoid incurring technical debt unnecessarily, and that they have what they need to retire elements of legacy technical debt on a regular basis. Internal customers must understand that communicating their long-term business strategies to Engineering is essential for limiting unnecessary creation of artifacts that become non-strategic technical debt. Only by understanding the technical debt concept can internal customers learn to avoid the behaviors that lead to non-strategic technical debt, and adopt behaviors that limit new technical debt.
Tensegrity structures provide a metaphor for organizations that have mastered the technical debt concept. Tensegrity structures use isolated rigid components in compression, held by a network of strings or cables in tension. The rigid components are usually struts or masts, and they aren’t in contact with each other.
The struts correspond to the users or customers of technological assets. The cables correspond to the engineering activities required to support the customers. The organization is stable relative to technical debt only when the two kinds of elements (struts and cables) work together, each playing its own role, but each appreciating the role of the other.
Advocating for cultural transformation
Advocates of any change to organizational culture are often seen as acting in their own self-interest. That’s a common risk associated with cultural transformation. It’s a risk that can lead to failure when inserting practices related to technical debt management into the culture. The risk is greatest when advocates for change are drawn exclusively from the technical elements of the enterprise. The ideal advocates for these ideas and practices are the internal customers of the technical organizations, and senior management.
References
[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
Failure to communicate long-term business strategy can lead to increased technical debt, because engineering decisions that aren’t aligned with business strategy can result in what later becomes technical debt. As business strategy veers away from the assumptions underlying those misaligned engineering decisions, engineers must alter implementations to track the strategy. Technical debt can form during those alteration efforts. Moreover, the expenditure of resources to support those alteration efforts might have been unnecessary if engineers had been better informed about long-term business strategy. In some cases, those resources could have been allocated to other pursuits, including technical debt retirement. To ensure alignment of engineering decisions with long-term business strategy, engineering decision-makers must be aware of long-term and intermediate-term enterprise strategy. When they’re well informed, they can anticipate the engineering needs of the enterprise. And they’re more likely to make decisions that are compatible with strategy.
Recharging a plug-in electric vehicle. The dominance of petroleum-powered vehicles is nearing its end. Further investment in the petroleum-based fuels infrastructure is now inconsistent with what the global economy has chosen as its strategic intent. Electric vehicles are still out of the reach of most consumers, but they would be wise to favor long range vehicles. As electric vehicles become ascendant, petroleum filling stations will become more widely spaced. During the transition to electric power dominance, long-range petroleum-powered vehicle offers clear advantages over its shorter-range cousins.
Moreover, the effect is bi-directional. Strategists can benefit from understanding the effect their strategies have on technological activity. For example, consider the process of choosing among strategic options. A favorable outcome is more likely if strategists know the effects of each option on the technical debt portfolio.
To gain effective control of technical debt, senior management must regard the technical elements of the enterprise as strategic partners [Woodard 2013] [Ross 2000] [Brenner 2016a]. Policymakers can make important contributions to enhancing communication between strategists and technologists.
For example, when engineers know the general direction of the enterprise, they can focus efforts on assets that are compatible with future needs. Inversely, when they’re unaware of what the business strategy might soon require, they’re more likely to make decisions that they must later rescind.
What about legacy technical debt retirement?
Analogous considerations apply to legacy technical debt retirement efforts. Major technical debt retirement efforts are often subject to review for alignment with enterprise strategy. But we tend not to review incidental retirement efforts that occur in the context of routine maintenance or development. Consequently, engineers might allocate effort to incidental debt retirement unnecessarily if the asset is due for overhaul or replacement. Communicating long-term strategy effectively is likely the most reliable way to prevent such misspent effort.
Some managers elect to communicate business strategy to technologists only when they “need to know.” Often, technologists needed to know long before that.
References
[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.
[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37(2), 537-564 (2013).
Non-strategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce non-strategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. In this group of posts I examine a variety of precursors of non-strategic technical debt that are not directly related to technology. Sources of these precursors include:
Communication between and among people
Organizational policies relating to job assignments
Some of the suspension cables of the Brooklyn Bridge. Washington Roebling, the chief engineer, designed the cables to be composed of 19 “strands” of wire rope [McCullough 1972]. Each strand was to be made of 278 steel wires. Thus, the original design called for a total of 5,282 wires in each of the main cables. After the wire stringing began, the bridge company made an unsettling discovery. The wire supplier, J. Lloyd Haigh, had been delivering defective wire by circumventing the bridge company’s stringent inspection procedures. In all, Roebling estimated that 221 U.S. tons (200 metric tons) of rejected wire had been installed in the bridge. This was a significant fraction of the planned total weight of 3,400 U.S. tons (3,084 metric tons). Because they couldn’t remove the defective wire, Roebling decided to add about 150 wires to each main cable. That extra wire would be provided at no charge by Haigh [Talbot 2011]. I can’t confirm this, but I suspect that Roebling actually added 152 wires, which would be eight wires for each of the 19 strands, to make a total of 286 wires per strand, for a total of 5,434 wires. The presence of the defective wire in the bridge cables—which remains to this day—is an example of technical debt. The fraud perpetrated by Haigh illustrates how malfeasance can lead to technical debt.I use the term precursor instead of cause because none of these conditions leads to technical debt inevitably. From the perspective of the policymaker, we can view these conditions as risks. It’s the task of the policymaker to devise policies that manage these risks.
McConnell has classified technical debt in a framework that distinguishes responsible forms of technical debt from other forms [McConnell 2008]. Briefly, we incur some technical debt strategically and responsibly, and we retire it when the time is right. We incur other technical debt for other reasons, some of which are inconsistent with enterprise health and wellbeing.
The distinction is lost on many. Unfortunately, most technical debt is non-strategic. We would have been better off if we had never created it. Or if we had retired it almost immediately. In any case we should have retired it long ago.
It’s this category of non-strategic technical debt that I deal with in this group of posts. Although all technical debt is unwelcome, we’re especially interested in non-strategic technical debt, because it is usually uncontrolled. In these posts I explore the non-technical mechanisms that lead to formation of non-strategic technical debt. Schedule pressure is one exception. Because it’s so important, it deserves a thread of its own. I’ll address it later.
Common precursors of non-strategic technical debt
Here are some of the more common precursors of non-strategic technical debt.
I’ll be adding posts on these topics, so check back often, or subscribe to receive notifications when they’re available.
References
[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.
[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37(2), 537-564 (2013).
Although deploying technical debt management policies is necessary, it might not be sufficient. Malfeasance can lead to incurring new technical debt. It can prevent retiring—or even identifying—existing technical debt. Correcting, detecting, and preventing malfeasance must be goals of technical debt management policy.
Because there is no known controlled study providing evidence that supports the application of the Broken Windows theory to technical debt management, Broken Windows as applied to technical debt management must be regarded as an intriguing conjecture awaiting scientific examination.
The urge to control technical debt by designating one group as the “owner” of the asset doesn’t avoid the tragedy of the commons. It leads inevitably to duplication and more technical debt. A more effective approach: Communities that share specific assets must devise customs, rules, and sanctions for managing them.
Organizational psychopathy can be the dominant contributing factor in generating technical debt when the primary beneficiary of a decision is the decision-maker or the advocate, and when he or she intends to move on to employment elsewhere before the true cost of the debt becomes evident.
Contract restrictions can lead to technical debt. When contractor organizations customize an asset they own for customers, they usually mean to retain ownership of the asset. But segregating the asset from the customizations can lead to technical debt. Here's how.
Using the term interest to refer to the metaphorical interest charges that are associated with a technical debt is risky. The risk arises from confusing the properties of financial interest with the properties of the metaphorical interest charges on technical debt. Using an alternative term that makes the metaphor obvious can limit this risk. One such term is metaphorical interest charges, or for convenience, MICs.
Loose change. The MICs on technical debt are accumulated in two ways: (a) as “loose change,” namely, small bits of lost time, delays, and depressed productivity; and (b) as major blows to enterprise vitality in the form of lost revenue, delayed revenue, and missed market opportunities. Hard to say which category does more damage.
MICs aren’t interest charges in the financial sense; rather, the MICs of a technical debt represent the total of reduced revenue, incidental opportunity costs, and increased costs of all kinds borne by the enterprise as a consequence of carrying that technical debt. (Actually, now that I think of it, MICs can include financial interest charges if we find it necessary to borrow money as a consequence of carrying technical debt.) Because the properties of MICs are very different from the properties of financial interest charges, we use the term MICs to avoid confusion with the term interest from the realm of finance.
Briefly, MICs are variable and often unpredictable [Allman 2012]. MICs differ from interest charges on financial debt for at least six reasons. For any particular class of technical debt:
I examine each of these properties in more detail in the posts listed above.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, vol 10 issue 3, March 23, 2012.
[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.
[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37(2), 537-564 (2013).
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
For financial debts, the interest charges associated with a unit of debt are (usually) the same for every unit of debt incurred under the same loan agreement. But for technical debt, the MICs associated with a given instance of a class of technical debt might differ substantially from any other instance of the same class of technical debt, even if those instances of technical debt were incurred at the same time as a result of a single decision or sequence of events.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
In some instances, technical debt is actually a missing or incompletely implemented capability. For example, absence of a fully automated regression test suite can create difficulties for testing a complex system after a series of modifications. Defects can slip through. That would result in reduced productivity and velocity. In this case, the projected cost of implementing, testing, and documenting the test suite, and training its users, would constitute the initial MPrin of the outstanding technical debt. This definition differs from some definitions, because it includes testing, documenting, and initial training. In general, from the enterprise perspective, when identifying the MPrin associated with missing or incompletely implemented capabilities, we must include all artifacts necessary to eliminate reductions in productivity and velocity.
A concrete building under construction. Concrete takes about a month to fully cure, and until it cures it doesn’t reach full strength. If we waited for full curing before pouring the floor above, multistory concrete construction would be slow and very expensive. So we start on the next floor after only a few days. Because the floors of concrete buildings can’t support themselves for about a week or so, they need to be shored up by the floor below, and re-shored by the floors below that. The shoring constitutes a technical debt incurred because of the “incomplete implementation” (partial cure) of each floor. We retire that debt incrementally, floor by floor, by removing the re-shoring as the building gets taller. More about shoring and re-shoringBut even if we include these items in the conventional definition, MPrin at retirement can exceed the savings at the time we incurred the debt. For example, the assets involved can change. Moreover, if retiring the debt a revenue stream interruption, and that revenue stream grows significantly, the MPrin, which includes the lost revenue, can also grow significantly.
Technical debt associated with incompletely implemented capability presents unique problems, because debt of this kind can be retired in three distinct ways. First, we can complete the implementation. The MPrin associated with this approach can grow beyond the initial cost of completion, for the usual reasons. Second, we can cancel the capability altogether. If that happens, retiring the debt completely would require removal of all artifacts that comprise the completed parts of the incomplete implementation. But full retirement can also require that we survey all components that interact with the retiring elements to determine whether they contain accommodations that are no longer necessary. Finally, we can choose a middle path, in which we adopt some parts that have been completed, reject other parts, and add whatever is necessary to create a limited version of what was originally planned.
It’s worth mentioning an important attribute of non-physical assets—software, procedures, legislation, regulations, and so on—that makes the technical debt associated with incomplete implementation so difficult to manage. The image above shows several levels of a concrete building under construction. The vertical members between the levels are part of a shoring system that supports the levels of the building until the concrete is cured well enough to support itself. They constitute a kind of technical debt that must be “retired” before the building can be completed. The teams constructing the building could never forget to remove the shoring because it gets in the way of installing the windows and walls.
But things are very different with non-physical assets. It’s easy to forget to remove intermediate artifacts, or elements that were being tried but didn’t work out. Many non-physical assets are perfectly functional carrying that kind of technical debt. The problems associated with that debt become evident with time, as the asset becomes increasingly difficult to maintain, extend, or defend.
It is this property of non-physical assets that makes technical debt management so much more difficult than it is with physical assets. Not more important, just more difficult.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, vol 10 issue 3, March 23, 2012.
[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.
[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37(2), 537-564 (2013).
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
The MPrin of technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. The conventional definition of the MPrin for this kind of technical debt includes only the cost of aligning the existing assets to the new standards or regs. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required not directly by the change in standards or regs, but by the requirement that other assets maintain compatibility with the assets affected directly by the change in standards or regs.
The phrase standards or regs is beginning to bother even me. I’ll switch to standards with the understanding that I mean standards or regulations (or regs) except when I say so explicitly.
A view of the left side guard on a truck operated by the City of Cambridge, Massachusetts. Side guards prevent vulnerable road users (pedestrians, bicyclists, and motorcyclists) from being swept under trucks and crushed (and often killed) by the truck’s rear wheels. Cambridge has a pilot program affecting city-operated trucks, but Boston is requiring all contractors to install side guards. This change in regulations creates a technical debt for all truck operators whose vehicles lack side guards. [Volpe 2017] City of Cambridge photo courtesy U.S. Department of Transportation.
The activity required to align existing assets to the new standards can have expensive consequences, which must be included in the calculation of MPrin, but which, unfortunately, are often overlooked or accounted for in other ways. For example, the testing required by the standards alignment effort might require a service interruption or product availability delays or interruptions, which could entail a revenue stream delay or interruption. That lost revenue is certainly a consequence of the debt retirement effort.
Deferring retirement of this class of technical debt can expose the enterprise to the risk of MPrin growth in two ways. First, when we defer debt retirement, the number of instances of violations of the new standards can increase, of course, as new assets are developed in compliance with the obsolete standards. Second, and less obvious, perhaps, is the potential for increases in the number of ripple effect instances when we defer debt retirement. These instances arise from increases in the number of artifacts that require updating, not because they are not compliant with the new standards, but because they need to be made compatible with the components that are eventually modified to comply with the new standards. In this way, MPrin at debt retirement time can differ from — and greatly exceed — the savings realized when we first incurred the debt.
However, as with most technical debts, deferring retirement of this class of debt can sometimes be wise. For example, if the assets that bear the debt are about to be retired, the technical debt they carry vanishes when those assets are retired.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, vol 10 issue 3, March 23, 2012.
[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.
[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37(1), 82–106 (2012). doi:10.5465/amr.2010.0181
[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28.3 (2002): 369-381.
[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.
[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37(2), 537-564 (2013).
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
