Resources for policymakers concerned with managing technical debt
Category: Defining technical debt
Of the numerous definitions of technical debt found in the literature, few are suitable for use as a foundation for enterprise policy. In this category you’ll find examples of definitions of technical debt, along with commentary about their strengths and weaknesses as bases for technical debt management policy.
An aisle in the stacks of a library. Some libraries are upgrading their book tagging systems from barcodes to RFID tags. This is essentially a platform component upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day [Boss 2011]. It’s a big job.
The MPrin of technical debt that forms as a consequence of a platform component upgrade depends on how we incur the debt. If we incur the debt by installing the upgrade, and then perform only some of the work made necessary by the upgrade, then the MPrin is the total cost of performing the deferred work. If we incur the debt by deferring the upgrade, then the conventional definition of the MPrin has two components. The first is the cost of the upgrade, and the second is the cost of any work made necessary by the upgrade, but not performed.
Hold on—it’s not so simple
In this latter instance, MPrin can increase over time. Increases can occur if the following three-step sequence happens for either maintenance or enhancements. In step 1, we perform work in the environment of the obsolete platform component, but after deferring the upgrade. Step 2 is performance of the upgrade. In step 3, we must repeat the work we performed in step 1 because the step 1 version isn’t compatible with the upgrade. This situation can be even worse if we discover the need for step 3 as a result of operational failure after the upgrade. In that case, maintainers must investigate the failure first. And the failure might cause database contamination, which would also need remedying. These additional costs are actually part of the debt retirement effort for the debt incurred by deferring the upgrade, but we usually account for them—mistakenly—as routine operational expense.
Last words
Advance knowledge of what can go wrong is always a nice-to-have. Most of us try to acquire this knowledge before or as we plan our projects. And most of us can do better. Before you consider a plan complete, ask yourself if anyone else might have already tried something similar. If you can guess who that might be, contact him or her to find out how it went. No point repeating someone else’s mistakes.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
The “basket bridge” of the Los Angeles Metro system. Constructed by the Metro Gold Line Foothill Extension Construction Authority, and opened in 2012, it carries the Los Angeles to Pasadena Metro Gold Line light rail across the eastbound lanes of the I-210 Freeway. The rail line provides transport service to a ridership that had been driving or using bus service. Although the bus and rail aren’t exact duplicates of each other, there is enough overlap that bus ridership dropped significantly after the rail line opened. That drop created a technical debt in the bus system. Retiring that debt requires reducing, rescheduling and re-routing bus service [Broverman 2017].
We usually regard the MPrin of new technical debt associated with a development project as the difference between the cost of implementing it sustainably and the cost of simply making it work. The effort saved by choosing the latter over the former is the initial MPrin of the technical debt.
For example, consider an enhancement project for an existing asset. After we achieve an operational capability, we might notice that we’ve duplicated some of the asset’s pre-existing functionality. The responsible debt-free approach has three stages. First, we eliminate the new and unnecessary duplicated capability. Next, we modify the asset to use the previously existing capability. Finally, we re-test the asset. The approach that generates new debt involves leaving the duplication in place.
Other generators of technical debt
In a closely related example, we might recognize that the duplicated functionality in the newly developed portion of the asset is superior to the pre-existing form elsewhere in the asset. We’d like to remove the pre-existing form and replace instances of that form with instances of the newly developed functionality. But that work is clearly outside the scope of the new development, and it must await budgetary authority before it can be undertaken. Consequently, it becomes technical debt for the larger asset.
As time passes, and the enterprise undertakes other development projects, the implementations of previous projects can accumulate additional technical debt. The more obvious mechanisms by which this occurs include defect discovery, customer requests for enhancements, the need to enhance cyberdefenses in response to new threats, or changes in law or regulation. Less obvious, but no less significant, are changes in markets, customer needs, and underlying technologies. All can contribute to technical debt formation
A final example
An example of a less obvious process might be insights gained in marketing one product (call it P1). Suppose those insights reveal an opportunity to introduce other related products—P2, P3, and P4—to form a suite. The latter products could employ some assets developed for P1, if the latter products had been constructed slightly differently. The changes required in P1 therefore constitute technical debt, because we would have been able to develop P2, P3, and P4 much more rapidly if we had recognized the opportunity earlier. The P1 changes then become technical debt. And if we pursue P2, P3, or P4 without first retiring that debt, the debt probably expands, because the subsequent products manifest it.
New product (or service) developments often generate these situations.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Some examples might help to clarify the differences between the principal of financial debts and the MPrin of a technical debt. The examples to come in the next four posts illustrate the unique properties of MPrins of technical debts.
Technical debt can arise as a result of:
Changes internally within the enterprise
External environmental changes that directly affect existing assets
Changes in the competitive environment
Insights and changes in perception that lead to changes in objectives
Existing technical debt
Deferring decisions about almost anything
By contrast, we incur financial debt only as a result of decisions to incur financial debt.
The examples below illustrate some of these phenomena. For each one, the full post contains a more complete explanation.
Development projects
This example illustrates how technical debt can develop as a result of unanticipated insight regarding a marketing opportunity for a new product line. It shows how technical debt can arise independent of any decision made within the enterprise, and without any changes to assets of any kind. More: “MPrin in a development project”
Platform component upgrades
We’ve already provided an example of technical debt arising from a platform upgrade. In this example, we show how deferring a platform upgrade creates new complications not present in the previous example, by illustrating how total MPrin can increase after the debt first forms. More: “MPrin in a platform component upgrade”
Standards or regulatory changes
Changes in standards or regulations, whether internal, industry-wide, or governmental, can create technical debt. In some cases, the enterprise might not even be aware of the new debt. More: “MPrin when standards or regulations change”
Missing or incompletely implemented capability
When a capability is incompletely implemented, it’s clear that the part left undone constitutes technical debt. What is less clear is what happens when the capability implementation is halted or rescinded. Trying to avoid new technical debt can actually be the cause of new technical debt, albeit of a different kind. More: “MPrin for missing or incomplete capability”
Whether or not any similar scenarios have happened in your organization, these discussions are helpful for gaining insight into what kinds of technical debt policies can assist your organization in managing its technical debt. Let me know if you have experience with situations in which problems can be traced, even if only in part, to treating technical debt as if it were financial debt.
Because the term technical debt is a metaphor, it causes us to think in ways that sometimes create barriers to managing technical debt responsibly. Like all metaphors, the technical debt metaphor carries with it unintended associations—attributes of the metaphor’s source that the listener or reader attributes to the metaphor’s target, even though the attributions aren’t intended by the metaphor’s author. For the technical debt metaphor, these unintended associations relate to the concepts of debtor, principal, and interest, and they can cause enterprise decision makers to arrive at erroneous decisions.
Because metaphors compel our minds to accept the identification between source and targetin toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not. This misidentification is an acceptable risk, if we understand the risk and if we manage it properly. Unfortunately, we rarely recognize that risk. And unrecognized risks usually remain unmitigated. A significant source of this risk is our inability to control which attributes of the metaphor’s source the reader or listener chooses to associate with the metaphor’s target. I call this phenomenon unintended association.
Interest and principal have unintended associations
University graduates celebrate commencement. Some, perhaps most, carry a burden of student loan debt in addition to their diplomas. Student loans, now familiar to many, act as a source for the technical debt metaphor. They are therefore also a source for its unintended associations.Two sets of unintended associations that frequently arise in the context of the technical debt metaphor relate to interest and principal. In the world of finance, we understand these concepts well. Unfortunately, our understanding from finance doesn’t fit well with the details of the corresponding properties of technical debt.
For example, van Haaster [van Haaster 2015] writes: “Financial debt has two well understood dimensions: the amount owing and its cost to repay over time, consequently when you take on financial debt, the total cost of that debt over time is either known or can be calculated.” Such beliefs about financial debt have consequences for our thinking about technical debt, because van Haaster’s statement is inapplicable to technical debt, for two reasons. First, the cost to repay a technical debt might not be well known. Second, the interest charges on technical debt might not be known, might not even be knowable, and often cannot be calculated [Falessi 2014]. These are just two examples of differences between financial debt and technical debt. We’ll explore these differences in some detail in coming posts.
How technical debt can be seen as evidence of mismanagement
A most significant unintended association is related to the concept of debt itself. Consider, for example, the social status of debtors in society. For many, excessive financial debt evokes images of profligate spending, laziness, and moral decay. These associations can hinder technology leaders within organizations as they urgently advocate for resources for technical debt management.
Because of unintended association, some decision makers outside the technology-oriented elements of the enterprise might regard technical debt as evidence of mismanagement. They might tend to attribute the cause of technical debt to professional malpractice by technologists. They see supportive evidence in the technology managers’ uncertainty about the size of the debt or how they acquired it. To the extent that nontechnical decision makers adopt this attitude, they are unlikely to support enterprise policy changes. They’re even less likely to support additional resources for technical debt management.
But the problems of the technical debt metaphor can be even more significant. The classic work of Lakoff and Johnson [Lakoff 1980] offers an explanation in terms of a metaphor (of course). In this metaphor:
Ideas are objects
Linguistic expressions are containers for ideas
Communication is the process of sending the containers along a conduit to a recipient
Metaphors have a significant weakness
Metaphors do have a significant weakness. When the recipient receives the container, he or she opens the container and extracts the idea. Unfortunately, things aren’t so simple. Lakoff and Johnson observe that the recipient must interpret the container’s contents relative to a context. But the recipient’s choice of context determines how well the metaphor serves the sender’s purposes. A broad array of context choices gives recipients freedom to interpret the linguistic expressions. That freedom is what leads to what I’ve been calling unintended associations.
For example, even within the enterprise’s technology sectors, the technical debt metaphor can create communication problems between technologists and their managers. Technologists are uniformly averse to technical debt. It makes their work more expensive and annoying. It limits their ability to enhance the assets they work with. To management, by contrast, the term debt evokes the idea of financial debt, which is useful when employed responsibly. Non-technologist managers don’t personally experience the frustrations and annoyance technical debt often causes. They don’t experience the visceral revulsion technologists feel when dealing with technical debt. The technical debt metaphor therefore accounts somewhat for differences in perceptions of technical debt.
When making the case for technical debt retirement, technologists must provide estimates of the scale of the problem, and explain how it arose. Those who interpret the term technical debt against a background of financial experience are likely to find unsettling the technologists’ admissions of total or partial ignorance of what led to the problem. Just as unsettling are the technologists’ admitted difficulties in estimating precisely the cost of retiring the technical debt. Such questions have definite answers for financial debt. For technical debt, they don’t, even though the terms financial debt and technical debt share the word debt.
Last words
Debates have erupted in the engineering literature about the meaning of the term technical debt. Is incomplete work technical debt if there had not been a conscious decision to postpone it? Is work performed shoddily to meet a tight schedule technical debt? The real problem isn’t ambiguity in the term technical debt; rather, it is that the term is only a metaphor. With regard to the technical debt metaphor, the range of possible interpretations is somewhat wider than some would like. Nearly all metaphors are subject to such problems.
It is this problem—a problem of all metaphors—that accounts for much of the difficulty enterprises have when they try to control technical debt.
But let’s turn now to a closer examination of the two most important unintended associations—principal and interest. We begin with principal next time.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
Metaphors are powerful communication tools, but with the power comes a risk that by using a metaphor we might introduce confusion about what we’re trying to communicate, through a mechanism I call unintended association.
The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.
Ward Cunningham, who coined the technical debt metaphor. Photo (cc) Carrigg Photography.
The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.
Ward Cunningham coined the technical debt metaphor in the context of developing a software asset [Cunningham 1992] [Cunningham 2011]. He observed that when the development process leads to new learning, re-executing the development project—or parts of the project—could lead to better results. For this reason, among others, newly developed software assets can contain, embody, or depend upon artifacts that, in hindsight, the developers recognize they could now remove, or replace with more elegant, effective, or appropriate forms. Those new forms would then enhance maintainability, defensibility, and extensibility. To deploy the asset in pre-hindsight condition would entail an obligation to return to it to implement the improvements hindsight revealed. That obligation is Cunningham’s conception of the asset’s technical debt.
This phenomenon applies to more than newly developed software. It applies to all technological assets. And it applies to new development, maintenance, cyberdefense, and enhancement.
Why Cunningham used a metaphor
Cunningham knew that he was using a metaphor to describe his situation. That situation is common in technological development projects—both software and hardware. He probably chose the debt metaphor because his audience was financially sophisticated. He was exploiting their knowledge of financial instruments to convey a concept that, from his perspective, properly belongs in the software engineering body of knowledge. Such uses of metaphors aren’t unusual.
For example, the leverage concept, which originates in mechanics, captures the idea of mechanical advantage. You’re exploiting mechanical advantage when using a rigid bar, resting on a fulcrum, to move a heavy object. Suppose a lever is arranged so that A is the distance from the heavy weight to the fulcrum. And suppose that B is the distance from the fulcrum to the point of application of the force. Then one gains a force “multiplier” of B/A. This concept has been used in the world of finance, renamed financial leverage, to describe how borrowing can confer on the borrower a financial “force multiplier” by increasing the borrower’s total financial power. From there, the term leverage has spread into the general business vocabulary. Indeed, we can now say that, “Cunningham leveraged his boss’s understanding of financial instruments to convey a concept from software engineering.”
Metaphors are powerful—and dangerous
The danger of metaphors arises when we rely on the audience to apply their experience to interpret them. But since everyone’s experience is different, we cannot be certain how the audience might interpret the metaphor. And that’s where the trouble begins.
Before we explore the technical debt metaphor, we must investigate the structure of metaphors. This study will clarify how the technical debt metaphor has evolved and how it continues to evolve. Even more important, a study of metaphors helps us understand and anticipate the communication problems that arise from the technical debt metaphor. We’ll look at the structure of metaphors next time.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
Because metaphors compel our minds to accept the identification between source and target in toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not.
Metaphors are powerful communication tools, but with the power comes a risk that by using a metaphor we might introduce confusion about what we’re trying to communicate, through a mechanism I call unintended association.
Recall that our definition of the metaphorical principal of a technical debt (MPrin) at a given time is the cost of retiring the debt at that time. This cost can change with time. For example, due to ongoing maintenance and enhancements, the part of the asset in which the debt is embedded might become rather more complicated or constrained than it was earlier. That increased complexity can increase MPrin. This can happen even if the elements that comprise the debt itself have remained unchanged.
Moreover, the MPrin of any extensions to the asset in question, or to other assets, might also change. Those assets that use services provided by a subsystem manifesting the debt might also require alteration at debt retirement time. Untangling the debt from its surroundings, making necessary modifications, and testing the result, can be a delicate and complex process. The process might actually cost more at debt retirement time than whatever was saved when the debt was incurred.
An example of MPrin changing spontaneously
On the other hand, in some circumstances, the cost of retiring the debt can decrease over time. Consider the following fictitious example.
A high pressure sodium streetlight at dusk. Photo (cc) Famartin courtesy Wikimedia Commons.
Zion is a small city of 110,000 that’s struggling with two problems related to street lighting. Its current streetlights use High-Pressure Sodium (HPS) lights. HPS lights use almost twice as much energy as do the newer LED streetlights for the same level of illumination. Zion’s second problem is that the existing streetlights provide only one level of illumination throughout the city. This is causing a stream of complaints from many residents who have concerns about street lighting spilling onto their property at night. The bright light interferes with the sleep patterns of people and their pets.
Both of these problems have technical solutions that became available after the current HPS lights were installed. That’s why we can view them as arising from technical debt. Zion had investigated resolving the light pollution problem, but couldn’t find an affordable solution. Time passed. When LED streetlights became widely available, Zion investigated retiring its HPS lights. They found an LED lighting system that was dimmable on a block-by-block basis using a wireless control system. By retiring the technical debt associated with the HPS lights, Zion was able to afford retiring the technical debt associated with its un-dimmable lighting system.
Zion was able to afford to retire both forms of technical debt at once because of the way they interacted, even though retiring them one at a time would have been too expensive.
This example shows that the MPrin of a technical debt can vary widely, depending on the assets involved, and on what other debts they carry. Such variation is far more common in the realm of technical debt than it is in the world of financial debt.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
The “metaphorical principal” amount of a technical debt isn’t like the principal amount of a financial debt. In fact, the two are so different that policymakers can make real trouble for their organizations if they fail to take the differences into account.
Accomac Debtors’ Prison in Accomac, Virginia. After construction in 1783 it served as a jailer’s house. It served as a debtors’ prison from 1824 to 1849. Debtors could regain freedom by working to earn their keep and retire their debts, if they couldn’t pay them off. Today, bankruptcy laws facilitate property seizures to retire debts. Requiring an organization’s engineering functions to fund technical debt retirement through expense budgets is analogous to debtors’ prison. Photo (cc) Ser Amantio di Nicolao, courtesy Wikimedia Commons.In finance, the principal amount of a loan at the time of origination is the amount the creditor transfers to the debtor. Over time, as the debtor makes payments according to the terms of the loan agreement, the principal either remains constant or declines gradually. The terms of the loan agreement determine the actual remaining principal at any given time. This is the layperson’s definition. It’s the basis of the association people make with respect to the metaphorical principal amount of a technical debt.
Consequences of the layperson’s definition of debt
For technical debt management, the associations most people make with the debt concept are unfortunate. For example, using the term principal to refer to the metaphorical principal of a technical debt creates risk. The risk arises because the metaphorical principal behaves very differently from financial principal. Financial principal is typically constant or slowly declining. The metaphorical principal of technical debt can exhibit sudden and dramatic fluctuations. These confusions arise because of unintended associations of the technical debt metaphor.
Why I use the term MPrin instead of principal
We need a way to limit the risk of confusing the metaphorical principal of technical debt with the principal of a financial debt. The term metaphorical principal is so inconvenient that I prefer MPrin.
Most people don’t distinguish the initial principal amount of a technical debt and MPrin. But some have addressed it [Seaman 2013]. For example, Avgeriou et al. define the initial principal of a particular class of technical debt as the savings realized by incurring the debt [Avgeriou 2016]. They define the current principal as the resources required to deploy a different or better solution at the current time.
The initial principal concept of Avgeriou et al. is what I call in this blog the MPrin at the time the debt is incurred, or initial MPrin. Although the initial MPrin does have some value for decision makers at the time the debt is incurred, it’s most valuable when deciding whether to incur the debt, if, indeed, one has an opportunity to make such a decision. However, once the organization incurs the debt, the current MPrin at debt retirement time is what matters; initial MPrin becomes irrelevant.
Last words
Policymakers must keep clearly in mind that the MPrin of a given kind of technical debt is the total cost of retiring that debt, at the time it is retired, including all cost sources.
We’ll have a look at the policy implications of the properties of MPrin next time.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Shown here are servers like the ones that made this page available to you. Cybersecurity is concerned with defending servers like these, among others.
Policymakers have in mind the best interests of the entire enterprise. They need a definition of technical debt that’s neutral relative to its causes and manifestations. Defining technical debt in terms of what caused it or where it lies in the enterprise could compromise that necessary neutrality.
That neutrality is important because it enables us to recognize technical debt in whatever form it takes. For example, suppose enterprise policy assumes that technical debt lies only in software. And suppose that the root causes of some instances of technical debt are new threats in the cybersecurity environment that render obsolete our cyberdefenses. Then enterprise policy vis-à-vis technical debt is likely to be ineffective. It might lead to decision makers focusing too much attention on the software development process and too little attention on the cybersecurity and threat intelligence processes.
A definition that’s useful for guiding policy
Here’s a cause-neutral and manifestation-neutral definition of technical debt. It’s what I call the policymaker’s definition [Brenner 2017a]:
Technical debt is any technological element that contributes, through its existence or through its absence, to lower productivity or to a higher probability of defects during development, maintenance, or enhancement efforts, or which depresses velocity in some other way. It is therefore something we would like to revise, repair, replace, rewrite, create, or re-engineer for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it.
Extending the technical debt metaphor just a bit, people often talk about the principal and the interest charges associated with a technical debt. These ideas are analogous to the principal and interest charges associated with a financial debt. They’re convenient concepts, but the parallels between finance and technology aren’t real, and that’s where the trouble lies. Read more
An important extension beyond conventional definitions
There’s one other generalization contained in this definition of technical debt that differs from most other definitions. It’s in the phrase “or missing from.” Our policymaker’s definition doesn’t require that the technical debt item actually exist. That is, the absence of something can constitute technical debt. My favorite example is one due to Ken Pugh, who defines acceptance test debt as “…the nonexistence or nonautomation of acceptance tests.” [Pugh 2010] If we want to include all sources of reduced organizational agility or unnecessary operating expense arising from technical debt, our definition must also address non-existence issues like those Pugh has identified.
The definition above is workable for systems of all kinds. Consider two examples of “hardware”:
But the definition also applies to anything that takes a technological form, including business plans, legislation, procedures, and microprocessor designs—almost anything.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management.
Policy is one of those concepts that we rarely define explicitly because we believe we all agree on what it means. But in this context, we must take some care to be certain that we do in fact agree. In this blog, policy is the set of general guidelines people follow when making decisions and taking action.
Acela Express rounds a curve in Connecticut. Shown is the trailing power car of a southbound Acela Express and the front of a northbound Metro-North railcar.
Most definitions of technical debt require that the asset bearing the debt be software. From the policymaker’s perspective, this requirement is rather limiting. So for the purposes of this blog, I define technical debt as any property of a technological asset that we would like to revise, replace, or create, and which limits the ability of the enterprise to gain or maintain a market dominance. (See “A policymaker’s definition of technical debt.”)
An example from the railroad industry
In the United States, the highest-speed rail line is Acela Express. Acela travels in the northeast corridor between Boston and Washington, D.C. Parts of the right-of-way, track, and catenary it uses are from legacy lower-velocity applications. That’s why Acela trains cannot operate at their highest possible speed [Maloney 2000]. On the 231-mile section from Boston to New York’s Penn Station, Acela achieves an average speed of only 63 mph (101 km/h), even though the trains can operate safely on straight track at 150 mph (240 km/h). Yet, Acela still manages to capture a 54% share of the total air and rail market between these two cities
How Acela’s technical debt slows its trains
That 54% share might be higher still if not for technical debt. To compensate for centrifugal forces as Acela rounds curves, its passenger cars tilt their passenger spaces. The tilt enables the train to round the curves at higher speeds than would otherwise be comfortable for passengers. In effect, the cars “lean into” the curves, just as a running athlete leans when rounding a curve. Although the cars could tilt by as much as 6.8º, the adjacent set of tracks is too close to permit this without risk of collision with trains on those tracks. The maximum permissible tilt in this system is therefore 4.2º, which reduces the maximum speed consistent with passenger comfort that the trains can attain on curves. The technical debt in the tracks Acela uses thus prevents Acela from offering a service that would be more competitive with alternative transport modes, especially airlines.
Last words
In August 2016, Amtrak announced that it will be upgrading its trainsets and tracks to exploit new technologies, including active tilt technologies. All existing trainsets are due to be replaced in 2021-22.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.
This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue isn’t actually technical debt.
This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker’s view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.
Here’s an example of using the technical debt metaphor in civil engineering. In this case, it helps us understand why, in some cases, leaving some technical debt in place can be a better choice than retiring it.
The ghost ramps of highway I-695 in Somerville, Massachusetts. Photo (cc) Nick Allen.
Interstate 695 was planned in 1955 as an “inner belt” circumferential route in Boston and adjacent towns. When cancelled in 1971, construction had already begun. The cancellation thus resulted in an incomplete implementation.
Newly constructed roads and mass transit now use the rights of way that were the original paths for I-695. Some never-used structures remain to this day, including a “ghost ramp” in Somerville that would have connected I-695 to I-93. Because this ramp is a mere stub that begins on an elevated stretch of I-93 and ends in mid-air, barriers across the stub entrance prevent accidental use. The ghost ramp constitutes technical debt because it’s an incomplete implementation. Google satellite view
For safety reasons, regular inspections, maintenance, and insurance are necessary for the ghost ramp. But it provides no utility and serves no civic purpose. Because the cost of retiring this technical debt—namely, demolition costs—would likely exceed the present value of the lifetime costs of inspection, maintenance, and insurance, the ghost ramp remains.
Sometimes, the best way to deal with technical debt is to leave it in place.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.
This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue isn’t actually technical debt.
This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker’s view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.
Here’s an example of using the technical debt metaphor to think about how elements of a hardware asset can limit the ability of the enterprise to attain and maintain market leadership. The asset in this example is the trackbed of Amtrak’s Acela Express.
