Resources for policymakers concerned with managing technical debt
Category: Defining technical debt
Of the numerous definitions of technical debt found in the literature, few are suitable for use as a foundation for enterprise policy. In this category you’ll find examples of definitions of technical debt, along with commentary about their strengths and weaknesses as bases for technical debt management policy.
The MPrin of technical debt incurred as a consequence of a platform component upgrade depends on how we incur the debt. If we incur the debt by installing the upgrade, and then performing only some of the work needed as a consequence of the upgrade, then the MPrin is the total cost of performing the work we deferred. If we incur the debt by deferring the upgrade, then the conventional definition of the MPrin is the cost of the upgrade plus the cost of any work necessitated by the upgrade, but not performed.
An aisle in the stacks of a library. Some libraries are upgrading their book tagging systems from barcodes to RFID tags — what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day [Boss 2011]. It’s a big job.
In this latter instance, MPrin can increase over time if some of the work performed in the environment of the obsolete platform component, but subsequent to deferring the upgrade, must later be repeated after the upgrade is ultimately installed. This situation can be even worse if the work performed in the environment of the obsolete platform component fails after the upgrade, but the maintainers do not recognize the reason for the failure. In that case, an investigative effort is required first, to determine the cause of the failure. These additional costs are actually part of the debt retirement effort for the debt incurred by deferring the upgrade, but they’re usually accounted for — mistakenly — as routine operational expense.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
The MPrin of new technical debt associated with a development project is usually taken to be the difference between the cost of implementing it in a sustainable manner and the cost of simply making it work. The effort saved by choosing the latter over the former is typically identified as the initial MPrin of the technical debt.
The “basket bridge” of the Los Angeles Metro system. Constructed by the Metro Gold Line Foothill Extension Construction Authority, and opened in 2012, it carries the Los Angeles to Pasadena Metro Gold Line light rail across the eastbound lanes of the I-210 Freeway. The rail line provides transport service to a ridership that had been driving or using bus service. Although the bus and rail aren’t exact duplicates of each other, there is enough overlap that bus ridership dropped significantly after the rail line opened, which created a technical debt in the bus system that is being retired by reducing, rescheduling and re-routing bus service [Broverman 2017].
For example, in an enhancement project for an existing asset, after we finally achieve an operational capability, we might recognize that we have duplicated some functionality that already existed elsewhere in the asset. In such cases, the responsible, debt-free approach would be to first remove the duplication, then to modify the asset to use the previously existing approach, and finally to re-test the asset. By leaving the duplication in place, we save all that effort (and time), which many regard as the MPrin of a technical debt.
In a closely related example, we might recognize that the duplicated functionality in the newly developed portion of the asset is superior to the pre-existing form elsewhere in the asset. We’d like to remove the pre-existing form and replace instances of that form with instances of the newly developed functionality. But that work is clearly outside the scope of the new development, and it must await budgetary authority before it can be undertaken. Consequently, it becomes technical debt for the larger asset.
As time passes, and the enterprise undertakes other development projects, the implementations of previous projects can accumulate additional technical debt. The more obvious mechanisms by which this occurs include defect discovery, customer requests for enhancements, the need to enhance cyberdefenses in response to new threats, or changes in law or regulation.
An example of a less obvious process might be insights gained in marketing one product, which we shall call P1, that reveal an opportunity to introduce other related products, P2, P3, and P4, to form a suite. The latter products could employ some of the assets developed for or contained in P1, if they had been constructed slightly differently. The changes required in P1 therefore constitute technical debt, because we would have been able to develop P2, P3, and P4 much more rapidly if we had recognized the opportunity earlier. The P1 changes then become technical debt. And if we pursue P2, P3, or P4 without first retiring that debt, the debt probably expands, because it’s mirrored in the subsequent products.
New product (or service) developments often generate these situations.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Some examples might help to clarify the differences between the principal of financial debts and the MPrin of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Technical debt can arise as a result of:
Changes internally within the enterprise
External environmental changes that directly affect existing assets
Changes in the competitive environment
Insights and changes in perception that lead to changes in objectives
Existing technical debt
Deferring decisions about almost anything
By contrast, we incur financial debt only as a result of decisions to incur financial debt.
The examples sketched below illustrate some of these phenomena. They’re described more completely described posts indicated.
Development projects
This example illustrates how technical debt can develop as a result of unanticipated insight regarding a marketing opportunity for a new product line. It shows how technical debt can arise independent of any decision made within the enterprise, and without any changes to assets of any kind. More: “MPrin in a development project”
Platform component upgrades
We’ve already provided an example of technical debt arising from a platform upgrade. In this example, we show how deferring a platform upgrade creates new complications not present in the previous example, by illustrating how total MPrin can increase after the debt is incurred. More: “MPrin in platform component upgrades”
Standards or regulatory changes
Changes in standards or regulations, whether internal, industry-wide, or governmental, can create technical debt. In some cases, the enterprise might not even be aware of the new debt. More: “MPrin when standards or regulations change”
Missing or incompletely implemented capability
When a capability is incompletely implemented, it’s clear that the part left undone constitutes technical debt. What is less clear is what happens when the capability implementation is halted or rescinded. Trying to avoid new technical debt can actually be the cause of new technical debt, albeit of a different kind. More: “MPrin for missing or incompletely implemented capability”
Whether or not any similar scenarios have happened in your organization, these discussions are helpful for gaining insight into what kinds of technical debt policies can assist your organization in managing its technical debt. Let me know if you have experience with situations in which problems can be traced, even if only in part, to treating technical debt as if it were financial debt.
Because the term technical debt is a metaphor, it causes us to think in ways that sometimes create barriers to managing technical debt responsibly. Like all metaphors, the technical debt metaphor carries with it unintended associations — attributes of the metaphor’s source that the listener or reader attributes to the metaphor’s target, even though the attributions are not intended by the metaphor’s author. For the technical debt metaphor, these unintended associations relate to the concepts of debtor, principal, and interest, and they can cause enterprise decision-makers to arrive at erroneous decisions.
Because metaphors compel our minds to accept the identification between source and targetin toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not. This misidentification is an acceptable risk, if it is properly managed and understood. Unfortunately, that risk is often unrecognized, and when it is unrecognized it remains unmitigated. A significant source of this risk is our inability to control which attributes of the metaphor’s source the reader or listener chooses to associate with the metaphor’s target. It is this phenomenon that I call unintended association.
University graduates celebrate commencement. Some, perhaps most, carry a burden of student loan debt in addition to their diplomas. Student loans, now familiar to many, act as a source for the technical debt metaphor. They are therefore also a source for its unintended associations.
Two sets of unintended associations that frequently arise in the context of the technical debt metaphor relate to interest and principal, two concepts we understand well in the realm of finance. Unfortunately, our understanding from finance does not fit well with the details of the corresponding properties of technical debt. For example, van Haaster [van Haaster 2015] writes: “Financial debt has two well understood dimensions: the amount owing and its cost to repay over time, consequently when you take on financial debt, the total cost of that debt over time is either known or can be calculated.” Such beliefs about financial debt have consequences for our thinking about technical debt, because van Haaster’s statement is inapplicable to technical debt, for two reasons. First, the cost to repay a technical debt might not be well known. Second, the interest charges on technical debt might not be known, might not even be knowable, and often cannot be calculated [Falessi 2014]. These are just two examples of differences between financial debt and technical debt. We shall explore these differences in some detail in the next three chapters.
A most significant unintended association is that related to the concept of debt itself. Consider, for example, the social status of debtors in society. For many, excessive financial debt evokes images of profligate spending, laziness, and moral decay. These associations can hinder technology leaders within organizations as they urgently advocate for resources for technical debt management.
Because of unintended association, some decision makers outside the technology-oriented elements of the enterprise might regard technical debt as evidence of mismanagement. They might tend to attribute the cause of technical debt to professional malpractice by technology managers. They see supportive evidence in the technology managers’ uncertainty about the size of the debt or how they acquired the debt. To the extent that non-technical decision makers adopt this attitude, they are unlikely to support enterprise policy changes. They are even less likely to support additional resources for technical debt management.
But the problems of the technical debt metaphor can be even more significant. The issue is explained in a classic work of Lakoff and Johnson [Lakoff 1980] in terms of a metaphor (of course). In this metaphor:
Ideas are objects
Linguistic expressions are containers for ideas
Communication is the process of sending the containers along a conduit to a recipient
Metaphors do have a significant weakness. When the recipient receives the container, he or she opens the container and extracts the idea, completing the communication. Unfortunately things are not so simple. Lakoff and Johnson observe that the recipient must interpret the linguistic expressions of the container relative to a context. Because the choice of context is left to the recipient, the breadth of choices possible can determine how well the metaphor serves the sender’s purposes. A broad array of possible context choices gives recipients relative freedom to interpret the linguistic expressions. That freedom is what leads to what I have been calling unintended associations.
For example, even within the technology sectors of the enterprise, the technical debt metaphor can create communication problems between technologists and their managers. To technologists, technical debt is unequivocally disfavored. It makes their work more expensive and more annoying, and it limits their ability to enhance the assets for which they are responsible. To management, by contrast, the term debt evokes the idea of financial debt, which is a useful tool when employed responsibly. Managers do not personally experience the frustrations and annoyance that technical debt often causes. They do not experience the visceral revulsion that technologists feel when contemplating instances of technical debt. The differences in degree of urgency perceived by managers and technologists are therefore due, in part, to the technical debt metaphor and the use of the word debt.
Moreover, when making the case for technical debt retirement, technologists must provide estimates of the scale of the problem, and explain how it arose. Those who interpret the term technical debt against a background of financial experience are likely to be troubled by the technologists’ admissions of total or partial ignorance of what led to the problem, or by their admitted difficulties in providing precise estimates of the cost of retiring the technical debt. Such questions have definite answers for financial debt. For technical debt, they do not, even though the terms financial debt and technical debt share the word debt.
Debates have erupted in the engineering literature about the meaning of the term technical debt. Is incomplete work technical debt if there had not been a conscious decision to postpone it? Is work performed shoddily to meet a tight schedule technical debt? The real problem is not ambiguity in the term technical debt; rather, it is that the term is only a metaphor. With regard to the technical debt metaphor, the range of possible interpretations is somewhat wider than some would like. Nearly all metaphors are subject to such problems.
It is this problem — a problem of all metaphors — that accounts for much of the difficulty enterprises have when they try to control technical debt.
But let’s turn now to a closer examination of the two most important unintended associations — principal and interest. We begin with principal next time.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
Metaphors are powerful communication tools, but with the power comes a risk that by using a metaphor we might introduce confusion about what we’re trying to communicate, through a mechanism I call unintended association.
The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.
The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.
Ward Cunningham, who coined the technical debt metaphor. Photo (cc) Carrigg Photography.
Ward Cunningham coined the technical debt metaphor in the context of developing a software asset [Cunningham 1992] [Cunningham 2011]. He observed that when the development process leads to new learning, re-executing the development project—or parts of the project—could lead to better results. For this reason, among others, newly developed software assets can contain, embody, or depend upon artifacts that, in hindsight, the developers recognize could be removed altogether, or could be replaced by more elegant, effective, or appropriate forms that can enhance maintainability, defensibility, and extensibility. To deploy the asset in pre-hindsight condition would entail an obligation to return to it in the future to implement the improvements revealed by that hindsight. That obligation is Cunningham’s conception of the asset’s technical debt.
This phenomenon is not restricted to newly developed software. It applies to all technological assets. And it applies to new development, maintenance, cyberdefense, and enhancement.
Cunningham was aware that he was using a metaphor to describe his situation, which is a common one in technological development projects—both software and hardware. He probably chose the debt metaphor because his audience was financially sophisticated. He was exploiting their knowledge of financial instruments to convey a concept that, from his perspective, properly belongs in the software engineering body of knowledge. Such uses of metaphors are not unusual.
For example, the concept of leverage, which originates in mechanics, captures the idea of mechanical advantage gained when one employs a rigid bar, resting on a fulcrum, to move a heavy or fixed object. When a lever is arranged so that A is the distance from the heavy weight to the fulcrum, and B is the distance from the fulcrum to the point of application of the force, one gains a force “multiplier” of B/A. This concept has been used in the world of finance, renamed financial leverage, to describe how borrowing can confer on the borrower a financial “force multiplier” by increasing the borrower’s total financial power. From there, the term leverage has spread into the general vocabulary of business. Indeed, we can now say that, “Cunningham leveraged his boss’s understanding of financial instruments to convey a concept that properly belongs in the software engineering body of knowledge.”
Metaphors are powerful.
And they are also dangerous. The danger arises when we rely on the audience to apply their experience to interpret the metaphor in the way we intend. But since the experience of every individual is different, we cannot be certain how the audience might interpret the metaphor. And that is where the trouble begins.
Before we undertake our exploration of the technical debt metaphor, we must investigate the structure of metaphors. This study will help us understand how the technical debt metaphor has evolved and how it continues to evolve. Even more important, a study of metaphors will help us understand and anticipate the communication problems that arise from the fact that the term technical debt is a metaphor. We’ll look at the structure of metaphors next time.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
Because metaphors compel our minds to accept the identification between source and target in toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not.
Metaphors are powerful communication tools, but with the power comes a risk that by using a metaphor we might introduce confusion about what we’re trying to communicate, through a mechanism I call unintended association.
The MPrin of technical debt left in place can increase or decrease, even if the artifacts that comprise the technical debt don’t change. This variability can be exploited to advantage in some rare cases. Most often, though, the MPrin increases with time. That’s why it’s risky to leave a technical debt in place under the assumption that its MPrin is fixed.
Recall that our definition of the metaphorical principal of a technical debt at a given time is the cost of retiring the debt at that time. This cost can change with time. For example, due to subsequent maintenance and enhancements, the part of the asset in which the debt is embedded might become rather more complicated or constrained than it was earlier, even if the elements that comprise the debt itself have remained unchanged. Moreover, any extensions to the asset in question, or to other assets, that use services provided by subsystem manifesting the debt might also require alteration at debt retirement time. Untangling the debt from its surroundings, making necessary modifications, and testing the result, can be a delicate and complex process that actually costs more at debt retirement time than whatever was saved when the debt was incurred.
On the other hand, in some circumstances, the cost of retiring the debt can decrease over time. Consider the following fictitious example.
A high pressure sodium streetlight at dusk. Photo (cc) Famartin courtesy Wikimedia Commons.
Zion is a small city of 110,000 that’s struggling with two problems related to street lighting. Its current streetlights use High-Pressure Sodium (HPS) lights, which use almost twice as much energy as do the newer LED streetlights for the same level of illumination. Zion’s second problem is that the existing streetlights provide only one level of illumination throughout the city. This is causing a stream of complaints from many residents who have concerns about street lighting spilling onto their property at night. The bright light interferes with the sleep patterns of people and their pets.
Because both of these problems have technical solutions that became available after the current HPS lights were installed, they can be viewed as arising from technical debt. Zion had investigated resolving the light pollution problem, but could not find a solution it could afford. Time passed. When LED street lights became widely available, Zion investigated retiring its HPS lights, and found an LED lighting system that was dimmable on a block-by-block basis using a wireless control system. By retiring the technical debt associated with the HPS lights, Zion was able to afford retiring the technical debt associated with its one-size-fits-all un-dimmable lighting system.
Zion was able to afford to retire both forms of technical debt at once because of the way they interacted, even though retiring them one at a time would have been too expensive.
This example shows that the MPrin of a technical debt can vary widely, depending on the assets involved, and on what other debts they carry. Such variation is far more common in the realm of technical debt than it is in the world of financial debt.
References
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
The “principal” amount of a technical debt is not like the principal amount of a financial debt. In fact, the two are so different that policymakers can make real trouble for their organizations if they fail to take the differences into account.
Accomac Debtors’ Prison in Accomac, Virginia. Built in 1783 as a jailer’s house, it served as a debtors’ prison from 1824 to 1849. Debtors imprisoned there were required to work to earn their keep and to retire their debt, if they could not find other means to do so. Today, bankruptcy laws facilitate seizure of property to retire debts, a much more enlightened approach. Requiring the product or service engineering functions to fund technical debt retirement efforts through expense budgets is analogous to debtors’ prison. A more enlightened approach is needed. Photo (cc) Ser Amantio di Nicolao, courtesy Wikimedia Commons.
In the field of finance, the principal amount of a loan at the time of origination is the amount that was borrowed. Over time, as the debtor makes payments according to the terms of typical loan agreements, the principal either remains constant, in which case it’s repaid in a lump sum at a specified date, or it declines gradually, in increments, with each payment. This is the widely accepted layperson’s definition, and it’s the basis of the association people make with respect to the metaphorical principal amount of a technical debt.
That’s unfortunate. Because the metaphorical principal amounts of most technical debts behave very differently from the principal amounts of financial debts, using the term principal to refer to the metaphorical principal associated with a given kind of technical debt is risky. The risk arises from confusing financial principal, which is typically fixed or slowly declining, with the metaphorical principal of technical debt, which can exhibit sudden and dramatic fluctuations. These confusions arise because of unintended associations of the technical debt metaphor.
Using an alternative term that makes the metaphor obvious can limit this risk. One such term is metaphorical principal, or for convenience, MPrin.
Although the distinction between the initial principal amount of a technical debt and MPrin is not universally recognized [Seaman 2013], some have addressed it. For example, Avgeriou et al. define the initial principal of a particular class of technical debt as the savings realized by incurring the debt, and the current principal as the resources required to deploy a different or better solution now [Avgeriou 2016].
The initial principal concept of Avgeriou et al. is what we call in this blog the MPrin at the time the debt is incurred, or initial MPrin. Although the initial MPrin does have some value for decision makers at the time the debt is incurred, it’s most valuable when deciding whether or not to incur the debt, if, indeed, one has an opportunity to make such a decision. However, once the debt has been incurred, the current MPrin is what matters; initial MPrin becomes irrelevant.
Policymakers must keep clearly in mind that the MPrin of a given kind of technical debt is the total cost of retiring that debt, at the time it is retired, including all cost sources.
We’ll have a look at the policy implications of the properties of MPrin next time.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Policymakers have in mind the best interests of the entire enterprise. They need a definition of technical debt that is neutral relative to its causes and manifestations, because defining technical debt in terms of what caused it or where it lies in the enterprise could compromise that necessary neutrality.
Servers like the ones that made this page available to you. Cybersecurity is concerned with defending servers like these, among others.
For example, if enterprise policy assumes that technical debt is something that lies only in software, and if the root causes of some instances of technical debt are new and developing threats in the cybersecurity environment, then enterprise policy vis-à-vis technical debt is likely to be ineffective. It might lead to decision makers focusing too much attention on the software development process and too little attention on the cybersecurity and threat intelligence processes.
Here’s a cause-neutral and manifestation-neutral definition of technical debt, what I call the policymaker’s definition [Brenner 2017a]:
Technical debt is any technological element that contributes, through its existence or through its absence, to lower productivity or to a higher probability of defects during development, maintenance, or enhancement efforts, or which depresses velocity in some other way, and which we would therefore like to revise, repair, replace, rewrite, create, or re-engineer for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it.
Extending the technical debt metaphor just a bit, people often talk about the principal and the interest charges associated with a technical debt, by analogy to the principal and interest charges associated with a financial debt. They’re convenient concepts, but the parallels between finance and technology aren’t real, and that’s where the trouble lies. Read more
There’s one other generalization contained in this definition of technical debt that differs from most other definitions. It’s in the phrase “or missing from.” Our policymaker’s definition doesn’t require that the technical debt item actually exist. That is, the absence of something can constitute technical debt. My favorite example of this was put forward by Ken Pugh, who defines acceptance test debt as “…the nonexistence or nonautomation of acceptance tests.” [Pugh 2010] If we seek to encompass all sources of reduced organizational responsiveness or unnecessary operating expense arising from technical debt, the definition of technical debt that we require must also address non-existence issues such as those identified by Pugh.
The definition above is workable for systems of all kinds. Consider two examples of “hardware”:
But the definition also applies to anything that is manifested in technological forms, including business plans, legislation, procedures, and microprocessor designs—almost anything.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management.
Policy is one of those concepts that we rarely define explicitly because we believe we all agree on what it means. But in this context, we must take some care to be certain that we do in fact agree. In this blog, policy is the set of general guidelines people follow when making decisions and taking action.
Acela Express rounds a curve in Connecticut. Shown is the trailing power car of a southbound Acela Express and the front of a northbound Metro-North railcar.
Most definitions of technical debt require that the asset bearing the debt be software. From the policymaker’s perspective, this requirement is rather limiting. So for the purposes of this blog, I define technical debt as any property of a technological asset that we would like to revise, replace, or create, and which limits the ability of the enterprise to gain or maintain a dominant market leadership position.
Consider an example from the railroad industry. In the United States, the highest-speed rail line is Acela Express, which travels in the northeast corridor between Boston and Washington, D.C. Parts of the right-of-way, track, and catenary it uses were not originally designed for this application, and therefore trains cannot operate at their highest possible speed [Maloney 2000]. On the 231-mile section from Boston to New York’s Penn Station, Acela achieves an average speed of only 63 mph (101 km/h), even though the trains can operate safely on straight track at 150 mph (240 km/h). Yet, Acela still manages to capture a 54% share of the total air and rail market between these two cities.
That 54% share might be higher still if not for technical debt. To compensate for centrifugal forces as Acela Express rounds curves, its passenger cars are able to tilt the passenger spaces to enable the train to round the curves at higher speeds than would otherwise be comfortable for passengers. In effect, the cars “lean into” the curves, just as a running athlete leans when making a sudden change of direction. Although the cars were designed to be able to tilt by as much as 6.8º, the adjacent set of tracks is too close to permit this without risk of collision with trains on those tracks. The maximum tilt is therefore set at 4.2º, which, in turn, reduces the maximum speed consistent with passenger comfort that the trains can attain on curves. The technical debt manifested in the tracks Acela Express uses thus prevents it from offering a service that would be more competitive with alternative transport modes, especially airlines.
Note: In August 2016, Amtrak announced that it will be upgrading its trainsets and tracks to exploit new technologies, including active tilt technologies. All existing trainsets are due to be replaced in 2021-22.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.
This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue is not actually technical debt.
This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker's view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.
Here’s an example of using the technical debt metaphor in civil engineering. In this case, it helps us understand why, in some cases, leaving some technical debt in place can be a better choice than retiring it.
The ghost ramps of highway I-695 in Somerville, Massachusetts. Photo (cc) Nick Allen.
Interstate 695 was planned in 1955 as an “inner belt” circumferential route in Boston and adjacent towns. When it was cancelled in 1971, construction had already begun. Rights of way that had been cleared have since been reused for roads and mass transit, but some never-used structures remain to this day, including a “ghost ramp” in Somerville that would have connected I-695 to I-93. This ramp, which is a mere stub that begins on an elevated stretch of I-93 and ends in mid-air, and which is blocked off to prevent use, constitutes technical debt in the form of incomplete implementation. Google satellite view
For safety reasons, the ghost ramp must be regularly inspected, maintained, and insured, but it provides no utility and it is not used for any civic purpose. Because the cost of retiring this technical debt—namely, demolition costs—would likely exceed the present value of the lifetime costs of inspection, maintenance, and insurance, the ghost ramp remains.
Sometimes, the best way to deal with technical debt is to leave it in place.
References
[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.
[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.
Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.
Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.
This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.
This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue is not actually technical debt.
This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker's view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.
Here’s an example of using the technical debt metaphor to think about how elements of a hardware asset can limit the ability of the enterprise to attain and maintain market leadership. The asset in this example is the trackbed of Amtrak’s Acela Express.
