For financial debts, creditors regularly inform debtors of interest charges, principal remaining, and other loan parameters. Laws usually require reports and explicit statements about interest charges. By contrast, for technical debt, MICs can be difficult to compute with useful precision, even if we know they’re accumulating. Many decision makers are actually unaware that MICs are accumulating. For an organization to appreciate the full financial consequences of carrying technical debt, everyone in the enterprise must appreciate the concept of MICs.
The heart of the problem
A stack of floppy disks. You don’t see many of these around much anymore. Very little of the software or hardware we use is as obsolete as these floppies. But much of it is obsolete, and it therefore comprises technical debt. It still works, but it’s slow and probably out of support life. On the basis of speed alone, the MICs it incurs can easily justify replacement. And some of it is vulnerable to cyberattack. One significant breach can ruin a brand.Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
The difficulty of measuring MICs arises from three sources. First, people whose productivity is most directly affected by technical debt—usually engineers—often have difficulty determining with precision the extent of the impact of technical debt on their efforts.
Second, many people are unaware of the impact technical debt has on their results. For example, if a product arrives late to market, we could compute the financial costs attributable to technical debt if we knew that technical debt is partially—or wholly—responsible for the delay. Too often, those who could perform such calculations aren’t sufficiently familiar with the concept of MICs. In any case, the data they would need for calculating a useful estimate is rarely available.
This brings us to the third source of difficulty.
The terrifying opportunity
Finally, a more insidious form of the consequences of technical debt is what we might call the terrifying opportunity. This situation arises when the organization rejects (or fails to recognize) a market opportunity because exploiting it would involve modifying an existing asset. The modification is terrifying because the asset harbors a heavy load of technical debt. The debt causes decision makers to assess that the probability of success is so low that the opportunity seems terrifying. They therefore reject the opportunity. Typically, terrifying opportunities would be exploitable if the debt-bearing assets didn’t exist at all. Then we would be starting fresh. But when opportunities require modifying assets that bear technical debt, commitment requires faith that we can address the technical debt successfully.
The sense of risk isn’t a reflection on the capabilities of the technical organization. Rather, it’s a result of the challenges involved in working with assets that bear high levels of technical debt. Given past performance of the technical organization relative to these debt-bearing assets, success can seem unlikely.
Computing the cost of a terrifying opportunity requires estimating the cost of not exploiting the opportunity, a difficult task in the best of circumstances. But whatever that cost is, it’s a form of MICs that we rarely recognize.
Last words
Building expertise in estimating MICs in all their forms is advantageous to any organization that seeks to make its technical debt more manageable. By making MICs visible, we can bring about better recognition of the cost of carrying technical debt, thereby providing an appropriate motivator for retiring technical debt.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
For financial debts, the interest charges associated with a unit of debt are (usually) the same for every unit of debt incurred under the same loan agreement. But for technical debt, the MICs associated with a given instance of a class of technical debt might differ substantially from any other instance of the same class of technical debt, even if those instances of technical debt were incurred at the same time as a result of a single decision or sequence of events.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
A rehabilitated Green Line car of the Massachusetts Bay Transit Authority. Trolley cars still travel on surface streets in and around Boston, in medians of divided roadways. Many streets still contain buried trolley tracks. They comprise a technical debt. MICs continue to accrue due to a near-continuous need to patch roadways. The need arises because of surface decomposition from the freeze-thaw cycle and small movements of buried tracks under traffic loads. A recent sewer upgrade project in Cambridge required removal of buried tracks to replace an old sewer line. This presented an opportunity to defer street surface maintenance (MICs) to take advantage of the surface rebuilding that was included in the sewer project. I don’t know whether the city actually exploited that opportunity.
Although rescheduling financial interest payments is possible only by special arrangement, or in bankruptcy, we can often defer or advance MICs on technical debt by rescheduling work that might incur them. For some kinds of technical debt, MICs accumulate only if we perform engineering work that involves that debt. This property is especially useful when we plan to retire an asset that bears technical debt. When we remove the asset from service, the technical debt it carries vanishes.
For most conventional financial debts, interest charges accumulate until we retire the debt. Interest charges might be zero for some time periods, but they’re never negative. Failure to meet the contractual payment schedule can result in penalties and additional interest charges.
MICs are different from financial interest charges
For technical debt, we can sometimes defer or advance MICs without penalty. We can arrange to temporarily nullify the MICs on a particular class of technical debt, or particular instances of that class. To do so, we need only reschedule a project or projects. This is possible when MICs accrue only if there is a need to perform work on the assets that carry the debt. In a given fiscal period, if we perform no work on those assets, the MICs can be zero. By scheduling projects accordingly, we can arrange for MICs to be zero.
There is one caveat. As I mentioned in “Debt contagion: how technical debt can create more technical debt,” as long as a particular class of technical debt remains in place, its associated MPrin might increase. Deferring retirement of a class of technical debt can be wise only if we can control its associated MPrin or if changes in its MPrin are acceptable.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
For financial debts, the interest charges associated with a unit of debt are (usually) the same for every unit of debt incurred under the same loan agreement. But for technical debt, the MICs associated with a given instance of a class of technical debt might differ substantially from any other instance of the same class of technical debt, even if those instances of technical debt were incurred at the same time as a result of a single decision or sequence of events.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
For financial debts, the interest charges associated with a unit of debt are (usually) the same for every unit of debt incurred under the same loan agreement. But for technical debt, the MICs associated with a given instance of a class of technical debt might differ from the MICs associated with any other instance of the same class of technical debt. They can differ even if we incurred those instances of technical debt at the same time. And they can differ even if they formed as results of a single decision or sequence of events. Unlike the transactions on a credit card, the interest charges can vary for instances of the same kind of technical debt.
Why MICs can differ from instance to instance
The I-35W Bridge collapse, day 4, Minneapolis, Minnesota, August 5, 2007. Underweight gusset plate design made the bridge vulnerable due to the increased static load from concrete road surfacing additions. And it was especially vulnerable due to the weight of construction equipment and supplies during a repair project that was then underway. But the root cause of the failure was that the bridge was “fracture critical.” It was vulnerable to collapse if any one of a set of critical bridge members failed. The 18,000 fracture critical bridges in the U.S. were built (or are being built) because they’re cheaper than are bridges that have zero fracture critical members [CBS News 2013]. Expedient shortcuts are among the most prolific generators of technical debt. For bridges, the MICs could include inspections, repairs, and temporary closures for inspections and repairs. Variations of design from bridge to bridge clearly could create variations in MICs from bridge to bridge. Photo by Kevin Rofidal, United States Coast Guard, courtesy Wikimedia Commons.For most financial debts, a single algorithm determines the interest charges for every unit of a particular class of debt. Following the technical debt metaphor, people tend to assume that the MICs on every instance of a particular class of technical debt are uniform across the entire class.
But in practice, uniformity assumptions with regard to MICs are generally invalid. Given two different instances of the same kind of technical debt, the MICs associated with modifying asset components in and around those two instances can differ significantly. For any given instance of a particular class of technical debt, MICs can depend on whether engineers must interact with that part of the asset. And when they do interact with a given asset component, MICs can also depend upon the transparency and condition of that asset component.
Two examples illustrating varying MICs
For example, an instance of technical debt might reside in a setting that relatively few local experts understand. The people who are capable of doing that work might be in high demand, or heavily committed, or expensive. Subsequent scheduling difficulty can lead to delays or service interruptions associated with completing the required work. That can result in lost revenue, which also contributes to MICs. Meanwhile, instances of the same kind of technical debt residing in other parts of the asset might be addressable by less expert staff. They might be in lesser demand, and less well compensated. Service interruptions might be shorter, and lost revenue less. The MICs associated with these two cases can therefore differ significantly.
As a second example, consider documentation deficits. Suppose an engineer needs documentation to determine how to proceed, and that documentation doesn’t exist. The engineer must then resort to alternatives that might be more time-consuming. He or she might read code or specifications, or interview colleagues. But for two instances of the same kind of technical debt, the need to refer to documentation can differ. The engineer might need documentation for one instance in one part of the asset, but not for another.
Last words
Another form of documentation deficit can be especially costly. Suppose engineers need documentation, and it does exist, but it’s out of date or incorrect. Those engineers might make costly, potentially irreversible errors when they undertake maintenance or extension activity. When testing uncovers the defects the engineers unwittingly introduced due to the defective documentation, the damage is less. But if testing doesn’t catch the defects, they might somehow find their way into production. If they do, the revenue or liability impact can be substantial. And the impact can vary from instance to instance of the technical debt in question. These effects are all forms of MICs.
So MICs can vary almost on an instance-by-instance basis. Or they might be constant across instances. It’s difficult to say. But the easy assumption—that MICs are the same for all instances of a given class of technical debt—the easy assumption is probably incorrect.
References
[CBS News 2013] CBS News and the Associated Press. “Thousands of U.S. bridges vulnerable to collapse,” May 25, 2013.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
Few senior management teams would seriously consider making decisions about financial instruments without carefully estimating their effects on revenue and expenses. Most enterprises support decision makers with an impressive array of tools, historical data, and skilled financial professionals. Yet few organizations invest at similar levels to support estimators of the MICs involved in undertaking engineering efforts. A similar dearth of resources affects those who estimate the effects on revenue due to carrying technical debt.
A composite satellite view of Antarctica. Composite by Dave Pape using NASA’s Blue Marble data set. Exploring unknown territory, as Roald Amundsen did in 1911-12, is far more difficult and riskier than exploring mapped territory. For this reason, managing technical debt is more successful when we have even minimal capability for estimating the MICs of carrying technical debt. Courtesy Wikimedia Commons.A resource shortage of this kind can have starkly negative effects. The inherent difficulties of projecting the effects of both carrying and retiring technical debt create uncertainties in project budgets and schedules.
MICs can fluctuate dramatically depending on a range of factors. These factors include:
The kind of work underway on the asset that carries the debt
How the debt affects customers and what they’re doing at any given time
The difficulty of researching engineering problems arising from the debt
The loss of revenue due to debt-related delays in reaching the market
A loss of sales due to semi-catastrophic failures in customer demonstrations
In short, MICs are often unpredictable [Allman 2012].
The state of the art
Most of the research into the effects of carrying or retiring technical debt has focused on engineering activity, and specifically, software engineering activity [MacCormack 2016] [Kamei 2016]. By comparison, research has been less intensive for effects on other activities—marketing, sales, regulatory compliance, to name a few. And in many cases, the effects of technical debt on these other activities are the most significant.
Consider first the effect of technical debt on enterprise expenses. The kind of maintenance and enhancement work performed on a set of assets bearing technical debt can determine the depressive effect on productivity. And declines in productivity directly affect MICs. In many cases, projecting future MICs associated with any given class of technical debt can be difficult. The difficulty arises because we might not know with sufficient certainty what projects will be active in the intermediate term or long term future, and what kind of work those projects will undertake. Even when we do know these things, the level of involvement with instances of particular classes of technical debt can be difficult to project enough certainty to be useful.
Turning to revenue, for most organizations, the picture is also bleak. Because we can’t retire some classes of technical debt incrementally, retirement projects can have significant impact on operations and revenue. Research in this area is even more limited than in the area of effects on productivity.
Last words
Projecting MICs with useful accuracy would be a valuable capability. Making MICs more predictable would require systematically gathering data and building expertise for projecting MICs for your enterprise. That problem is more tractable than the more general problem of projecting MICs absent specific knowledge of enterprise characteristics.
An enterprise-specific MICs projection capability could elevate the quality of decisions regarding resource allocation for projects of all kinds, including technical debt retirement projects. Policymakers can play an important advocacy role in establishing such a capability.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
A common assumption vis-à-vis technical debt is that we can model its productivity-depressing and velocity-reducing effects. We model them as the “interest” on the technical debt (MICs). And we assume that MICs are relatively constant over time. In practice, MICs can fluctuate dramatically. Those fluctuations provide planners valuable insight and flexibility, if they choose to use it. Unfortunately, most plans I have seen make the assumption that MICs are relatively stable.
An example of MICs behavior
30-year average fixed mortgage rates in the United States, 2012-2017, in %. Over this five-year period, rates did fluctuate. But they did so in a narrow range of from 3.3% to just over 4.5%. When we speak of “interest,” we evoke an impression of relative stability. This happens even when we’re speaking of technical debt. MICs for technical debt can vary from 0 to well above MPrin in any given time period. That’s one thing that makes the term “interest” so misleading in the context of technical debt. Data provided by U.S. Federal Reserve Bank of St. Louis [Federal Reserve 2017].As an example of this assumption is available in a paper by Buschmann [Buschmann 2011b]. He states that the longer we wait to retire technical debt in design and code, the larger the amount of interest. This presumes constant or non-negative MICs. That assumption that might be valid for some situations, but it isn’t universally applicable.
Consider a project that entails maintenance or extension of parts of the system that don’t manifest a specific class of technical debt. And suppose that the assets in question don’t depend on elements that do manifest that debt. Such a project is less likely to incur the MICs associated with that debt. So with respect to any particular class of technical debt, there might be time periods in which no projects incur MICs. During those periods, the interest accrued can be zero. In other time periods, the interest accrued on account of that same class of technical debt could be very high indeed.
A capacity for projecting MICs associated with a particular class of technical debt can be useful to planners as they work out schedules for maintenance projects, development projects, and technical debt retirement projects. Technical debt retirement projects are also subject to MICs, including from classes of technical debt other than the debt they’re retiring.
Analogous to the functioning of governance boards, a technical debt resources board could provide resources for evaluating assessments of likely MICs for maintenance projects, development projects, and technical debt retirement projects. Decision makers could use these assessments when they set priorities for these various efforts. I’ll say more about technical debt resources boards in future posts.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
Using the term interest to refer to the metaphorical interest charges of a technical debt is risky. The risk arises from confusing the properties of financial interest with the properties of the metaphorical interest charges on technical debt. Using an alternative term that makes the metaphor obvious can limit this risk. One such term is metaphorical interest charges, or for convenience, MICs.
Loose change. The MICs on technical debt accumulate in two ways: (a) as “loose change,” namely, small bits of lost time, delays, and depressed productivity; and (b) as major blows to enterprise vitality in the form of lost revenue, delayed revenue, and missed market opportunities. Hard to say which category does more damage.MICs aren’t interest charges in the financial sense. Rather, the MICs of a technical debt represent the total of reduced revenue, incidental opportunity costs, and increased costs of all kinds resulting from carrying that technical debt. (Actually, now that I think of it, MICs can include financial interest charges if we find it necessary to borrow money as a consequence of carrying technical debt.) Because the properties of MICs are very different from the properties of financial interest charges, we use the term MICs to avoid confusion with the term interest from the realm of finance.
What exactly are “metaphorical interest charges?”
Briefly, MICs are variable and often unpredictable [Allman 2012]. MICs differ from interest charges on financial debt for at least eight reasons. For any particular class of technical debt:
I examine each of these properties in more detail in the posts listed above.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
The common understanding of interest on financial debts doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor. Formulating sound technical debt policy depends on understanding the nature of the difference between interest on financial debt and the metaphorical interest charges associated with technical debt.
Credit cards. Revolving unsecured charge accounts are perhaps the most familiar form of financial debt. They do have one thing in common with technical debt: with either one, getting into debt over your head is easy.
Second only to the term debt, the term interest is perhaps the most common financial term in the technical debt literature. In the financial realm, interest charges are the cost of using money. Usually, we express interest charges as a percentage rate per unit time. By contrast, metaphorical interest charges (MICs) on technical debt work differently. Failure to fully appreciate that difference can create problems for organizations as they try to manage their technical debt.
The notion of interest is deep in our culture. We understand it well. But the way we understand it corresponds to fixed or slowly varying interest rates. This understanding biases our perception of technical debt.
The root of the problem
Because we’re so familiar with financial interest, we perceive the elements of technical debt as imposing a cost that’s a relatively stable fraction, per fiscal period, of the initial MPrin. This belief doesn’t correspond to the reality of technology-based systems, which are the targets of the technical debt metaphor.
MICs on technical debt differ from the interest on financial debt in two ways.
MICs depend strongly on whether and how the people of the enterprise interact with the assets bearing the technical debt.
The MICs on technical debt include the value of opportunities lost (opportunity costs). These losses are due to depressed productivity and reduced organizational agility.
Neither of these factors has a financial analog. In finance, interest charges depend solely on a mathematical formula involving the interest rate and principal.
Last words
In the next few posts, I’ll explore the properties of metaphorical interest charges. This investigation helps clarify how they differ from financial interest charges. It also clarifies how that difference contributes to difficulties in managing technical debt.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
The Principal Principle is that a focus on the metaphorical principal of a technical debt can be your undoing. Focus on the metaphorical interest charges. Drive them to Zero and keep them there.
Misunderstandings about the metaphorical interest charges on technical debt are costly. They prevent us from exploiting the properties of technical debt that reduce carrying costs and retirement costs. And the misunderstandings arise from the fact that the technical debt metaphor is only a metaphor—technical debt and financial debt are different.
The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of technical debt.
Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.
Rescheduling interest payments on financial debt is possible only by prearrangement or in bankruptcy, but MICs on technical debt can often be rescheduled by rescheduling work that might incur them. This is useful when we plan to retire assets bearing technical debt, because their technical debt vanishes.
Because the term technical debt is a metaphor, it causes us to think in ways that sometimes create barriers to managing technical debt responsibly. Like all metaphors, the technical debt metaphor carries with it unintended associations—attributes of the metaphor’s source that the listener or reader attributes to the metaphor’s target, even though the attributions aren’t intended by the metaphor’s author. For the technical debt metaphor, these unintended associations relate to the concepts of debtor, principal, and interest, and they can cause enterprise decision makers to arrive at erroneous decisions.
Because metaphors compel our minds to accept the identification between source and targetin toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not. This misidentification is an acceptable risk, if we understand the risk and if we manage it properly. Unfortunately, we rarely recognize that risk. And unrecognized risks usually remain unmitigated. A significant source of this risk is our inability to control which attributes of the metaphor’s source the reader or listener chooses to associate with the metaphor’s target. I call this phenomenon unintended association.
Interest and principal have unintended associations
University graduates celebrate commencement. Some, perhaps most, carry a burden of student loan debt in addition to their diplomas. Student loans, now familiar to many, act as a source for the technical debt metaphor. They are therefore also a source for its unintended associations.Two sets of unintended associations that frequently arise in the context of the technical debt metaphor relate to interest and principal. In the world of finance, we understand these concepts well. Unfortunately, our understanding from finance doesn’t fit well with the details of the corresponding properties of technical debt.
For example, van Haaster [van Haaster 2015] writes: “Financial debt has two well understood dimensions: the amount owing and its cost to repay over time, consequently when you take on financial debt, the total cost of that debt over time is either known or can be calculated.” Such beliefs about financial debt have consequences for our thinking about technical debt, because van Haaster’s statement is inapplicable to technical debt, for two reasons. First, the cost to repay a technical debt might not be well known. Second, the interest charges on technical debt might not be known, might not even be knowable, and often cannot be calculated [Falessi 2014]. These are just two examples of differences between financial debt and technical debt. We’ll explore these differences in some detail in coming posts.
How technical debt can be seen as evidence of mismanagement
A most significant unintended association is related to the concept of debt itself. Consider, for example, the social status of debtors in society. For many, excessive financial debt evokes images of profligate spending, laziness, and moral decay. These associations can hinder technology leaders within organizations as they urgently advocate for resources for technical debt management.
Because of unintended association, some decision makers outside the technology-oriented elements of the enterprise might regard technical debt as evidence of mismanagement. They might tend to attribute the cause of technical debt to professional malpractice by technologists. They see supportive evidence in the technology managers’ uncertainty about the size of the debt or how they acquired it. To the extent that nontechnical decision makers adopt this attitude, they are unlikely to support enterprise policy changes. They’re even less likely to support additional resources for technical debt management.
But the problems of the technical debt metaphor can be even more significant. The classic work of Lakoff and Johnson [Lakoff 1980] offers an explanation in terms of a metaphor (of course). In this metaphor:
Ideas are objects
Linguistic expressions are containers for ideas
Communication is the process of sending the containers along a conduit to a recipient
Metaphors have a significant weakness
Metaphors do have a significant weakness. When the recipient receives the container, he or she opens the container and extracts the idea. Unfortunately, things aren’t so simple. Lakoff and Johnson observe that the recipient must interpret the container’s contents relative to a context. But the recipient’s choice of context determines how well the metaphor serves the sender’s purposes. A broad array of context choices gives recipients freedom to interpret the linguistic expressions. That freedom is what leads to what I’ve been calling unintended associations.
For example, even within the enterprise’s technology sectors, the technical debt metaphor can create communication problems between technologists and their managers. Technologists are uniformly averse to technical debt. It makes their work more expensive and annoying. It limits their ability to enhance the assets they work with. To management, by contrast, the term debt evokes the idea of financial debt, which is useful when employed responsibly. Non-technologist managers don’t personally experience the frustrations and annoyance technical debt often causes. They don’t experience the visceral revulsion technologists feel when dealing with technical debt. The technical debt metaphor therefore accounts somewhat for differences in perceptions of technical debt.
When making the case for technical debt retirement, technologists must provide estimates of the scale of the problem, and explain how it arose. Those who interpret the term technical debt against a background of financial experience are likely to find unsettling the technologists’ admissions of total or partial ignorance of what led to the problem. Just as unsettling are the technologists’ admitted difficulties in estimating precisely the cost of retiring the technical debt. Such questions have definite answers for financial debt. For technical debt, they don’t, even though the terms financial debt and technical debt share the word debt.
Last words
Debates have erupted in the engineering literature about the meaning of the term technical debt. Is incomplete work technical debt if there had not been a conscious decision to postpone it? Is work performed shoddily to meet a tight schedule technical debt? The real problem isn’t ambiguity in the term technical debt; rather, it is that the term is only a metaphor. With regard to the technical debt metaphor, the range of possible interpretations is somewhat wider than some would like. Nearly all metaphors are subject to such problems.
It is this problem—a problem of all metaphors—that accounts for much of the difficulty enterprises have when they try to control technical debt.
But let’s turn now to a closer examination of the two most important unintended associations—principal and interest. We begin with principal next time.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
Metaphors are powerful communication tools, but with the power comes a risk that by using a metaphor we might introduce confusion about what we’re trying to communicate, through a mechanism I call unintended association.
The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.
Flooding from Hurricane Katrina in New Orleans, 2005. The forces of Nature can overtop or undermine any levee humans can build. So it is with technology. Organizational policy and politics can overcome or undermine any technology humans can devise to attain mastery over technical debt. To master technical debt, technology isn’t enough—we must also deal with policy and politics.
Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—should create significant savings and competitive advantages. These benefits arise from reductions in metaphorical interest charges (MICs) that result from retiring technical debt. But these benefits become available only if engineering capacity increases relative to the total debt-related workload. After the technical debt management program is in place, if the balance between engineering resources and debt-related workload becomes more favorable, then organizational effectiveness can improve. But if the balance becomes less favorable, as a result of reductions in engineering resources, organizational effectiveness won’t improve, even at lower levels of technical debt.
Unfortunately, some organizations adopt advanced technical debt management practices while reducing engineering capacity. If reductions are dramatic enough, engineering effectiveness is no better than it was before initiating the technical debt management program. The reason for this is that the engineering process isn’t the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves nontechnical causes in place. That’s why technological solutions to the technical debt management problem might not produce benefits in organizational effectiveness and agility.
The focus of technical debt research has been technology
The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has appeared [Izurieta 2017]. Effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has transcended debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].
Such developments might or might not have much impact on the limiting the effects of carrying technical debt. Given the necessary resources, engineering organizations could retire much of the technical debt now extant. That is, the will and the capacity to invest in debt retirement determines debt retirement rates. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.
Technological development isn’t enough
The open question is this: will technological developments alone give us control of the problem of technical debt? Perhaps not. Advancements in technical debt management do benefit organizations. But they could use that benefit to execute reductions in engineering staffing. If they do, they could divert savings to other parts of the enterprise. That would allow technical debt to remain at reduced levels that could still compromise the effectiveness of that reduced engineering staff.
For example, research has shown that schedule pressure contributes to technical debt formation and persistence. Suppose that the engineering groups of an organization become more adept at managing and preventing technical debt. Suppose further that the organization’s marketing and sales groups don’t improve their own intelligence and planning processes. Then Marketing might demand new capabilities with ever shorter timelines. That could lead to increased schedule pressure for the engineering groups. Then the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.
Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of nontechnical causes of technical debt to persist, and possibly even to grow.
This blog focuses on the nontechnical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Shown here are servers like the ones that made this page available to you. Cybersecurity is concerned with defending servers like these, among others.
Policymakers have in mind the best interests of the entire enterprise. They need a definition of technical debt that’s neutral relative to its causes and manifestations. Defining technical debt in terms of what caused it or where it lies in the enterprise could compromise that necessary neutrality.
That neutrality is important because it enables us to recognize technical debt in whatever form it takes. For example, suppose enterprise policy assumes that technical debt lies only in software. And suppose that the root causes of some instances of technical debt are new threats in the cybersecurity environment that render obsolete our cyberdefenses. Then enterprise policy vis-à-vis technical debt is likely to be ineffective. It might lead to decision makers focusing too much attention on the software development process and too little attention on the cybersecurity and threat intelligence processes.
A definition that’s useful for guiding policy
Here’s a cause-neutral and manifestation-neutral definition of technical debt. It’s what I call the policymaker’s definition [Brenner 2017a]:
Technical debt is any technological element that contributes, through its existence or through its absence, to lower productivity or to a higher probability of defects during development, maintenance, or enhancement efforts, or which depresses velocity in some other way. It is therefore something we would like to revise, repair, replace, rewrite, create, or re-engineer for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it.
Extending the technical debt metaphor just a bit, people often talk about the principal and the interest charges associated with a technical debt. These ideas are analogous to the principal and interest charges associated with a financial debt. They’re convenient concepts, but the parallels between finance and technology aren’t real, and that’s where the trouble lies. Read more
An important extension beyond conventional definitions
There’s one other generalization contained in this definition of technical debt that differs from most other definitions. It’s in the phrase “or missing from.” Our policymaker’s definition doesn’t require that the technical debt item actually exist. That is, the absence of something can constitute technical debt. My favorite example is one due to Ken Pugh, who defines acceptance test debt as “…the nonexistence or nonautomation of acceptance tests.” [Pugh 2010] If we want to include all sources of reduced organizational agility or unnecessary operating expense arising from technical debt, our definition must also address non-existence issues like those Pugh has identified.
The definition above is workable for systems of all kinds. Consider two examples of “hardware”:
But the definition also applies to anything that takes a technological form, including business plans, legislation, procedures, and microprocessor designs—almost anything.
References
[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.
[Falessi 2014] D. Falessi, Philippe Kruchten, Robert L. Nord, and Ipek Ozkaya. “Technical Debt at the Crossroads of Research and Practice: Report on the Fifth International Workshop on Managing Technical Debt,” ACM SIGSOFT Software Engineering Notes 39:2, 31-33, 2014.
[Federal Reserve 2017] Federal Reserve Bank of St. Louis. “30-Year Fixed Rate Mortgage Average in the United States (MORTGAGE30US).” Weekly time series.
[MacCormack 2016] Alan MacCormack and Daniel J. Sturtevant. “Technical debt and system architecture: The impact of coupling on defect-related activity,” The Journal of Systems and Software 120, 170–182, 2016.
[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10
As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management.
Policy is one of those concepts that we rarely define explicitly because we believe we all agree on what it means. But in this context, we must take some care to be certain that we do in fact agree. In this blog, policy is the set of general guidelines people follow when making decisions and taking action.
