When we first set out to plan a large technical debt retirement project (DRP), a question that arises very early in the planning process is this: Which assets are carrying the kind of technical debt we want to retire? And a second question is: Which operations will be affected—and when—by the debt retirement work? Although these questions are clear, and easily expressed, the answers might not be. And the answers are important. So where is the technical debt?
Determining which of the enterprise’s many technological assets might be carrying the Technical Debt In Question (TDIQ) can be a complex exercise in itself, because inspecting the asset might be necessary. Inspection might require temporarily suspending operations, or determining windows of time during which inspection can be performed safely and without interfering with operations. Further, inspection might require knowledge of the asset that the DRP team doesn’t possess. Moreover, access to the asset might be restricted in some way. In these cases, staff from the unit responsible for the asset must be available to assist with the inspection.
Although asset inspection might be necessary or preferable, it might not be sufficient for determining which assets are carrying the TDIQ. This is easy to understand for physical assets, like, say, determining the release version of the firmware of the hydraulic controller electronics of a tunnel boring machine. But asset inspection might also be insufficient for purely software assets. For determining the presence of the TDIQ in software assets, reading source code might not be sufficient or efficient. It might be easier, faster, and more accurate to operate the asset under special conditions. For example, an inspector might want to provide specific inputs to the asset and then examine its responses. As a second example, we might use automation assistance to examine the internal structure of the asset, searching for instances of the TDIQ. And as with other assets, the assistance of the staff of the unit responsible for the asset might be necessary for the inspection.
Which enterprise operations depend on debt-bearing assets?
Knowing which assets bear the TDIQ is useful to the DRP team as it plans the work to retire the TDIQ. But part of that plan could include service disruptions. If so, it’s also necessary to determine how those disruptions might affect operations, to enable the team to control the effects of the disruptions, and negotiate with affected parties. Thus for each asset that bears the TDIQ, we need to determine what operations would be affected if the asset is removed from service temporarily.
Observing actual operations in conditions in which the asset is out of service in whole or in part might be the only economical way to discover which enterprise functions depend on the assets that carry the TDIQ. Other techniques include examining historical data such as trouble reports and outstanding defect lists, and correlating them across multiple asset histories and operations histories.
In some cases, these investigations produce results that have a limited validity lifetime, owing to ongoing evolution of the debt-bearing assets and the assets that interact with them. For that reason, the actual work of retiring the TDIQ must begin as soon as possible after the inventory is complete, and possibly even before that. This suggests that the size of the DRP team is a critical success factor, because size enables the team to complete the inventory inspections rapidly, before the end of the validity lifetime of the team’s research results.
Managing teams of great size is a notoriously difficult problem. For this reason, delegating some of the DRP research effort directly to the business units that own the assets in question can provide the labor hours and expertise needed for the research. In this way, the DRP can deploy a team-of-teams structure, known as a Multi-Team System (MTS) [Mathieu 2001] [Marks 2005]. The DRP team can then bring to bear a large force in a way that renders the overall MTS manageable.
References
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
In deciding whether or not to undertake technical debt retirement projects, organizations are at risk of making inappropriate decisions because of a synergy between the reification error and confirmation bias. Together, these two errors of thought create conditions that make committing appropriate levels of resources difficult. And in those cases in which resources are committed, there is a tendency to underestimate costs, which can lead to an elevated incidence of failures of technical debt retirement projects.
The reification error and confirmation bias
As explained elsewhere in this blog, the reification error is an error of reasoning in which we treat an abstraction such as technical debt as if it were a real, concrete, physical thing, which, of course, it is not. (See “Metrics for technical debt management: the basics”)
The reification error might be responsible, in part, for a widely used management practice that often appears in the exploratory stages of undertaking projects. Let’s start with an illustration from the physical world.
A feedback loop that now provides budgetary control in most organizations.
In the physical world, when we want cherries, we go to the produce section of the market and check the price per pound or kilo. Then we decide how many pounds or kilos we want. If the price is high, we might decide that fewer cherries will suffice. If the price is low, we might purchase more cherries. We have in mind a total cost target, and we adjust the weight of the cherries to meet the target, given the price. In the physical world, we can often adjust what we purchase to match our ability to pay for it.
Retiring technical debt doesn’t work like that, in part, because technical debt is an abstraction. But we try anyway; here’s how it goes. Management decides to retire a particular class of technical debt, and asks an engineer to work up an estimate of the cost. Sometimes Management reveals the target they have in mind if they have one; sometimes not. The estimate comes back as Total ± Uncertainty. Management decides that’s too high—or the Uncertainty is too great—and asks the engineer to find a way to do it for less, with less Uncertainty, maybe by being clever or doing less.
Management—the “customer” in this scenario—makes this request, in part, based on the belief that it’s possible to adjust the work to meet a (possibly unstated) target, in analogy to buying cherries in the produce department. That thinking is an example of the reification error. In this dynamic, we rarely take into account the fact that retiring technical debt isn’t exactly like buying cherries.
How confirmation bias affects engineering estimates
Now back to the interaction between Management and estimator. The engineer now suspects that Management does have a target in mind. Some engineers might ask what the target is. Some don’t. In any case, the engineer comes back with a lower estimate, which might still be too high. This process repeats until either Management decides against retiring the debt, or accepts the lowest Total ± Uncertainty, hoping for a final cost that isn’t too high.
In adjusting their estimates, engineers have a conflict of interest that can compromise their objectivity through the action of confirmation bias. In the case of technical debt retirement efforts, engineers are usually highly motivated to gain Management approval of the project, because the technical debt in question depresses engineering productivity and induces frustration. And since engineers typically sense that Management approval of the project is contingent on finding an estimate that’s low enough, the engineers have a preconception. That is, they have an incentive to convince themselves that the budget and schedule adjustments they make are reasonable. Because of the confirmation bias, they tend to seek justifications for the belief that lowering costs and tightening schedules is reasonable, while they tend to avoid seeking justifications for believing that their adjustments might not be feasible. That’s the confirmation bias in action.
How synergy between the reification error and confirmation bias comes about
So, because of the reification error, Management tends to believe that the work needed to retire technical debt of a particular kind is more adjustable than it actually is. And because of confirmation bias, engineers tend to believe that they can do the work for a cost and within a schedule that Management is willing to permit. Too often, the synergy between the two errors of thinking provides a foundation for disaster.
Why this synergy creates conditions for disaster in technical debt retirement projects
Management usually interprets estimates as commitments. Engineers do not equate estimates with commitments. Management usually forgets or ignores the upside Uncertainty. So typically, when Management finally accepts an estimate, the engineering team finds that it has made a commitment to deliver the work for the cost Total, with zero upside Uncertainty. Few engineering teams are actually asked to make an explicit commitment to perform the work for a cost Total with zero upside Uncertainty. An analogous problem occurs with schedule.
By ignoring the Uncertainty, Management (the buyer) transfers the uncertainty risk to the project team. That strategy might work to some extent with conventional development or maintenance projects, where we can adjust scope and risk before the work begins. But for technical debt retirement projects, this practice creates problems for two reasons.
Adjusting the scope of debt retirement projects is difficult
First, with technical debt retirement we’re less able to adjust scope. To retire a class of technical debt, we must retire it in toto. If we retire only some portion of a class of technical debt, we would leave the asset in a mixed state that can actually increase MICs. So it’s usually best to retire the entirety of any class of technical debt, so as to leave the asset in a uniform state.
Debt retirement efforts are notoriously unpredictable
Second, the work involved in retiring a particular class of technical debt is more difficult to predict than is the work involved in more conventional projects. (See “Useful projections of MPrin might not be attainable”) Often, we must work with older assets, or older portions of younger assets. The people who built them aren’t always available, and documentation can be sparse or unreliable. Moreover, it’s notoriously difficult to predict with accuracy when affected assets must be temporarily withdrawn from production—and for how long—to support the technical debt retirement effort. Revenue stream interruptions, which can be significant portions of total costs, can be difficult to schedule or predict. Thus, technical debt retirement projects tend to be riskier than other kinds of projects. They have wider uncertainty bands. Ignoring the Uncertainty, or trying to transfer responsibility for it to the project team, is foolhardy.
A strategy for reducing the effects of this synergy
To intervene in the dynamic between the consequences of the reification error and the consequences of confirmation bias, we must find a way to limit how their consequences can interact. That will curtail the ability of one phenomenon to reinforce the other. This task is well suited for application of Donella Meadows’ concept of leverage points [Meadows 1999], which I discussed in an earlier post, “Leverage points for technical debt management.”
In that post, I summarized Meadows’ idea that to alter the behavior of a complex system, one can intervene at one or more of 12 categories of leverage points. These are elements in the system that govern the behavior of the people and institutions that comprise the system. In that post, I sketched the use of Leverage Point #9, Delays, to alter the levels of technical debt in an enterprise.
In this post, I’ll sketch the use of interventions at Leverage Point #8, which Meadows calls, “The strength of negative feedback loops, relative to the impacts they are trying to correct against.”
A feedback loop that now provides budgetary control in most organizations
One feedback loop at issue in this case, illustrated above, provides budgetary control. It influences managers who might otherwise overrun their budgets by triggering some sort of organizational intervention when they do overrun their budgets. And it leads to increases in the portfolios of managers who handle their budgets responsibly. Presumably, that’s why managers compel estimators to find approaches that cost less. The feedback loop to which managers are exposed causes them to establish another feedback loop involving the engineer/estimator, and later the engineering team, to hold down their estimates, and later their actual expenditures.
We can use a diagram of effects [Weinberg 1992] to illustrate the feedback mechanism commonly used to control the performance of managers who are responsible for portfolios of project budgets. In the diagram, the oval blobs represent quantities indicated by their respective captions. Each of these quantities is assumed to be measurable, though their precise values and the way we measure them are unimportant for our rather qualitative argument.
Notice that arrows connect the blobs. The arrows represent the effect of changes in the value represented by one blob on the value represented by another. The blob at the base of the arrow is the effector quantity. The blob at the point of the arrow is the affected quantity. Thus, the arrow running from the blob labeled “Actual Spend” to the blob labeled “Overspend” expresses the idea that a positive (or negative) change in the amount of actual spending on projects causes a positive (or negative) change in Overspend. When a change in the effector quantity causes a like-signed change in the affected quantity, we say that their relationship is covariant.
Because increases in Budget Authority tend to decrease Overspend, all other things being equal, the relationship between Budget Authority and Overspend is contravariant. We represent a contravariant relationship between the effector quantity and the affected quantity as an arrow with a filled circle on it.
Finally, notice that the arrow from Overspend (effector) to Promotion Probability (affected) has a filled Delta on it. This represents the idea that as Overspend increases, it negatively affects the probability that the manager will be promoted at some point in the future. The Delta indicates a delayed effect; that the Delta is filled indicates a contravariant relationship. (An unfilled Delta would indicate a delayed covariant effect.)
This diagram, which contains a loop connecting Budget Authority, Overspend, and Promotion Probability, has the potential to “run away.” That is, as we go around the loop, we find self-re-enforcement, because the loop has an even number of contravariant relationships. It works as follows:
As Overspend increases, after a delay, the Probability of Promotion decreases. This causes reductions in Budget Authority because, presumably, the organization has reduced faith in the manager’s performance. Reductions in Budget Authority make Overspend more likely, and round and round we go.
Similarly:
As Overspend decreases, after a delay, the Probability of Promotion increases. This causes increases in Budget Authority because, presumably, the organization has increased faith in the manager’s performance. Increases in Budget Authority make Overspend less likely, and round and round we go.
Fortunately, other effects usually intervene when these self-re-enforcing phenomena get too large, but that’s beyond the scope of this argument. For now, all we need observe is that managers who manage their budgets effectively tend to rise in the organization; those who don’t, don’t.
The result is that managers seek to limit spending so as to avoid overspending their budget authority. And that’s one reason why they push engineers to produce lower estimates for technical debt retirement projects.
How this feedback loop overlooks important drivers of technical debt formation
To break the connection between the managers’ reification error and the engineers’ confirmation bias, our intervention must cause the managers and the engineers to make calculations differently. We can accomplish this by requiring that they consider more than the mere cost of retiring the class of technical debt under consideration. They must estimate the consequences of not retiring that technical debt, and they must also estimate costs beyond the cost of retiring the debt. In what follows, I’ll use the shorthand TDBCR to mean the class of Technical Debt Being Considered for Retirement.
Specifically, the estimates that are now typically generated for such projects cover only the cost of performing the work required to retire the TDBCR. It’s then left to Management to decide whether, when, and to what extent to commit resources to execute the project. The primary consideration is the effect on the decision-maker’s budget, and the consequences for achieving the goals for which the decision-maker is responsible.
Since the retirement project can potentially provide benefits beyond the manager’s own portfolio, failing to undertake the project can have negative consequences for which the manager ought to be held accountable. That’s the heart of the problem. So let’s look at some examples of considerations that must be taken into account.
Adjustments that would be needed in these feedback loops to gain control of technical debt
In making a resource allocation decision for a technical debt retirement project, there are considerations beyond the cost of retiring the debt. A responsible decision regarding undertaking technical debt retirement projects is possible only if other kinds of estimates are also generated and available. Here are some examples:
The effects of retiring TDBCR on the cost of executing any other development or maintenance efforts contemplated or already underway
The effects of retiring TDBCR on revenue and market share for all existing assets that directly produce revenue and which could be affected by retiring TDBCR
The revenue that would be generated (and timing thereof) by any new products or services that would be enabled by retiring TDBCR
The effects of retiring TDBCR on the cost of executing other technical debt retirement efforts
And these items might not be related to anything for which the decision-maker is responsible. That’s the core of the problem we now face: the feedback loop we now use to influence the decision-maker excludes considerations that are affected by the decision-maker’s decisions. Until we install feedback loops that cause the decision-maker to consider these consequences, or until we make decisions at levels that include these other consequences, the effects of the decision-maker’s decisions are uncontrolled, and might not lead to decisions optimal for the enterprise.
References
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
Technical debt arises in enterprise assets through the effects of two classes of drivers: obsolescence and decision-making. When technologies advance, or new technologies arise, or laws or regulations evolve or are introduced, existing assets or assets under development can sometimes be left behind. That’s how obsolescence produces technical debt. Debt driven mainly by decision-making is more difficult to describe, but anything that biases decisions away from strictly rational results presents risk. Three cognitive biases likely have strong effects on technical debt formation and persistence.
Photo of Daniel Ellsberg, speaking at a press conference in New York City in 1972. Best known for his role in releasing the Pentagon Papers, Dr. Ellsberg made important contributions to decision theory while at the RAND Corporation. Photo by Bernard Gotfryd, courtesy U.S. Library of Congress.Decision-making produces technical debt as the people of the enterprise make choices in design, development, and resource acquisition or allocation. Typically, both obsolescence and decision-making contribute to producing any particular instance of technical debt, though either obsolescence or decision-making might be more important than the other in any given instance.
Managing debt driven principally by obsolescence isn’t difficult, but I’ll leave that topic for another time. For now, let’s focus on decision-making. Already widely accepted is the contribution of engineering decisions to technical debt formation. Indeed, many believe that all — or most — technical debt arises as a result of faulty decisions by engineers. While some engineering decisions are indeed faulty, the current scale of technical debt is so large as to create doubt about the idea that the only decisions contributing to technical debt formation are engineering decisions. Investigating how resource allocation decisions might contribute to technical debt formation is certainly worthwhile.
In this post, I propose three examples illustrating how resource allocation decisions might contribute to technical debt formation and persistence. Each example illustrates how people make faulty decisions while believing they’re proceeding objectively and rationally. In each case, what causes the problem is a phenomenon called cognitive bias, though each example in this post illustrates the action of a different cognitive bias.
Loss aversion
The cognitive bias known as loss aversion, first identified by Amos Tversky and Daniel Kahneman [Kahneman 1984], is the tendency to prefer options that avoid losses to options that lead to gains that are equivalent or even greater. A decision-maker affected by loss aversion bias might conclude that it’s better to not lose $5 than to find $5 or even $10. In this way, loss aversion skews decisions so as to favor options that enable the enterprise to protect or enhance existing revenue streams, even if that decision causes increases in operating expenses. And this bias has effect even if the increases in operating expenses exceed the value of whatever revenue that decision protected.
Retiring technical debt usually entails deferring revenue in the short term, for two classes of reasons. First, we must turn the attention of some part of the engineering organization to debt retirement, instead of whatever they were doing. Assuming that they would have been working on maintaining or enhancing existing products or services, this redirection of their attention can lead to reducing or deferring revenue. Second, during the debt retirement operation, some work might require short-term interruptions of revenue streams, for the purpose of installing or testing the assets that are being revised for debt retirement purposes.
Thus, debt retirement efforts often do reduce revenue — or reduce revenue increases — in the short term, and some decision-makers can perceive that effect as a loss.
However, the long-term effects of debt retirement can be gains, and those gains can be considerable. Typically, by retiring an asset’s technical debt, we reduce the difficulty (read: time required, effort, cost, and risk) of future maintenance and enhancement efforts involving the asset. We also reduce the probability of debt contagion.
Since these long-term effects of debt retirement are ongoing, their impact on the enterprise can be significant. But unless one is experienced with dealing with the consequences of technical debt, recognizing the value of retiring technical debt can be difficult. When loss aversion is in play, intuitive comparisons of the effects of (a) a short-term revenue loss or delay to (b) a long-term benefit of debt retirement are biased in favor of not retiring technical debt.
Insulating decisions about debt retirement from the effects of loss aversion bias requires objective mathematical modeling of revenue losses and operating cost benefits for all options under consideration. Those models must also account for uncertainty, which makes them inherently ambiguous. And that leads us to consider our next cognitive bias, the ambiguity effect.
The ambiguity effect
The cognitive bias known as the ambiguity effect causes us to prefer options for which the probability of a desirable outcome is relatively better known, over options for which the probability of a desirable outcome is less well known, even if the expected value of that more ambiguous outcome exceeds the expected value of the less ambiguous outcome. The effect was first described by Daniel Ellsberg [Ellsberg 1961].
Consider a choice between allocating resources to a new development project and allocating resources to a technical debt retirement project. In most enterprises, decision-makers are familiar with new development projects. Likewise, project champions, project sponsors, and project managers are also familiar with new development projects. All parties are less familiar with debt retirement. It’s reasonable to suppose that when confronted with such a choice, decision-makers are likely to see debt retirement as carrying with it a probability of positive outcome that is less well known than the probability of a positive outcome for the new development project.
Because of the ambiguity effect, resource allocation decisions are likely to be biased against technical debt retirement, and in favor of maintenance or new development.
But there’s more. Most projects, of any kind, encounter trouble from time to time. When that happens, the urge to reallocate organizational resources can be powerful. Troubled projects might receive more resources if they’re viewed as important to the organization. If so, those resources often come from other projects. The ambiguity effect biases these resource reallocation decisions in a way analogous to initial resource allocation decisions, as described above. In other words, because of the ambiguity effect, when projects encounter trouble, debt retirement projects are less likely to be able to retain previously allocated resources than are maintenance or new development projects.
The availability heuristic
The availability heuristic is a method humans use to evaluate the validity or effectiveness of decisions, concepts, methods, or propositions [Tversky 1973]. According to the heuristic, if we recognize the item being evaluated as familiar, or related to something with which we are familiar, we’re more likely to regard it as valid or workable. And when making comparisons between two alternative decisions, concepts, methods, or propositions, we’re likely to assess more favorably the decision, concept, method, or proposition with which we’re more familiar, all other things being equal.
In organizations where decision-makers have more experience evaluating maintenance or development project proposals than they have with technical debt retirement proposals, the availability heuristic acts to reduce the relative assessed favorability of technical debt retirement proposals. It does this in three ways.
First, in most organizations, technical debt retirement projects are less familiar to decision-makers than are maintenance or development projects. On that ground alone the technical debt retirement project proposals are at a disadvantage.
But the second effect of the availability heuristic is more important, because the effect of the availability heuristic extends to the consequences of the decision, concept, method, or proposition under consideration. To grasp the value of a maintenance or development project, one must understand how it will affect the users of the assets being developed or maintained. Likewise, to grasp the value of a technical debt retirement project, one must understand how the presence of the technical debt hampers the enterprise in its attempts to achieve its objectives. On must also understand how retiring the technical debt might confer advantages in terms of future engineering efforts. Usually, understanding the consequences of maintenance or development projects is more “available” to decision-makers than is understanding the consequences of technical debt retirement projects. Even more dramatic is the difference between understanding the consequences of not funding a maintenance or development project and the consequences of not funding a technical debt retirement project.
Finally, much of the benefit of a technical debt retirement project is indirect. That is, although there is some direct benefit in terms of the assets from which the debt has been retired, the most dramatic benefits are manifested in projects that follow the debt retirement project, and which depend on the assets that have been relieved of debt. Sometimes, those follow-on projects are known at the time decision-makers are considering funding the debt retirement project. Sometimes those follow-on projects have yet to be specified or even recognized. In either case, they are less “available” to decision-makers because those follow-on projects are indirect beneficiaries.
These three effects of the availability heuristic cause decisions about resource allocations to tend to favor maintenance or development projects over debt retirement projects.
Mitigating the risks of these three cognitive biases
Over time, as everyone becomes more familiar with technical debt retirement projects, these effects may wane somewhat. But waiting for that to happen isn’t exactly what one might call risk mitigation. For one thing, familiarity grows only if one is motivated and pays attention. As busy as are decision-makers in modern organizations, depending on them to actively enhance their own familiarity with technical debt retirement projects is probably not the safest course.
An effective program of actively mitigating the risks of these three cognitive biases probably should focus on four areas.
Familiarity
Do what you can to increase decision-maker familiarity with the concept of technical debt, and with the consequences of carrying existing technical debt. Conventional presentation-based training will help, but interactive, experiential training is far more effective. Participants must actually experience the consequences of technical debt in a well-designed and professionally facilitated simulation of a problem-solving task. A faithful simulation should include estimation, changing and ambiguous requirements, and team composition volatility.
Retrospectives
Retrospectives (also known as after-action reviews, post mortems, debriefings, or lessons-learned sessions) are meetings convened to review processes that just completed pieces of work [Kerth 2001]. Typically, attendance is restricted to the project team members. To maintain psychological safety and to encourage truth telling, attendance by enterprise decision-makers is not recommended, unless the organizational culture includes appropriate safeguards. In any case, a section of the retrospective dedicated to investigating the causes and consequences of technical debt in the context of the current project can ensure capture of relevant knowledge and experience.
Mathematical modeling practice
Mathematical modeling is one path to creating a more objective foundation for decisions. It’s essential for improving estimation quality. Also helpful are high quality effort data and metrics data related to the formation and lifetime of technical debt. Reviews of estimates and projections during retrospectives can help improve their quality over time.
Metrics development
Determining the effects of risk mitigation failure provides important guidance for corrective action in risk mitigation. Developing metrics that reveal these failures is therefore essential to managing cognitive bias risk. I’ll be suggesting some valuable metrics in a future post.
Last words
These three cognitive biases are by no means the only cognitive biases that can affect the formation or persistence of technical debt. Of the more than 200 identified cognitive biases, those most likely to be relevant are those that affect decision-making. Watch this space for links to posts about additional cognitive biases and their affects on technical debt formation or persistence.
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
[Tversky 1973] Amos Tversky and Daniel Kahneman. "Availability: A heuristic for judging frequency and probability." Cognitive Psychology 5:2, 207-232, 1973.
Mastering understanding of exogenous technical debt—debt that arises from causes not directly related to the asset that bears the debt—is essential to controlling technical debt formation. Exogenous technical debt is particularly troublesome to those who work on the affected assets. They can’t control its formation, and they’re rarely responsible for creating it. But their internal customers and those who control resources often fail to understand this. Indeed, those who work on the affected assets are often blamed for the formation of exogenous technical debt even though they had no role in its formation, and could have done nothing to prevent its formation.
Asbestos with muscovite. Asbestos is a family of six minerals that occur naturally in fibrous form. The fibers are all known carcinogens. Until 1990, it was widely used in many common building materials, including insulation, plaster, and drywall joint compound. It is now banned, but it’s present in many existing structures, including homes and offices. The installation of the ban caused these structures to incur exogenous technical debt. Photo by Aramgutang courtesy Wikipedia.
Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that don’t involve the asset directly.
Because so much technical debt is produced indirectly, controlling its direct formation—for example, by engineering teams—isn’t sufficient for achieving enterprise control of technical debt formation. To control technical debt formation, we must track which activities produce it, including both direct and indirect effects. Allocating technical debt retirement costs to the activities that brought that debt about, even if the allocation doesn’t affect budget authority for those activities, is therefore a useful practice. Knowledge about which past activities created technical debt, and how much, is helpful for long-term reduction in the rate of technical debt formation.
When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries—or will be carrying—the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing). So endogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that directly involve the asset.
More about endogenous technical debt in future posts. For now, let’s look more closely at exogenous technical debt, and its policy implications.
Examples of exogenous technical debt
In “Spontaneous generation,” I examined one scenario in which technical debt formation occurs spontaneously—that is, in the absence of engineering activity. Specifically, I noted how the emergence of the HTML5 standard led to the formation of technical debt in some (if not all) existing Web sites, in the sense that they didn’t exploit capabilities that had become available in HTML5. Moreover, some sites whose developers had elected to emulate capabilities of the new standard by exploiting alternative technologies needed rehabilitation to remove the emulation and replace it with use of facilities in the HTML5 standard. All of these artifacts—including those that existed, and those that didn’t—comprised technical debt. This scenario thus led to the formation of exogenous technical debt.
In a second example, AMUFC, A Made-Up Fictitious Corporation, incurs technical debt when the vendor that supplies the operating system (OS) for AMUFC’s desktop computers announces the date of the end of extended support for the version of the OS in use at AMUFC. Because the end of extended support brings an end to security updates, AMUFC must retire that debt by migrating to the next version of that vendor’s OS before extended support actually ends.
In both of these examples, the forces that lead to formation of exogenous technical debt are external to both the enterprise and the enterprise’s assets. But what makes technical debt exogenous is that the forces that led to its formation are unrelated to any of the engineering work being performed on the asset that carries the debt. This restriction is loose enough to also include technical debt that arises from any change or activity external to the asset, but within the enterprise.
Exogenous technical debt arising from actions within the enterprise
Exogenous technical debt can arise from activities or decisions that take place entirely within the enterprise.
For example, consider a line of mobile devices developed and marketed by AMUFC (A Made-Up Fictitious Corporation). Until this past year, AMUFC has been developing ever more capable devices, thereby extending its line of offerings at the high end—the more expensive and capable members of the line. But this past quarter, AMUFC developed a low-end member of the line, and as often happens, price constraints led to innovations that could produce considerable savings in manufacturing costs if those innovations were applied to all members of the line. In effect, then, the designs of the previously developed models in this line of devices have incurred exogenous technical debt. The debt is exogenous because the activity that led to debt formation was not performed on the assets that now carry the debt, even though the activity that led to debt formation occurred within the enterprise. This kind of exogenous technical debt might be termed asset-exogenous. Exogenous technical debt of the kind that’s incurred by activity beyond the enterprise might be termed enterprise-exogenous.
Exogeneity versus endogeneity
For asset-exogenous technical debt, ambiguity between endogeneity and exogeneity can arise. The example above regarding the line of mobile devices produced by AMUFC provides an illustration.
For convenience, call the team that developed one of the high-end devices Team High. Call the team that developed the low-end device Team Low. From the perspective of Team High, the technical debt due to the innovations discovered by Team Low is exogenous. But from the perspective of the VP Mobile Devices, that same technical debt might be regarded as endogenous. The debt can be endogenous at VP level because it’s possible to regard the entire product line as a single asset, and that might actually be the preferred perspective of VP Mobile Devices.
Exogeneity and legacy technical debt
The technical debt portfolio of a given asset can contain a mix a technical debt that arose from various past incidents. In assessing the condition of the asset, it’s useful to distinguish this existing debt from debt that’s incurred as a consequence of any current activity or decisions. Call this pre-existing technical debt legacy technical debt.
The legacy technical debt carried by an asset is technical debt associated with that asset, and which exists in that asset in any form prior to undertaking work on that asset. For example, in planning a project to renovate the hallways and common areas of a high-rise apartment building, workers discover that beneath the existing carpeting is a layer of floor tile containing asbestos. Management has decided to remove the tile. In this context, the floor tile can be viewed as legacy technical debt. It isn’t directly related to the objectives of the current renovation, but removing it will enhance the safety of future renovations, enable certification of the building as asbestos-free, increase the property value, and reduce the cost of eventual demolition. In this situation asbestos removal amounts to retirement of legacy technical debt, and accounting for it as part of the common-area renovation would be misleading.
When contemplating efforts to retire legacy technical debt, exogeneity becomes a factor in allocating the necessary resources. If the debt in question is enterprise-exogenous, then we can justifiably budget the effort from enterprise-level accounts if appropriate. For other cases, other pools of resources become relevant depending on what actions created the debt. For example, if the exogenous technical debt arose because of a departmental change in standards, debt retirement costs can justifiably be allocated to the standards effort. If the exogenous technical debt arose from innovations in other members of the asset’s product line, those debt retirement costs can justifiably be allocated to the product line.
Policy insights
Understanding the properties of exogenous technical debt can be a foundation for policy innovations that enhance enterprise agility.
Culture transformation
Widespread understanding the distinction between exogenous and endogenous technical debt is helpful in controlling blaming behavior that targets the engineering teams responsible for developing and maintaining technological assets.
Understanding of asset-exogenous technical debt helps non-engineers understand how their actions and decisions can lead to technical debt formation, even when there is no apparent direct connection between those actions or decisions and the assets in question.
Resource allocation
Data about the technical debt creation effects of enterprise activities is helpful in allocating technical debt retirement costs. For example, when we know all the implications of reorganization, including its impact on internal data about the enterprise itself, we can charge data-related activity to the reorganization instead of to general accounts of the Information Technology function. This helps the enterprise understand the true costs of reorganization.
Similarly, data about enterprise-exogenous technical debt helps planners understand how to deploy resources to gather external intelligence about trends that can affect internal assets. Such data is also useful for setting levels of support and participation in industrial standards organizations or in lobbying government officials.
Knowing the formation history of exogenous technical debt provides useful guidance for those charged with allocating the costs of retiring technical debt or preventing its formation.
References
[Ellsberg 1961] Daniel Ellsberg. "Risk, ambiguity, and the Savage axioms." The quarterly journal of economics, 643-669, 1961.
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
[Tversky 1973] Amos Tversky and Daniel Kahneman. "Availability: A heuristic for judging frequency and probability." Cognitive Psychology 5:2, 207-232, 1973.
Separating responsibility for maintenance and acquisition of technical assets can lead to uncontrolled growth of technical debt. When the performance of the business acquisition function or the performance of the development organization is measured without regard for the technical debt that arises as a consequence of their actions, technical debt is likely to expand unchecked. To limit such expansion, policymakers must devise performance measures that hold these organizations accountable for the technical debt arising from their actions.
Road damage in Warwick, Rhode Island, resulting from historic storms in March 2010 [NOAA 2013]. The storms were so severe that at least one river flood gauge “flat-lined” — the floodwaters overtopped the gauge’s measurable range. Moreover, the National Weather Service (NWS) lacked a database of measurable ranges for flood gauges. Quoting the NWS report: “A lesson learned here was that maximum recordable river levels should be known by NWS staff, not only so staff aren’t surprised when this type of issue arises, but also to notify USGS personnel so that they can install a temporary gage and remove or elevate threatened equipment.” Technical debt, if ever I’ve seen it.For systems consisting solely of software, separation of responsibility for system maintenance and system development or acquisition enables the acquiring organization to act with little regard for the consequences of its decisions vis-à-vis maintenance matters [Boehm 2016]. This is an unfortunate state of affairs that increases the rate of accumulation of new technical debt, and increases the lifetime of legacy technical debt, because the MICs associated with the technical debt aren’t borne by the acquiring organization.
For example, a focus on performance of the organization that’s responsible for acquisition biases them in favor of attending to the direct and immediate costs of the acquisition, with little or no regard for ongoing maintenance issues. The maintenance organization is then left to deal with whatever the acquired system contains (or lacks).
An analogous mechanism operates for organizations that develop, market, and maintain products or services with software elements in their respective infrastructures. In that case, separation of the development function from the maintenance function enables the development function to act independently of the consequences of its decisions for maintenance matters.
But the separation-of-responsibilities mechanism that leads to uncontrolled technical debt isn’t restricted to software. Any technological asset that has ongoing maintenance needs (and most of them do) can potentially present this problem.
For example, in the United States, and many other countries, two streams of resources support publicly-owned infrastructure [Blair 2017]. The funding stream covers construction, operations and maintenance, and repairs. Its usual sources are taxes, tolls, licenses, other user fees, sale of ad space, and so on. The financing stream covers up-front construction costs, to bridge the period from conception through construction, until the funding stream begins delivering resources. The financing stream usually comes from bond sales.
Although both streams are controlled by legislatures, or by agencies they establish, the effects of the two streams differ fundamentally. The financing stream is dominant in the early stages of the asset’s lifecycle — during construction. The funding stream is dominant after that — when maintenance and operations are most important. Legislators and agencies are generally reluctant to supply funding because of the impact on taxpayers and users. Legislators and agencies find financing much more palatable. For this reason, among others, U.S. infrastructure maintenance is generally under-resourced, and technical debt gradually accumulates.
So it is with technological assets in organizations. For accounting purposes, capital expenses are treated differently from operational expenses, and the result is that operational expenses can have a more significant impact on current financial results than capital expenses do. This leads organizations to underfund operations and maintenance, which contributes to the accumulation of technical debt.
Control of new technical debt accumulation and enhancement of technical debt retirement rates is possible only if the acquisition or development organizations can somehow be held accountable for the MICs that result from their actions. Securitization of the debt incurred, as I’ll address in a forthcoming post, is one possible means of imposing this accountability. But reserves are also required, because some of the debt incurred might not be known at the time the asset is acquired or created.
Separation of responsibility for system maintenance and system acquisition or system development is actually a form of stovepiping. See “Stovepiping can lead to technical debt” for more on stovepiping.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
[Tversky 1973] Amos Tversky and Daniel Kahneman. "Availability: A heuristic for judging frequency and probability." Cognitive Psychology 5:2, 207-232, 1973.
Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—shouldcreate significant savings, and many competitive advantages. These benefits should arise from the reductions in metaphorical interest charges that result from retiring technical debt. But these benefits become available to the organization only if engineering capacity increases relative to the burdens presented by the remaining reduced levels of technical debt. After the technical debt management program is in place, if the balance between engineering resources and the burdens imposed by the remaining technical debt becomes more favorable, then organizational effectiveness will improve. But if the balance becomes less favorable, as a result of reductions in engineering resources, organizational effectiveness won’t improve, even at lower levels of technical debt.
Flooding from Hurricane Katrina in New Orleans, 2005. Any levee humans can build can be overtopped or undermined by the forces of Nature. So it is with technology. Any technology humans can devise to attain mastery over technical debt can be overcome or undermined by organizational policy and organizational politics. To master technical debt, technology is not enough—we must also deal with policy and politics.
Unfortunately, it’s possible to adopt advanced technical debt management practices while at the same time reducing engineering capacity to a level such that engineering effectiveness is no better than it was before the technical debt management program was initiated. The reason for this is that the engineering process is not the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves non-technical causes in place. That’s why technological solutions to the technical debt management problem might not be sufficient to produce benefits in organizational effectiveness and agility.
The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has been developed [Izurieta 2017]. It’s reasonable to assume that effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has not been confined to debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].
But it’s also reasonable to ask whether such developments will have much impact on the limiting effects of carrying technical debt, even in software. Given the necessary resources, much of the technical debt now extant could be retired. That is, debt retirement rates are determined only by the will and the capacity to invest in debt retirement. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.
The open question is this: will technological developments alone suffice to gain control of the problem of technical debt? Perhaps not. Organizations could exploit the advancements in technical debt management to execute reductions in engineering staffing—and therefore cost—while they divert savings to other parts of the enterprise, thereby allowing technical debt to remain at levels that, although much reduced, are nevertheless sufficient to compromise the effectiveness of that reduced engineering staff.
For example, schedule pressure is widely recognized as contributing to technical debt formation and persistence. If engineering groups become more adept at managing and preventing technical debt, but marketing and sales groups do not improve their own intelligence and planning processes and consequently demand new capabilities with even shorter timelines than they now do, the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.
Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of non-technical causes of technical debt to persist, and possibly even to grow.
This blog focuses on the non-technical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Marks 2005] Michelle A. Marks, Leslie A. DeChurch, John E. Mathieu, Frederick J. Panzer, and Alexander Alonso. “Teamwork in multiteam systems,” Journal of Applied Psychology 90:5, 964-971, 2005.
[Mathieu 2001] John E. Mathieu, Michelle A. Marks and Stephen J. Zaccaro. “Multi-team systems”, in Neil Anderson, Deniz S. Ones, Handan Kepir Sinangil, and Chockalingam Viswesvaran, eds., Handbook of Industrial, Work, and Organizational Psychology Volume 2: Organizational Psychology, London: Sage Publications, 2001, 289–313.
[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10
[Tversky 1973] Amos Tversky and Daniel Kahneman. "Availability: A heuristic for judging frequency and probability." Cognitive Psychology 5:2, 207-232, 1973.
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
