Tag: leverage point

Posted on

Legacy technical debt retirement decisions

Last updated on July 15th, 2021 at 07:17 pm

Two alternatives to retiring legacy technical debt in irreplaceable assets
Two alternatives to retiring legacy technical debt in irreplaceable assets. Neither one works very well.

Some irreplaceable assets carry legacy technical debt. Although retiring an asset retires its technical debt, that option isn’t available for irreplaceable assets. We need another option. For irreplaceable assets, we need to find a way to retire the debt. As decision makers gather information and recommendations from around the organization, most will come to an unsettling conclusion. They’ll find that information and recommendations aren’t sufficient for making sound decisions about technical debt retirement. The issues are complex. Education is also needed. It’s entirely possible that in some organizations, the existing executive team might be out of its depth. To understand how this situation can arise, let’s explore the nature of legacy technical debt retirement decisions.

A common technical debt retirement scenario

What compels the leaders of a large enterprise to consider retiring the technical debt encumbering one of its irreplaceable assets is fairly simple: cost. Decision makers usually begin by investigating the cost of replacing the asset. This is the option I’ve cleverly called “Replace the Asset.” They then typically conclude that replacement isn’t affordable. At this point, many decision makers choose the option I’ve called “Do nothing.” Time passes. A succession of incidents occurs, in which teams attempt required repairs or enhancements of the asset. And I use the term required here to mean “essential to the viability of the business.”

Engineers then do their best to meet the need, but the cost is high, and the work takes too long. The engineers explain that the problems are due, in part, to the heavy burden of technical debt. Eventually someone asks the engineers to estimate the cost of “cleaning things up.” Decision makers receive the estimates and conclude that it’s “unaffordable right now.” They ask the engineers to “make do.” In other words, they stick with the Do Nothing option.

After a number of cycles repeating this pattern, decision makers finally agree to provide time and resources for technical debt retirement, but only because it’s the least bad alternative. The other alternatives—Replace the Asset, and Do Nothing—clearly won’t work and haven’t worked, respectively.

So there we are. Events have forced the organization to address the technical debt problems in this irreplaceable asset. And that’s where the trouble begins.

Decisions about retiring legacy technical debt

In scenarios like the one above, people have already made the fundamental decision: the enterprise will be retiring legacy technical debt from an irreplaceable asset. But that’s just the first ripple of waves of decisions to come. Many people in a variety of roles throughout the enterprise will be making many decisions. Let’s now have a look at a short catalog of what’s in store for such an enterprise.

Recall that most large technical debt retirement projects probably exhibit a high degree of wickedness in the sense of Rittel and Webber [Rittel 1973]. One consequence of this property is the need to avoid do-overs. That is, once we make a decision about how to proceed to the next bit of the work, we want that decision to be correct, or at least, good enough. The consequences of that decision should not leave the enterprise in a state that’s more difficult to resolve than the state in which we found it. Since another property of wicked problems is the prevalence of surprises, most decisions must be made in a collaborative context, which affords the greatest possibility of opening the decision process to diverse perspectives. We must therefore regard collaborative decision-making at every level as a highly valued competency.

What follows is the promised catalog of decision types.

Strategic decisions

This decision category leads the list. It provides the highest leverage potential for changing enterprise behavior vis-à-vis technical debt. Organizations confronting the problem of technical debt retirement from irreplaceable assets would do well to begin by acknowledging that although they might be able to deal with the debt burdening these assets right now, they must make a strategic change if they want to avoid even worse trouble. Accumulating debt to a level sufficient to compel chartering a major debt retirement project took years of deferring the inevitable. A significant change of strategy is necessary.

When changing complex social systems, applying the concept of leverage provides a critical advantage. Following the work of Meadows [Meadows 1997] [Meadows 1999] [Meadows 2008], we can devise interventions at several points that can have great impact on the rate of accumulation of technical debt. The leverage points of greatest interest are Feedback Loops, Information Flows, Rules, and Goals. For example, the enterprise can set a strategic goal of a specific volume of incremental technical debt incurred per project, normalized by project budget. See “Leverage points for technical debt management.”

One might reasonably ask why enterprise strategy must change; wouldn’t a change in technology strategy suffice? Changing how engineers go about their work would help—indeed in most cases it’s necessary. But because the conditions and processes that lead to technical debt formation and persistence transcend engineering activities, additional changes are required to achieve the objective of controlling technical debt.

Some technical debt is incurred as the result of a conscious decision. But some is nonstrategic. We might even be unaware of how it occurred. Both kinds of technical debt can arise as a result of nontechnical factors. Read a review of nontechnical precursors of nonstrategic technical debt.

Organizational decisions

Before chartering a technical debt retirement project (DRP) for an irreplaceable asset, it’s wise to consider how to embed the DRP in the enterprise.

The default organizational form for DRPs concerned with an asset A is usually analogous to that used for major projects focused on asset A. If the Information Technology (IT) unit would normally address issues in asset A, the debt retirement effort usually would be organized under IT. If A is a software product normally attended to in a product group, that same group would likely have responsibility for the DRP for asset A.

Although these default organizational structures are both technically and politically sensible, there’s an alternative approach worth investigating. It entails establishing a technical debt retirement function that becomes a center of excellence for executing technical debt retirement projects. That unit is also responsible for developing sound technical debt management practice. Such an approach is especially useful if the organization contemplates multiple debt retirement projects.

The fundamental concept that makes the center-of-excellence approach necessary is the wickedness of the technical debt retirement problem. To address the problem at scale requires capabilities beyond what IT, product units, or any conventional organizational elements can provide. The explosion of technical debt in most organizations is an emergent phenomenon. Every organizational unit contributed to the formation of the problem. And every organizational unit must contribute to its resolution.

Engineering decisions

Engineers tend to identify and classify technical debt items on technical grounds. Further, they tend to set technical debt retirement priorities on a similar basis. That is, they tend to set priorities highest for those debt items that they (a) recognize as debt items and (b) see as imposing high levels of MICs charged to engineering accounts. Engineers are less likely to assign high priorities to technical debt that generates MICs that are charged to revenue, or to other accounts, because those MICs are less evident—and in many cases less visible—to engineers.

Decisions regarding recognition of technical debt items and setting priorities for retiring them must take technological imperatives into account, but they must also account for MICs of all forms. Priorities must be consistent with enterprise imperatives.

Decisions about pace

Paraphrasing Albert Einstein, technical debt retirement projects should be executed as rapidly as possible, and no faster. The tendency among nonengineers and nontechnical decision makers is to push for rapid completion of debt retirement projects, for three reasons. First, everyone, like the engineers, wants the results that debt retirement will bring. Second, everyone, like the engineers, wants an end to the inevitable disruptions debt retirement projects cause. And finally, the longer the project is underway, the more it might cost.

For these reasons, once the decision to retire the debt is firmly in hand, the enterprise might have a tendency to apply financial resources at a rate that exceeds the ability of the project team to execute the project responsibly. When that happens, rework results. And for wicked problems like debt retirement, rework is the path to catastrophe.

Decisions about pace and team scale need to be regarded as tentative. Regular reviews can ensure that the resource level is neither too low nor too high. Even when the engineers are given control over these decisions, these decisions must be reviewed, because pressures for rapid completion can be so severe that they can compromise the judgment of engineers about how well they can manage the resources applied to the project.

Resource decisions

Debt retirement projects concerned with legacy irreplaceable assets are different from most other projects. Estimates of the labor hours required are more likely to be incorrect on the low side, because so much of the work involves pieces of assets with which few staff have experience. But with respect to resources, underestimating labor requirements isn’t the real problem. Nonlabor resources are the real problem.

Irreplaceable assets probably provide critical support of ongoing operations. In some cases, the need for the assets is continuous. Many organizations have kept such assets operational by using periods of low demand for maintenance, usually scheduled and announced in advance. These practices are likely adequate for routine maintenance and enhancement. But debt retirement need a level of access to the asset that continuous delivery practices can provide [Humble 2010].

However, assets whose designs predate the widespread use of modern practices such as continuous delivery might not be compatible with the infrastructure that these practices require. In organizations that haven’t yet adopted such practices, few if any staff are experienced with them. We must therefore regard as developmental any projects whose objectives are retiring technical debt from irreplaceable assets. They’re retiring the technical debt, but they’re also developing the practices and infrastructure needed for debt retirement projects. This dual purpose drives the surprisingly high nonlabor costs associated with early debt retirement projects.

The investments required might include such “items” as a staging environment, which “is a testing environment identical to the production environment” [Humble 2010]; extensive test automation, including results analysis; blue-green deployment infrastructure; automation-assisted rollback; and zero-downtime release infrastructure. Decisions to make investments require an appreciation of their value to the enterprise. They enable the enterprise to deal effectively with the wicked problem of technical debt retirement.

Last words

Because every situation and every organization is unique, few general guidelines are available for making these decisions. The criteria most organizations have been using for dealing with (or avoiding) the issue of technical debt have produced the problems they now face. So, to succeed from this point, whatever criteria they use in the future must be different. My own view is that short-term thinking is at the heart of the problem, but it’s a wicked problem. The long-term solution will not be simple.

References

[Humble 2010] Jez Humble and David Farley. Continuous delivery: reliable software releases through build, test, and deployment automation, Pearson Education, 2010.

Cited in:

[Meadows 1997] Donella H. Meadows. “Places to Intervene in a System,” Whole Earth, Winter 1997.

Available: here; Retrieved: June 28, 2018

Cited in:

[Meadows 1999] Donella H. Meadows. “Leverage Points: Places to Intervene in a System,” Hartland VT: The Sustainability Institute, 1999.

Available: here; Retrieved: June 2, 2018.

Cited in:

[Meadows 2008] Donella H. Meadows and Diana Wright. Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing, 2008.

Order from Amazon

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

Posted on

Synergy between the reification error and confirmation bias

Last updated on July 10th, 2021 at 08:53 am

In deciding whether to undertake technical debt retirement projects, organizations risk making inappropriate decisions because of a synergy between the reification error and confirmation bias. Together, these two errors of thought create conditions that make committing appropriate levels of resources difficult. And when organizations do commit resources, they tend to underestimate costs. That underestimate can elevate the chance of failure in technical debt retirement projects.

The reification error and confirmation bias

As explained elsewhere in this blog, the reification error is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing. Because technical debt is an abstraction, we risk committing the reification error when we deal with it. (See “Metrics for technical debt management: the basics”)

Confirmation bias is a cognitive bias that causes us to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. (See “Confirmation bias and technical debt”)

How the reification error affects management

The reification error might be responsible, in part, for a widely used management practice that often appears in the exploratory stages of undertaking projects. Let’s start with an illustration from the physical world.

In the physical world, when we want cherries, we go to a market and check the price per pound or kilo. Then we decide how much we want. If the price is high, we might decide to buy fewer cherries. If the price is low, we might buy more cherries. We have in mind a total cost target, and we adjust the weight of the cherries to meet the target. In the physical world, we can often adjust what we purchase to match our willingness to pay.

Retiring technical debt doesn’t work like that, in part, because technical debt is an abstraction. But we try anyway; here’s how it goes. Management decides to retire a particular class of technical debt. They ask an engineer for an estimate of the cost. Sometimes Management reveals the target they have in mind if they have one; sometimes not. The estimate comes back as Total ± Uncertainty. Management decides that’s too high, or the Uncertainty is too great. They then ask the engineer to find a way to do it for less, or to reduce the Uncertainty.

Management—the “customer” in this scenario—makes this request, in part, based on the belief that adjusting the work is possible. Management hopes that the engineer can adjust the work to meet a (possibly unstated) target, in analogy to buying cherries. That thinking is an example of the reification error. In this dynamic, we rarely take into account the fact that retiring technical debt isn’t exactly like buying cherries.

How confirmation bias affects engineering estimates

Return now to the interaction between Management and the engineer/estimator. The engineer now suspects that Management does have a target in mind. Some engineers might ask what the target is. Some don’t. In any case, the engineer makes a lower estimate, which might still be too high. This process repeats until either Management decides against retiring the debt, or accepts the lowest Total ± Uncertainty.

In adjusting their estimates, engineers have a conflict of interest. That conflict of interest can compromise their objectivity through the action of confirmation bias. For technical debt retirement efforts, engineers are usually highly motivated to gain Management approval of the project. The motivation arises, in part, from the frustrating loss of engineering productivity. And since engineers typically sense that Management approval of the project is contingent on finding an estimate that’s low enough, the engineers have a preconception. That is, engineers have an incentive to convince themselves that Management’s adjustments to budget and schedule are reasonable. Because of the confirmation bias, engineers tend to seek justifications for the adjustments. And they tend to avoid seeking justifications for believing that their adjustments might not be feasible. That’s the confirmation bias in action.

How synergy between the reification error and confirmation bias comes about

Because of the reification error, Management tends to believe that retiring technical debt is a more adjustable activity than it actually is. Because of confirmation bias, engineers tend to believe that Management’s proposed cost and schedule are feasible. Too often, the synergy between the two errors of thinking provides a foundation for disaster.

Why this synergy creates conditions for disaster in technical debt retirement projects

Management usually equates estimates with commitments. Engineers don’t. Management usually forgets or ignores the upside Uncertainty. Typically, when Management accepts an estimate, the engineering team finds that it has made a commitment to deliver the work for the cost Total, with zero upside Uncertainty. Rarely does Management make this explicit. An analogous problem occurs with schedule.

By ignoring the Uncertainty, Management (the buyer) transfers the uncertainty risk to the project team. That strategy might work to some extent with conventional development or maintenance projects, where we can adjust scope and risk before the work begins. But for technical debt retirement projects, this practice creates problems for two reasons.

Adjusting the scope of debt retirement projects is difficult

First, with technical debt retirement we’re less able to adjust scope. To retire a class of technical debt, we must retire it in toto. If we retire only some portion of a class of technical debt, we would leave the asset in a mixed state that can actually increase MICs. So it’s usually best to retire the entirety of any class of technical debt, so as to leave the asset in a uniform state.

Debt retirement efforts are notoriously unpredictable

Second, the work involved in retiring a particular class of technical debt is more difficult to predict than is the work involved in more conventional projects. (See “Useful projections of MPrin might not be attainable”) Often, we must work with older assets, or older portions of younger assets. The people who built them aren’t always available, and documentation can be sparse or unreliable. Moreover, it’s notoriously difficult to predict with accuracy when or for how long affected assets will be out of production. Revenue stream interruptions, which can comprise a significant portion of total costs, can be difficult to schedule or predict. Thus, technical debt retirement projects tend to be riskier than other kinds of projects. They have wider uncertainty bands. Ignoring the Uncertainty, or trying to transfer responsibility for it to the project team, is foolhardy.

A strategy for reducing the effects of this synergy

To intervene in the dynamic between the consequences of the reification error and the consequences of confirmation bias, we must find a way to limit how their consequences can interact. That will curtail the ability of one phenomenon to reinforce the other. This task is well suited for application of Donella Meadows’ concept of leverage points [Meadows 1999]. See “Leverage points for technical debt management.”

In that post, I summarized Meadows’ concepts of using leverage points to alter the behavior of complex systems. One can intervene at one or more of 12 categories of leverage points. These are elements in the system that govern the behavior of the people and institutions that comprise the system. In that post, I sketched the use of Leverage Point #9, Delays, to alter the levels of technical debt in an enterprise.

In what follows I sketch the use of interventions at Leverage Point #8, “The strength of negative feedback loops, relative to the impacts they are trying to correct against.”

Our strategy is this:

A feedback loop that now provides budgetary control in most organizations

One feedback loop at issue in this case, illustrated above, influences managers who might otherwise overrun their budgets. It does so by triggering some sort of organizational intervention when a manager overruns his or her budget. And the feedback loop leads to increases in the size and stature of the portfolios of managers who handle their budgets responsibly.  Presumably, that’s one reason why managers compel estimators to find approaches that cost less. The feedback loop to which managers are exposed causes them to establish another feedback loop involving the engineer/estimator, and later the engineering team. That second loop causes engineers to hold down their estimates, and later to limit actual expenditures.

A diagram of effects analysis

A feedback loop that now provides budgetary control in most organizations
A feedback loop that now provides budgetary control in most organizations.

We can use a diagram of effects [Weinberg 1992] to illustrate the feedback mechanism commonly used to control the performance of managers who are responsible for portfolios of project budgets. In the diagram (above), the oval blobs represent quantities indicated by their respective captions. Each of these quantities is assumed to be measurable, though their precise values and the way we measure them are unimportant for our rather qualitative argument.

What the arrows mean

Notice that arrows connect the blobs. The arrows represent the effect of changes in the value represented by one blob on the value represented by another. The blob at the base of the arrow is the effector quantity. The blob at the point of the arrow is the affected quantity. Thus, the arrow running from the blob labeled “Actual Spend” to the blob labeled “Overspend” expresses the idea that a positive (or negative) change in the amount of actual spending on projects causes a positive (or negative) change in Overspend. When a change in the effector quantity causes a like-signed change in the affected quantity, we say that their relationship is covariant.

Because increases in Budget Authority tend to decrease Overspend, all other things being equal, the relationship between Budget Authority and Overspend is contravariant. We represent a contravariant relationship between the effector quantity and the affected quantity as an arrow with a filled circle on it.

Finally, notice that the arrow from Overspend (effector) to Promotion Probability (affected) has a filled Delta on it. This represents the idea that as Overspend increases, it negatively affects the probability that the manager will be promoted at some point in the future. The Delta indicates a delayed effect; that the Delta is filled indicates a contravariant relationship. (An unfilled Delta would indicate a delayed covariant effect.)

Loops in the diagram of effects

This diagram, which contains a loop connecting Budget Authority, Overspend, and Promotion Probability, has the potential to “run away.” That is, as we go around the loop, we find self-re-enforcement, because the loop has an even number of contravariant relationships. It works as follows:

As Overspend increases, after a delay, the Probability of Promotion decreases. This causes reductions in Budget Authority because, presumably, the organization has reduced faith in the manager’s performance. Reductions in Budget Authority make Overspend more likely, and round and round we go.

Similarly:

As Overspend decreases, after a delay, the Probability of Promotion increases. This causes increases in Budget Authority because, presumably, the organization has increased faith in the manager’s performance. Increases in Budget Authority make Overspend less likely, and round and round we go.

Fortunately, other effects usually intervene when these self-re-enforcing phenomena get too large, but that’s beyond the scope of this argument. For now, all we need observe is that managers who manage their budgets effectively tend to rise in the organization; those who don’t, don’t.

The result is that managers limit spending to avoid overspending their budget authority. And that’s one reason why they push engineers to produce lower estimates for technical debt retirement projects.

How this feedback loop overlooks important drivers of technical debt formation

To break the connection between the managers’ reification error and the engineers’ confirmation bias, our intervention must cause the managers and the engineers to make calculations differently. We can accomplish this by requiring that they consider more than the mere cost of retiring the class of technical debt under consideration. They must estimate the consequences of not retiring that technical debt, and they must also estimate costs beyond the cost of retiring the debt. In what follows, I’ll use the shorthand TDBCR to mean the class of Technical Debt Being Considered for Retirement.

Specifically, estimates for technical debt retirement projects cover only the cost of performing the work required to retire the TDBCR. Management then decides whether, when, and to what extent to commit resources to execute the project. The primary considerations budgetary.

Since the debt retirement project can potentially provide benefits beyond the manager’s own portfolio, failing to undertake the project can have negative consequences. Mnagers who decline to undertake debt retirement projects are responsible for the consequences. But accountability for these decisions is rare. That’s the heart of the problem. So let’s look at some examples of relevant considerations.

Adjustments that would support these feedback loops to gain control of technical debt

In allocating resources for a technical debt retirement project, there are considerations beyond the cost of retiring the debt. A responsible decision is possible only if other kinds of estimates are also available. Here are some examples of the estimates we need:

  • The effects of retiring TDBCR on the cost of executing any other development or maintenance effortsy
  • The effects of retiring TDBCR on revenue and market share for all existing assets that directly produce revenue and which could be affected by retiring TDBCR
  • The revenue that would become available (and timing thereof) from any new products or services that become possible because of retiring TDBCR
  • The effects of retiring TDBCR on the cost of executing other technical debt retirement efforts

And these items might not be related to anything for which the decision maker is responsible. But the feedback loop we now use to influence the decision maker excludes considerations that are affected by the decision maker’s decisions. Until we install feedback loops that cause the decision maker to consider these indirect consequences, or until we make decisions at levels that include these other consequences, the effects of the decision maker’s decisions are uncontrolled, and might not lead to decisions optimal for the enterprise.

References

[Humble 2010] Jez Humble and David Farley. Continuous delivery: reliable software releases through build, test, and deployment automation, Pearson Education, 2010.

Cited in:

[McConnell 2006] Steve McConnell. Software Estimation: Demystifying the Black Art. Microsoft Press, 2006.

Order from Amazon

Cited in:

[Meadows 1997] Donella H. Meadows. “Places to Intervene in a System,” Whole Earth, Winter 1997.

Available: here; Retrieved: June 28, 2018

Cited in:

[Meadows 1999] Donella H. Meadows. “Leverage Points: Places to Intervene in a System,” Hartland VT: The Sustainability Institute, 1999.

Available: here; Retrieved: June 2, 2018.

Cited in:

[Meadows 2008] Donella H. Meadows and Diana Wright. Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing, 2008.

Order from Amazon

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Weinberg 1992] Gerald M. Weinberg. Quality Software Management Volume 1: Systems Thinking. New York: Dorset House, 1989.

This volume contains a description of the “diagram of effects” used to explain how obstacles can induce toxic conflict. Order from Amazon

Cited in:

Other posts in this thread

Posted on

Managing technical debt

Last updated on July 11th, 2021 at 02:56 am

A jumble of jigsaw puzzle pieces. Managing technical debt can be like solving a puzzle.
A jumble of jigsaw puzzle pieces. Managing technical debt can be like solving a puzzle. Where do we begin? With jigsaw puzzles, we usually begin with two assumptions. First, we assume that we have all the pieces. Second, we assume that they fit together to make a coherent whole. These assumptions might not be valid for the puzzle of technical debt in any given organization.

Managing technical debt is something few organizations now do. And fewer do well. Several issues make managing technical debt difficult and they’re discussed elsewhere in this blog. This thread explores tactics for dealing with those issues from a variety of initial conditions. For example, tactics that work well for an organization that already has control of its technical debt, and which wants to keep it under control, might not work at all for an organization that’s just beginning to address a vast portfolio of runaway technical debt. The needs of these two organizations differ. The approaches they must take might then also differ.

What’s in this thread

The first three posts in this thread illustrate the differences among organizations in different stages of developing technical debt management practices. In “Leverage points for technical debt management,” I begin to address the needs of strategists working in an organization just beginning to manage its technical debt. They ask the question, “Where do we begin?” In “Undercounting nonexistent debt items,” I offer an observation about a risk that accompanies most attempts to assess the volume of technical debt. Such assessments are frequently undertaken in organizations at early stages of the technical debt management effort. In “Crowdsourcing debt identification,” I discuss a method for maintaining the contents of a database of technical debt items. Data maintenance is something we might undertake in the context of a more advanced technical debt management program.

Obstacles we must address

Whatever approach is adopted, it must address factors that include technology, business objectives, politics, culture, psychology, and organizational behavior. So what you’ll find in this thread are insights, observations, and recommendations that address one or more of the issues related to these fields. “Demodularization can help control technical debt” considers mostly technical strategies. “Undercounting nonexistent debt items” is an exploration of a psychological phenomenon.  “Leverage points for technical debt management” considers the organization as a system and discusses tactics for altering it. And “Legacy debt incurred intentionally” explores how existing technical debt can grow as long as it remains outstanding.

Accounting issues also play a role. For example, “Metrics for technical debt management: the basics” is a basic discussion of measurement issues. A second example: “Accounting for technical debt” looks into the matter of accounting for technical debt financially. And “Three cognitive biases” is a study of how the way we think about technical debt affects the technical debt portfolio.

Posts in this thread:

References

[Humble 2010] Jez Humble and David Farley. Continuous delivery: reliable software releases through build, test, and deployment automation, Pearson Education, 2010.

Cited in:

[McConnell 2006] Steve McConnell. Software Estimation: Demystifying the Black Art. Microsoft Press, 2006.

Order from Amazon

Cited in:

[Meadows 1997] Donella H. Meadows. “Places to Intervene in a System,” Whole Earth, Winter 1997.

Available: here; Retrieved: June 28, 2018

Cited in:

[Meadows 1999] Donella H. Meadows. “Leverage Points: Places to Intervene in a System,” Hartland VT: The Sustainability Institute, 1999.

Available: here; Retrieved: June 2, 2018.

Cited in:

[Meadows 2008] Donella H. Meadows and Diana Wright. Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing, 2008.

Order from Amazon

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Weinberg 1992] Gerald M. Weinberg. Quality Software Management Volume 1: Systems Thinking. New York: Dorset House, 1989.

This volume contains a description of the “diagram of effects” used to explain how obstacles can induce toxic conflict. Order from Amazon

Cited in:
Posted on

Leverage points for technical debt management

Last updated on July 10th, 2021 at 07:17 am

McMurdo Station, Antarctica, as seen from nearby Observation Hill
McMurdo Station, Antarctica, as seen from nearby Observation Hill. The United States Antarctic Program, a unit of the National Science Foundation, operates the station. It can house as many as 1258 people in Summer. Photo (cc) Gaelen Marsden courtesy McMurdo Station in Antarctica.

Adopting a technical debt management programs entails significant organizational change. The problem can seem so daunting that we don’t know where to begin. The places to begin are the places where the change agents have greatest leverage—what systems analysts call leverage points. Consider this scenario:

You’re sitting in the kickoff meeting of the new Technical Debt Management Task Force. The CEO is talking about how she realized that the company had a technical debt problem. It was when the Marigold project went through delay after delay, and was finally declared done, with multiple objectives waived. She’s saying something about, “we were trying to do backflips with millstones around our necks. So I want this task force to show us how to get rid of the millstones, and then get rid of them.”

OK, you think. But how? We’re a global enterprise with thousands of engineers and operations on every continent. Except maybe Antarctica. No wait, we’re there, too. McMurdo I think. We have software we don’t even know much about, acquired long ago along with the companies that built it. And we’re building new systems or modifying old ones all the time, trying to move everything to the cloud while enhancing data security. Where do we begin to look for the millstones of technical debt?

Have you been in that meeting? If not, can you imagine being in that meeting? Meetings like that are happening around the globe. We’re all in the same soup.

Leverage points: how to get rid of the millstones

It turns out that the answers to the millstone questions are available, but the pioneers and deep thinkers who have shown the way aren’t working on technical debt. Their field is called systems analysis. They work on problems like the collapse of the North Atlantic fishery, urban deterioration, unemployment, poverty, climate change, and the causes of the Great Recession of 2008—really difficult problems. Although the technical debt problem isn’t quite that challenging, it’s challenging enough to justify taking a look at the methods of systems analysis.

And when we do that, we immediately encounter a concept many call leverage points.

What are leverage points?

Leverage points are places in complex systems where a small change in one thing can produce big changes in system behavior. In a brilliant 1997 article, Donella Meadows describes what she calls “places to intervene in a system.” [Meadows 1997] She followed this article, making improvements each time, in 1999 [Meadows 1999] and 2008 [Meadows 2008]. Let me summarize Meadows’ work here.

To alter the behavior of a complex system, intervene at one or more of 12 categories of leverage points. For example, one category is called “Rules.” It consists of the incentives, punishments, and constraints that govern the behavior of the people and institutions that comprise the system. By adjusting the system’s rules, we can alter overall system behavior.

One more thing: the leverage points form an ordered hierarchy, ordered by effectiveness. Acting at a higher-level leverage point is more effective than acting at a lower-level leverage point. And more difficult, too. The ordering of the categories is a bit fuzzy, because every situation has its own quirks, but generally, the order is as given in the list below.

The twelve leverage points

In a moment I’ll give an example of using leverage point #9, Delays, to bring about change in the way the enterprise deals with technical debt. But first, here’s a brief summary of the leverage points in increasing order of leverage; not enough to truly understand what they are, but probably enough to pique your interest. As I write posts that illustrate interventions at these leverage points, I’ll link to them from here.

  1. Numbers: Constants and parameters such as subsidies, taxes, and standards
  2. Buffers: The sizes of stabilizing stocks relative to their flows
  3. Stock-and-Flow Structures: Physical systems and their nodes of intersection
  4. Delays in feedback loops
  5. Balancing Feedback Loops: The strength of the feedbacks relative to the impacts they are trying to correct
  6. Reinforcing Feedback Loops: The strength of the gain of driving loops
  7. Information Flows:  The structure of who does and does not have access to information
  8. Rules: Incentives, punishments, and constraints
  9. Self-Organization: The power to add, change, or evolve system structure
  10. Goals: The purpose or function of the system
  11. Paradigms: The mind-set out of which the system—its goals, structure, rules, delays, parameters—arises
  12. Transcending Paradigms

Delays in feedback loops

When we use feedback to control systems, and there are delays in the feedback, we can potentially create destructive system behavior. And that can happen when we try to control technical debt.

Whenever we try to control a quantity in an enterprise process, we must (a) set a target value for that quantity; then (b) measure its current value; and then (c) take action as appropriate to move the current value toward the target value. Systems analysts (and control theorists) call that arrangement a feedback loop. The action taken to move the current value to the target value is sometimes called the control signal. Under certain conditions, the feedback works as expected.

For example, to control the profitability of the enterprise, we can examine its net income, say, quarterly. And at the end of each quarter we can make adjustments if net income isn’t in the target range.

Feedback loops generally work pretty well, but under some conditions, oscillations can develop. One of those troublesome situations occurs when there’s a delay in the loop that’s of the same order as (or longer than) the time the system takes to respond to adjustments. Meadows uses the example of adjusting the water temperature of a shower when there’s a long delay between making the adjustment and feeling its effects. Overcorrection is almost inevitable, and that’s what causes system oscillation.

How controlling technical debt can create feedback loops

So let’s suppose that we’re trying to control the rate of accumulation of technical debt. One approach is to set a target for TDnew, the new technical debt generated in a project. To be fair to all projects, we decide to normalize this quantity according to the project budget B. So we set targets for each project’s N = TDnew/B, and we require that projects estimate N, on an ongoing basis, with a goal of having N in some target range when the project is complete.

Identifying technical debt isn’t straightforward

One problem with this approach is that we rarely identify accurately all the technical debt we’ve incurred until some time has passed after project delivery. With time, as the newly produced assets go into production and learning accumulates, we acquire the wisdom needed to identify more of the technical debt we created. This is one source of delay in this feedback loop.

So let’s assume that this happens for several projects, and management decides that delayed recognition of incurred technical debt is a common occurrence. To account for this, management lowers the target ranges for N for future projects. This causes project managers and project sponsors to include in their project plans additional effort directed at retiring more of their incremental technical debt before their projects complete, to enable them to project lower values of N. They must therefore identify as much of the incremental technical debt as they can, and retire it, to meet the lower targets for N.

How oscillations set in

But recall that technical debt identification sometimes requires time and experience using the newly produced asset. And the reverse process also occurs. Technical artifacts that we thought were technical debt prove to be useful in unexpected ways, and actually turn out not to be debt items after all. As a result, some of the incremental technical debt that got retired before the project was completed actually should not have been retired. Eventually, people realize that this happens with uncomfortable frequency, and so the targets for N are raised once more.

Oscillations thus set in. Long delays inevitably cause them. To prevent oscillations, shorten the delays.

How to shorten delays in feedback controlling technical debt

When we use feedback to control a system, delays in that feedback can lead to instability. Trying to control technical debt is no exception. With technical debt we can shorten delays in several ways.

  • If the asset is meant for human use, involve representatives of the user population in the development and design process as soon as practical. Have them exercise the asset, or prototypes, early. Listen to their suggestions. Observe how they use the asset.
  • If the asset must interact with non-human assets, exercise it early and often. Don’t think of this as testing, though it might look very much like testing. What you’re actually doing is searching for shortcomings in how the asset interacts with non-human assets, in design and implementation in an asset that already works.
  • Subject the asset to multiple reviews all along the development trajectory. Don’t wait for final release to review it.

These practices expose technical debt items early—potentially, during initial design—thereby reducing delays in identifying what is and what isn’t technical debt. They help to advance the date at which we uncover missing capabilities or capabilities designed or implemented in awkward ways. No surprise, I’m sure, but these practices are consistent with Agile approaches to technological development.

Indirect effects can add to delayed recognition of technical debt

Most of the argument above assumed that the incremental technical debt associated with the project was incurred within the asset undergoing development or maintenance. But technical debt can occur in other assets as well. When the development team is unaware of such “remote” or “indirect” incremental technical debt, recognition of that new incremental technical debt can be significantly delayed. The project’s N (the ratio of incremental technical debt to project budget) will appear to be smaller that it actually is, until that remote incremental technical debt is recognized.

This form of delay is likely to occur when the debt incurred is asset-exogenous. Recall the example of line extension of mobile phones. In that example, the enterprise incurs technical debt in one set of products as a result of the introduction of a different product. In some cases, the newly incurred technical debt is immediately evident. When it is not, delays can be substantial.

This effect is by no means rare. Any organizational change can potentially add to the technical debt portfolio—reorganizations, acquisitions, expansions, wholly new products, and much more.

Last words

Interventions at the leverage points of an organization can produce the changes we want with a minimum of effort. Some subtlety is involved, because Meadows’ leverage points are expressed at a high level of abstraction.  But applying them to the problem of technical debt management is a promising approach.

Bookmark this post. I’ll be linking to more examples of using leverage points to manage technical debt. So far:

References

[Humble 2010] Jez Humble and David Farley. Continuous delivery: reliable software releases through build, test, and deployment automation, Pearson Education, 2010.

Cited in:

[McConnell 2006] Steve McConnell. Software Estimation: Demystifying the Black Art. Microsoft Press, 2006.

Order from Amazon

Cited in:

[Meadows 1997] Donella H. Meadows. “Places to Intervene in a System,” Whole Earth, Winter 1997.

Available: here; Retrieved: June 28, 2018

Cited in:

[Meadows 1999] Donella H. Meadows. “Leverage Points: Places to Intervene in a System,” Hartland VT: The Sustainability Institute, 1999.

Available: here; Retrieved: June 2, 2018.

Cited in:

[Meadows 2008] Donella H. Meadows and Diana Wright. Thinking in Systems: A Primer. White River Junction, VT: Chelsea Green Publishing, 2008.

Order from Amazon

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Weinberg 1992] Gerald M. Weinberg. Quality Software Management Volume 1: Systems Thinking. New York: Dorset House, 1989.

This volume contains a description of the “diagram of effects” used to explain how obstacles can induce toxic conflict. Order from Amazon

Cited in:

Other posts in this thread

Show Buttons
Hide Buttons