Controlling incremental technical debt

Last updated on July 16th, 2021 at 03:40 pm

A sinking rowboat provides a metaphor for controlling incremental technical debt — A sinking rowboat provides a useful metaphor for illustrating the process of controlling incremental technical debt. The enterprise is the rowboat. The leaks are the properties of the enterprise and its environment that lead to creating incremental technical debt. The water entering the boat through leaks is incremental technical debt. The accumulated water in the bottom of the boat is legacy technical debt.

All technical debt in enterprise assets is either incremental technical debt or legacy technical debt or both. Incremental technical debt is technical debt newly incurred. It can be newly incurred exogenous technical debt, or it can be endogenous technical debt incurred either in projects currently underway, or projects just recently completed. Legacy technical debt is technical debt associated with assets, and which wasn’t incurred recently or which exists in any form prior to undertaking work on those assets. All legacy technical debt was at some point incremental technical debt. The vast amounts of legacy technical debt most organizations now carry are nothing more than the accumulation of incremental technical debt. The path to managing legacy technical debt therefore begins with controlling incremental technical debt.

The leaky rowboat metaphor

Organizations are more likely to gain control of their legacy technical debt portfolio if they begin by controlling the formation of incremental technical debt, and its transformation into legacy technical debt. A metaphor might make this clear:

If you find yourself in a sinking rowboat, bailing out some of the water is a good idea. It might even be necessary in the short term. But at some point, fixing the leaks where the water comes in is advisable. Unless you address the existing leaks, and prevent new ones from forming, your fate is sealed. You’ll spend increasing portions of your time, energy, and resources bailing out your leaky rowboat. You’ll spend declining portions of your time, energy, and resources rowing the boat towards your objective. And when you do devote some time and energy to rowing towards your objective, you’ll find the rowing surprisingly difficult. The boat’s leaks make it ride lower in the water. And because you must propel not only the boat and its payload, but also the dead weight of the water in the bottom of the boat.

In this metaphor, legacy technical debt is the water in the bottom of the boat, and incremental technical debt is the water coming in through the leaks. The leaks are the proximate “causes” of technical debt. The root causes of the leaks are the root causes of technical debt.

Setting technical debt management priorities

If the enterprise is in the midst of a legacy technical debt emergency, retiring some of it is necessary in the short term. But unless the enterprise addresses incremental technical debt and its root causes, a new burden of legacy technical debt will accumulate. That accumulation is then likely to eliminate the benefits of having retired the current burden of legacy technical debt.

So after the legacy technical debt emergency is passed—or if resources permit, during the emergency—establishing measures, procedures, and practices for controlling incremental technical debt would be prudent.

This change might be less challenging than it sounds. With respect to endogenous incremental technical debt, the teams that incurred it are either still at work, or just recently dispersed. Their understanding of the incremental technical debt is still fresh in their minds. If their projects are still underway, and if budget and schedule permit, retiring the incremental technical debt in the context of those projects is a superior strategy. For projects that have already delivered their work, a less preferable but still practical approach involves re-assembling some of the team. They then work to retire the incremental technical deb, while memories are still fresh.

For incremental exogenous technical debt, the team that was active when the debt formed might have little knowledge of how the debt came to be. In those cases, re-assembling the team provides little advantage. Specialized knowledge of the technical debt might prove more helpful in devising a strategy for retiring it.

For the most part, the problem of controlling incremental technical debt isn’t a technical one. It usually reduces to a problem of finding time and resources to undertake the task.

Why resources aren’t available to retire incremental technical debt

The immediate reason why most teams don’t have enough resources to retire their incremental technical debt is that the organization, as a whole, doesn’t plan for retiring incremental technical debt incrementally. This immediate reason, though, isn’t fundamental. The lack of resources is a symptom of deeper dysfunctions in the organization. The real question is this: Why do so many organizations fail to allocate time and resources to retire incremental technical debt incrementally? Here are three reasons.

Misunderstanding (or no understanding) of the concept of technical debt

The organization is unlikely to be able to manage any kind of technical debt unless its people understand the concept. They must understand that technical debt isn’t necessarily the result of engineering malpractice. For example, much technical debt arises as a natural result of working with technology. Another example: it can be a result of organizational forms that compel people to behave in ways that lead to technical debt. Unless the people of the organization accept these truths, allocating sufficient resources to managing incremental technical debt is unlikely.

Not appreciating the MICs

Decisions regarding technical debt management ultimately reduce to a choice between allocating precious resources to technical debt retirement, and allocating them elsewhere. To make this choice responsibly, it’s necessary to fully appreciate the cost of carrying technical debt. Most believe that these costs appear in the form of lost engineering productivity. While that is indeed a factor, other factors can be far more important.

For example, if entry into an important market is delayed by even as little as 30 days due to debt-depressed engineering productivity, the financial consequences can be enormous and insurmountable. Or delays in diagnosing and repairing a fault in a product can produce financial liabilities that can actually sink the company. When one considers all possible financial consequences of carrying technical debt, it becomes clear that managing technical debt effectively is actually a strategy for survival. The decision to allocate appropriate resources to incremental technical debt retirement does require modeling these costs—calculations that few organizations actually undertake.

Miscalculating projections of returns on investments

Failing to estimate MICs with sufficient precision is problematic because it reduces the quality of decisions regarding short-term resource allocations. But it also affects long-term projections, which depend on estimating returns on investments. For example, to choose between investing in retiring incremental technical debt from an asset and investing in new capabilities for the same asset, one must compare the projected value of each choice. The investment decision could be biased in favor investing in new capabilities for many reasons. For example, the decision maker’s understanding of technical debt might be deficient. Another example: the calculations of MICs might be incomplete or underestimated. Incremental technical debt retirement might therefore be systematically deferred or avoided altogether.

The widespread belief that MICs consist largely of lost productivity of engineers almost ensures a dramatic underestimate of MICs.

Policy recommendations for controlling incremental technical debt

The simplistic approach to controlling incremental technical debt is to provide more money to projects and to engineering functions. While that approach will be somewhat helpful, its results will likely be disappointing when compared to approaches that combine resource augmentation with changes in enterprise policy, processes, and culture.

Let’s begin with an example of a needed cultural change. Nearly anyone who makes or influences decisions might occasionally bear some responsibility for incurring incremental technical debt. To achieve effective control of technical debt requires that everyone understand how to change their behavior. Any guiding principle must be simple to state and easy to understand, because we must communicate it to nearly everyone. Here’s a sample statement of a useful such a principle:

Those whose decisions lead to new technical debt are accountable for securing the resources needed to retire that debt. They might also be accountable for supplying compensating resources to those within the enterprise, or among its customers, who are negatively affected. For example, those negatively affected might suffer depressed operational effectiveness during the period in which that technical debt is outstanding.

I call this the Principle of Accountability. It’s a corollary to what Weinberg calls “Ford’s Fundamental Feedback Formula” [Weinberg 1985], which captures the idea that people make better choices when they must live with the consequences of those choices.

General guiding principles are necessary, but not sufficient. Here are five examples of changes that help in controlling incremental technical debt [Brenner 2017b].

1. Adopt a shared concept vocabulary

There must be general agreement among all parties about the meanings of concepts that relate to incremental technical debt formation.

Examples: the definitions of “done” vis-à-vis projects, strategic technical debt, reckless technical debt, unethical technical debt, exogenous-technical-debt, endogenous technical debt, MICs, MPrin, and more. This material must be available in an education program and a new-employee orientation program.

2. Accept that technical debt is a fact of technological life

There is a widespread belief that most technical debt results from engineering malpractice. Although some technical debt does arise this way, most does not. For examples of other causes, see “Nontechnical precursors of nonstrategic technical debt.” Some technical debt arises because of advances external to the enterprise, beyond its control. Development-induced or field-revealed discoveries are especially difficult to avoid. In many instances, technical debt is an inevitable result of using technology.

3. Track the cost of carrying technical debt

The cost of retiring a particular class of technical debt (its MPrin) is significant only in the context of planning or setting priorities for resource allocation. In all other contexts, knowing that cost has little management value. What does matter, at all times, is the cost of carrying that technical debt—the MICs, or metaphorical interest charges. (See “The Principal Principle: Focus on MICs.”) MICs can fluctuate wildly [Garnett 2013].

Build and maintain expertise for estimating and tracking the costs of incurring and carrying each class of technical debt. Know how much each kind of technical debt contributes to these costs, now and for the next few years.

4. Assign accountability for kinds of technical debt

Some kinds of incremental technical debt result from actions (or inactions) within the enterprise. Some do not. To control the incremental technical debt that arises from internal causes, hold people accountable for the debt their actions generate. Use Fowler’s Technical Debt Quadrant [Fowler 2009] as the basis for assessing and distributing internal financial accountability. It’s useful for both debt retirement costs (MPrin) and metaphorical interest charges (secured technical debt is technical debt for which resources have been allocated (possibly in a forward time period) to guarantee the debt’s retirement and possibly its associated MICs.

This policy implies that deliberately incurred technical debt, whether incurred strategically or recklessly, must be secured. If anyone involved feels that technical debt has been incurred, a dispassionate third party reviews project deliverables for the presence of technical debt. However, allocating resources might require securing commitments of resources for future fiscal periods. For many organizations, such forward commitments might require modifying the management accounting system.

Last words

Controlling incremental technical debt requires changes well beyond the behavior and attitudes of engineering staff, or the technologies they employ. Achieving control of incremental technical debt formation requires engagement with enterprise culture to alter the behavior and attitudes of most of the people of enterprise.

References

[Brenner 2017b] Richard Brenner. “Managing Technical Debt: Nine Policy Recommendations,” Cutter Consortium Executive Update 18:4, 2017.

Available: here; Retrieved: December 29, 2017

Cited in:

Controlling incremental technical debt

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Garnett 2013] Steve Garnett, “Technical Debt: Strategies & Tactics for Avoiding & Removing it,” RippleRock Blog, March 5, 2013.

Available: here; Retrieved February 12, 2017.

Cited in:

Controlling incremental technical debt

[Weinberg 1985] Gerald M. Weinberg. The Secrets of Consulting. New York: Dorset House, 1985.

Ford’s Fundamental Feedback Formula. Order from Amazon

Cited in: