Tag: schedule pressure

Posted on

Using SMART goals for technical debt reduction

Last updated on July 8th, 2021 at 01:06 pm

Attempting to reduce technical debt by setting so-called “SMART goals” in the obvious way can often disappoint. SMART, due to George T. Doran [Doran 1981], is widely used for expressing management goals. “SMART” is an acronym for “Specific, Measurable, Attainable, Realistic, and Time-boxed.” The last three words are available in various alternative ways. Doran himself used “assignable, realistic, and time-related.”

SMART is deeply embedded in management culture. Many assume without investigation that expressing technical debt goals using the SMART pattern will produce desired results. Also embedded in management culture is the aphorism, “You get what you measure.” [Ariely 2010]  [Bouwers 2010] A typical technical debt reduction goal: “Reduce technical debt by 20% per year for the next five years.”

SMART goals in their simplest form are ineffective for technical debt

Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives
Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives. Watch a 2010 interview of Prof. Doran at YouTube.
There’s ample support for a claim that applying the SMART technique in direct ways will be ineffective. Much employee behavior affects technical debt indirectly. It can overwhelm the effects of employee behaviors that affect technical debt indirectly. The direct approach does cause some employees to adopt desirable behaviors. But their impact isn’t significant enough compared to the effects of the behaviors that affect technical debt indirectly. Employees who see little connection between their own activities and the burden of technical debt can unwittingly have enormous impact. Moreover, many are subject to competing constraints on their behaviors that then cause them to act in ways that increase technical debt.

That’s why it’s necessary for management to develop a series of SMART goals that affect behaviors that have indirect effects on technical debt. In the first part of this post, “Setting a direct SMART goal for technical debt reduction is problematic,” I explore the problems inherent in the direct approach. In the second part, “How to set SMART goals for technical debt,” I provide examples of SMART goals that touch on behaviors that have indirect effects on technical debt.

Setting a direct SMART goal for technical debt reduction is problematic

Let’s begin by exploring some of the problems with the direct approach. In this section, I assume that management has set a SMART goal for the enterprise in the form, “Reduce technical debt by 20% for each of the next five years.” But there’s nothing special about the numbers. My comments below apply to the form of the goal, rather than the specific numbers.

The direct approach assumes measurability

To attain a goal of a 20% reduction in technical debt in a given year, we must be able to measure the level of technical debt. We measure it at the beginning of the year and at the end of the year. Presumably we do so with confidence in the 90% range or better. Such a measurement with the precision required might not be possible. Moreover, in most cases the probability that such a measurement is possible is low. For these reasons, setting periodic goals for total technical debt isn’t a useful management tool.

Consider a simple example. One common form of technical debt is missing or incompletely implemented capability. In some instances, the metaphorical principal (MPrin) of a given instance of this debt in the current year can change spontaneously to a dramatically larger value in the following year (or even the following week). This can happen due to changes in the underlying asset unrelated to the technical debt. Ot it can happen due to debt contagion. Or it can happen due to any number of other reasons. When this happens, the technical debt retirement effort for that year can appear to have suffered a serious setback. Setbacks like this can happen even though the technical debt retirement teams have been performing perfectly well.

The direct approach assumes a static principal

With most financial debts, a loan agreement sets the principal amount. Moreover, we can compute the principal at any time given the mathematical formulas specified in the loan agreement.

By contrast, in many cases, the metaphorical principal amount of a technical debt might be neither fixed nor readily computable. We can estimate the MPrin of a given kind of technical debt at a given time, and we can even make forward projections. But they are merely estimates, and their error bars can be enormous. See “Policy implications of the properties of MPrin” and “Useful projections of MPrin might not be attainable.”

The direct approach focuses on MPrin, not MICs

Objectives expressed in terms of the volume of technical debt—the total MPrin—are at risk of missing the point. Total MPrin isn’t what matters most. What matters is MICs—the total cost of carrying the debt. Even more important is the timing of arrival of the MICs. See “The Principal Principle: Focus on MICs.”

And like MPrin, MICs can vary in wild and unpredictable ways. For example, the MICs for a piece of technical debt borne by an asset that isn’t undergoing maintenance or enhancement can be zero; in a later time period, when that asset is undergoing enhancement, the MICs can be very high indeed. See “MICs on technical debt can be unpredictable” for a detailed discussion.

Priority setting for technical debt retirement is most effective when it accounts for the timing of MICs. For example, suppose we know that we must enhance a particular asset by FY21 Q3. And suppose we know that it bears technical debt that adds to the cost of the enhancement. Then retiring that debt in FY20 would be advisable. But if that technical debt has zero MICs for the foreseeable future, retiring it might not be worth the effort.

The direct approach fails to distinguish legacy technical debt from incremental technical debt

Unless policies are already in place governing the formation of incremental technical debt, technical debt retirement programs might encounter severe difficulty. New development and maintenance and enhancement of existing assets are ongoing. They generally produce technical debt in various forms. The technical debt retirement program might simply be unable to keep up with new debt formation.

The direct approach fails to anticipate the formation of enterprise-exogenous technical debt

Technical debt can sometimes form as a result of innovations, changes in standards, or changes in regulations that occur entirely external to the enterprise. I call such technical debt enterprise-exogenous. When this happens, the technical debt retirement effort can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well. Before initiating a technical debt reduction program, it’s wise to first deploy a program that’s capable of retiring technical debt at a pace that at least equals the pace of formation of enterprise-exogenous technical debt.

Incurring technical debt is sometimes the responsible thing to do

At times, incurring technical debt is prudent. In these situations, accepting the debt you’ve incurred—even for the long term—might be appropriate. Strict goals about total technical debt can lead to reluctance to incur debt that has a legitimate business purpose. To prevent this, goals for total technical debt must be nuanced enough to deal with these situations. Goals for total technical debt that adhere strictly to the SMART goal pattern sometimes lack the necessary level of nuance.

How to set SMART goals for technical debt

SMART goals can work for technical debt management, but we must relate them to behavioral choices. Here are some examples of SMART goals that can be effective elements of a technical debt management program. Some of these examples are admittedly incomplete. For example, I offer no proof of assignability, attainability, or realism. Such attributes can vary from organization to organization. And we must allocate the goal in question across multiple organizational elements in ways peculiar to the organization.

At least 30% of incremental technical debt will be secured technical debt at the end of Year 1; 60% by the end of Year 2

Incremental technical debt is technical debt that’s incurred in the course of work currently underway or just recently completed. Because it’s so well understood, its MPrin can be estimated with higher precision than is usually possible with legacy technical debt. That precision is needed for defining the collateral and resources used to secure the debt.

A secured technical debt, like a secured financial debt, is one for which the enterprise reserves the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. Beyond financial resources, they might include particular staff, equipment, test beds, and downtime. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes.

Within one year, at least 50% of all incremental technical debt will be retired within one year of its origination; 70% within 18 months

This goal also exploits the fact that we can estimate incremental technical debt with relatively high precision. As a goal, it builds on the goal above by requiring that the organization actually expend as intended the resources pledged to retire incremental debts.

Within one year, all engineers and their direct supervisors will be educated in basic technical debt concepts

The educational materials will be developed in the next five months and piloted with 10% of the technical staff within seven months. The material will include an online proficiency test that 90% of engineers will have successfully passed within a year.

Within one year, 90% of project plans will include projections of the MPrin of the incremental technical debt they expect to generate for each delivery cycle

This information is useful for making forward projections of resources needed to secure incremental technical debt. Tracking the accuracy of these projections helps project planners improve their estimates.

Within one year, initiate a practice of identifying the top five forms of legacy technical debt, ranked by the volume of the contagion

Debt contagion is the propagation of a given form of technical debt by creating new system elements or assets in forms compatible with elements already identified as technical debt. By examining the body of incremental technical debt created enterprise-wide in a given time period (say, by fiscal quarter), we can determine the portion of that incremental debt that results from contagion, for each type of contagious legacy technical debt. This data is needed to identify the most contagious forms of legacy technical debt. They are prime candidates for debt retirement.

Within one year, initiate an industrial intelligence practice that is responsible for projecting the formation of enterprise-exogenous technical debt

This group must have a sophisticated grasp of the technologies in use within the enterprise that already bear enterprise-exogenous technical debt, or which could be subject to the formation of enterprise-exogenous technical debt. Its responsibilities cover enterprise products and services, as well as enterprise infrastructure. It issues advisories as needed, and an annual forecast. The group is also responsible for recommending and monitoring participation in industrial standards organizations. The group reports to the CIO or CTO.

References

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

Other posts in this thread

Posted on

How budget depletion leads to technical debt

Last updated on July 8th, 2021 at 01:38 pm

Some projects undergo budget depletion exercises after budget cuts. Or the exercises might occur when there’s evidence that the funds remaining won’t cover the work remaining. Formats vary, but the typical goal of these exercises is downscoping. We remove, relax, defer, or suspend some requirements. With limited funds, we execute downscoping in a manner that leads to technical debt.

A physical example

The Old River Control Complex on the Mississippi River
Photo courtesy USACE
The accompanying photo shows the Old River Control Complex on the Mississippi River. The US Army Corps of Engineers (USACE) built it and operates it. It controls the flow from the Mississippi into the Atchafalaya River, a distributary. The Mississippi would otherwise have rerouted itself into the Atchafalaya, which has a steeper gradient to the ocean. Since that would have deprived New Orleans and its industrial facilities of water and navigational channels, USACE maintains flow control facilities.

The industrial facilities of the lower Mississippi constitute a technical debt. Their existence is no longer compatible with the “update” Nature is trying to deploy. But our national budget won’t support repositioning New Orleans and its industrial facilities. So we redirect the flow of water from Nature’s course to one more compatible with the industrial base. The Old River Control Complex, with levees, dredging projects, and gates throughout lower Louisiana, are the MICs we pay for the technical debt that is the outdated position of New Orleans and its industrial base. For more about Atchafalaya, see the famous New Yorker article by John McPhee [MacFee 1987].

A broad array of effects

Here’s an illustrative scenario. At the time downscoping begins, the work product might contain incomplete implementations of items that are due for removal from the list of objectives. This removal renders unnecessary a set of accommodations contained in surviving artifacts. They comprise a most insidious type of debt that’s difficult to detect. It’s difficult to detect because the affected system components appear to be merely overly complicated. Recognizing them as a residual of a cancelled capability requires knowledge of their history. Unless we document these artifacts at the time of the downscoping, that knowledge may be lost.

Other items of technical debt that arise from budget depletion include tests that no longer serve a purpose, or documentation that’s no longer consistent with the rest of the work product, or user interface artifacts no longer needed. When budgets become sufficiently tight, funds aren’t available for documenting these items of technical debt as debt. The enterprise might then lose track of them when team members move on to other work.

Sometimes, budget depletion takes effect even before the work begins. This happens, for example, when project champions unwittingly underestimate costs to gain approval for the work they have in mind. The unreasonableness of the budget becomes clear soon after the budget approval, and its effects take hold soon thereafter.

Budget depletion can also have some of the same effects as schedule pressure. When the team devises the downscoping plan, it must make choices about what to include in the revised project objectives. In some cases, the desire to include some work can bias estimates of the effort required to execute it. If the team underestimates the work involved, the result is increased pressure to perform that work. With increased pressure comes technical debt. See “With all deliberate urgency” for more.

Last words

A policy that could limit technical debt formation in response to budget depletion would require identifying the technical debt such action creates, and later retiring that debt. Because these actions do require resources, they consume some of the savings that were supposed to accrue from downscoping. In some cases, they could consume that amount in its entirety, or more. Most decision makers probably over-estimate the effectiveness of the downscoping strategy. Often, it simply reduces current expenses by trading them for increased technical debt, which raises future expenses and decreases opportunities in future periods.

References

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

Other posts in this thread

Posted on

Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma

Last updated on July 8th, 2021 at 01:31 pm

In a 1977 report, Daniel Kahneman and Amos Tversky identify one particular cognitive bias [Kahneman 2011], the planning fallacy, which afflicts planners [Kahneman 1977] [Kahneman 1979]. They discuss two types of evidence planners use. Singular evidence is specific to the case at hand. Distributional evidence is specific to similar past efforts. The planning fallacy is planners’ tendency to pay too little attention to distributional evidence and too much to singular evidence. They do this even when the singular evidence is scanty or questionable. Failing to harvest lessons from the distributional evidence, which is inherently more diverse than singular evidence, the planners tend to underestimate cost and schedule. So there’s a tendency to promise lower costs, faster delivery, and greater benefits than anyone can reasonably expect.

Enter the n-person prisoner’s dilemma

Boehm et al. [Boehm 2016] describe a dynamic that exacerbates the problem. They observe that because organizational resources are finite, project champions compete with each other for resources. This competition compels them to be unrealistically optimistic about their objectives, costs, and schedules. Although Boehm et al. call this mechanism the “Conspiracy of Optimism,” possibly facetiously, it isn’t actually a conspiracy. Rather, it’s a variant of the N-Person Prisoner’s Dilemma [Hamburger 1973].

A special property of pressure-induced debt

Hoover Dam, aerial view, September 2017
Hoover Dam, aerial view, September 2017. Under construction from 1931 to 1936, the cost of the dam was $48.8M ($639M in 2016 dollars) under a fixed-price contract. It was complete two years early. Apparently the planning fallacy doesn’t act inevitably. 112 men died in incidents associated with its construction. 42 more died of a condition diagnosed as pneumonia, but which experts now believe to have been carbon monoxide poisoning due to poor ventilation in the dam’s diversion tunnels during construction. There’s little doubt that unrealistic optimism affects more than budget and schedule projections. It also affects risk projections, including deaths. Photo (cc) Mariordo (Mario Roberto Durán Ortiz), courtesy Wikimedia Commons.
Unrealistic optimism creates budget shortfalls and schedule pressures. In turn, they both contribute to conditions favorable for creating nonstrategic technical debt. And this mechanism, or any mechanism associated with schedule or budget pressure, tends to produce technical debt that’s subtle—it’s the type least likely to become evident in the short term. For example, technical debt that might make a particular enhancement more difficult in the next project is more likely to appear than technical debt that creates trouble in the current effort. Debt that creates trouble in the current effort is more likely to be retired in the short term, if not in the current effort. Awkward architecture might be more difficult to identify. It’s therefore more likely to survive in the intermediate or long term.

The bad news of schedule pressure

In other words, the technical debt most likely to be generated is that which is the most benign in the short term, and which is therefore more likely to escape notice. If noticed, it’s more likely to be forgotten unless carefully documented. And that action is unlikely under schedule and budget pressure. In this way, the nonstrategic technical debt created as a result of unrealistic optimism is more likely than most technical debt to eventually become legacy technical debt.

Last words

Policymakers can assist in addressing the consequences of unrealistic optimism by advocating for education about it. They can also advocate for changes in incentive structures and performance management systems. It’s good business to establish organizational standards with respect to realism in promised benefits, costs, and schedules.

References

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

Other posts in this thread

Show Buttons
Hide Buttons