Last updated on December 11th, 2018 at 07:11 am
In my last post I provided a list of attributes of wicked problems [Rittel 1973], and listed the reasons why I feel that technical debt retirement projects can qualify as wicked problems. As a quick review, here are the attributes of a wicked problem as Rittel and Webber see them, rephrased for brevity:
- There is no clear problem statement
- There’s no way to tell when you’ve “solved” it
- Solutions aren’t right/wrong, but good/bad
- There’s no ultimate test of a solution
- You can’t learn by trial-and-error
- There’s no way to describe the set of possible solutions
- Every problem is unique
- Every problem can be seen as a symptom of another problem
- How you explain the problem determines what solutions you investigate
- The planner (or designer) is accountable for the consequences of trying a solution
A subset of wicked problems can be viewed as super wicked [Levin 2012]. Levin, et al. list the following four properties of super wicked problems. With each one, I’ve added my thoughts about how retiring technical debt can qualify as a super wicked problem.
Time is running out
Given the reality of anthropogenic climate change [Cook 2016], further increases in storm intensity and frequency are likely, and air bases located in vulnerable coastal regions are at risk [Phillips 2018b]. Because these bases were constructed during the period before the general recognition of anthropogenic climate change, they now constitute a technical debt. Relocating them would undoubtedly be a wicked problem, and possibly a super wicked problem. However, if federal policies continue to fail to account for anthropogenic climate change, those policies could prevent relocating these bases. If that happens, the policies themselves represent a technical debt. Changing federal policy to account for anthropogenic climate change is certainly a super wicked problem. U.S. Air Force photo by TSgt Ben Bloker, courtesy Wikipedia
The problem presents difficulty that has an inherent timescale. For example, many believe that climate change will become irreversible if it isn’t dealt with successfully within 30 years.
Technical debt retirement can have an inherent time scale, though it varies with the debt in question. For example, Microsoft ended mainstream support for Windows 7 in January 2015. At that point, by any definition, every computer running Windows 7 incurred a technical debt. Yet by September 2018, almost four years later, 46.7% of all computers running Windows were running Windows 7, and that number represented an increase from the previous month by 0.6% [Keizer 2018]. The safe-operability window for Windows 7 is closing fast. Well, to be honest, no faster than the calendar. But the end of standard support is January 2020, which represents the inherent timescale for this kind of technical debt. The consequences are much more serious than just converting to Windows 10. All applications running on those machines are also potentially reaching end of life, unless they operate correctly on Windows 10. And all users of converted machines will need to learn how to use the new Windows 10 OS and any applications that must be updated or replaced. So there’s also a training issue, a learning curve, and period of elevated user error rates to deal with, too.
Other technical debts might also have explicitly inherent timescales, of course. But for enterprises whose securities are publicly traded, all technical debt has a shared inherent timescale associated with the cost of retiring it. For any technical debt that must eventually be retired, if the cost of retirement is high enough to be noticed as a decline in net income in enterprise financial reports, the impact of debt retirement can be severe and negative. Letting debt remain in place, unaddressed and growing, can be a financially dangerous strategy. A safer strategy is to spread the cost of retirement across as many fiscal quarters as possible. For Windows 10 conversions, there are only about five fiscal quarters remaining for that purpose. For other technical debts, the number of fiscal quarters available for diluting the costs of retirement might be more—or less.
Those who cause the problem also seek to provide a solution
The phrase “seek to provide a solution” might be somewhat tactful. I expect that some super wicked problems have the property that those who cause the problem exert some degree of control over what kinds of solutions are acceptable, or even discussible. In many cases, this represents a conflict of interest that can prevent the organization from deploying the more effective options.
That conflict of interest is certainly present in the context of technical debt retirement projects. Technical debt formation and persistence are due, in part, to a failure to commit resources to retiring it, or, at least, to inhibiting its formation. That failure is the responsibility of those in leadership roles in the enterprise. Typically, these are the same people who must decide to commit resources to retire technical debt in the future.
The central authority needed to address it is weak or non-existent
Again, I find this description unnecessarily limiting. I would prefer a phrasing such as, “The central authority, for whatever reason, chooses to exert, or is unable to exert, its authority in furtherance of solution, or even investigation.” In other words, the central authority need not be weak for it to be a source of difficulty in addressing the super wicked problem. It need only choose not to act. This can happen when those who cause the problem are the people who constitute the central authority, or they capture the central authority, or they capture the function to which the central authority has delegated responsibility for solution.
With respect to technical debt retirement, consider this scenario. At AMUFC, A Made-Up Fictitious Corporation, the sales and marketing functions have repeatedly struggled with the engineering function for shares of budget resources. Engineering has argued repeatedly, and unsuccessfully, that it needs additional resources to address the technical debt that has accumulated in several products. But the CEO is a former VP Sales, and a close friend of the CFO. Together, they have always decided to defer technical debt retirement in favor of new products and enhancements favored by the VP Marketing, by customers, and by investors.
Scenarios like this are common. Enterprise leadership is strong, but not inclined to address the technical debt retirement issue.
Partly as a result, policy responses discount the future irrationally
Irrational discounting of future costs and benefits occurs when policies are deployed that give too much emphasis to producing short-term benefits and/or to avoiding short-term costs or inconveniences. Benefits are pulled in from the future towards the present; costs and inconveniences are pushed out toward the future and deferred. One form of this discounting scheme—one of many—is hyperbolic discounting.
This tendency is one way of distracting attention from the actual problem. It is the principal tactic that enables the persistence of technical debt, and the means by which enterprises repeatedly defer attention to the problem of retiring technical debt.
Both the problem of managing technical debt and the problem of designing technical debt retirement projects, exhibit all of these properties to some degree. It’s likely, in my view, that these problems are super wicked problems.
Intervention strategies for super wicked problems
Levin, et al., recommend four distinct strategies for resolving super wicked problems. They are all approaches to devising policies that are difficult to alter, thus committing the organization to a particular path forward.
Exploit lock-in
Lock-in is usually regarded as dysfunctional adherence to a strategy or course of action despite the existence of superior alternatives [Brenner 2011]. It occurs when a policy confers some kind of immediate benefit on a subset of the population. If that benefit is significant, if the population subset would be harmed by alterations of the policy that remove the benefit, and if the subset has enough political power to defend the benefit, the policy will be “locked in” and thus difficult to change. Levin, et al., suggest that this phenomenon can serve a beneficial purpose by protecting a constructive policy, thus preventing its abandonment.
Most technical debt retirement efforts are explicitly designed to focus solely on retiring the debt, with all (or most) of the benefit appearing in the form of increased engineering productivity, decreased sources of frustration for engineers, or increased engineering agility. Benefits for non-engineering stakeholders tend to be indirect. To establish policies that exploit lock-in we must craft them so that they provide ongoing, direct benefit to the most politically powerful stakeholders. For example, addressing first the forms of technical debt that are most likely to lead to product innovations that non-engineering stakeholders would value highly could cause those stakeholders to favor further technical debt retirement efforts.
Positive feedback
This strategy makes policies durable when people or organizations already supporting the policy derive some kind of increased benefit, leading to others not yet supporting or covered by the policy to decide to support it. This mechanism is sometimes known as a “network effect.” When network effects are present, the value of a product or service increases as the size of the population using it increases [Shapiro 1998].
To exploit network effects when devising technical debt retirement efforts, focus on retiring the kinds of technical debts that confer benefits on stakeholders of platform assets. A platform asset is an asset that supports multiple other assets. Examples: an application development tool suite, a product line architecture, or an enterprise data network. Platform assets that support collaboration communities are more likely to generate network effects.
Increasing Returns
Policies and interventions that enable increasing returns to the population are more likely to be durable than those that offer steady returns. Because people adapt to steady levels of stimuli, policies that produce a change in the context only during the period immediately following initial adoption of those policies are less likely to maintain popular support than are policies that continue to provide increasing returns as long as they’re in place.
But among policies that provide increasing returns, Levin, et al., identify two types. Type I policies, which are less durable, confer their benefits on an existing population of supporters. Type I policies don’t cause others to become supporters. Type II policies also confer benefits on their supporters, but they do cause others to become supporters. Type II policies are thus far more durable than are Type I, because they foster growth in the supporting population.
Framing technical debt retirement projects as individual projects with the objective of retiring a specified kind of technical debt is likely to lead to the enterprise population viewing the effort as a Type I policy at best. But framing each project as a phase of a longer-term effort could position the larger effort as a Type II policy, if the larger effort is designed to affect increasing portions of the enterprise population.
Self-reinforcing
A self-reinforcing policy is a policy that creates a dynamic that causes that policy to be maintained. Reinforcement can come about either as a result of increases in the benefits the policy generates, or by causing increases in the cost of rescinding the policy, or a combination of both effects. As with the strategy of Increasing Returns, there are two types of self-reinforcing policies. Type I self-reinforcing policies focus on maintaining support for the policy within the subset of the population consisting of its original supporters. In analogy with Type II Increasing Returns policies, Type II Self-reinforcing policies affect both the original supporting population and portions of the population not yet affected directly by the policy.
To exploit self-reinforcement, technical debt retirement programs must emphasize retiring debts that have curtailed organizational agility in recognizable ways, or which have prevented introduction of capabilities that the population values. Communicating these objectives is an important part of the program, because self-reinforcing popular support is possible only if the population understands the strategy and how it benefits the enterprise.
Because of the essential uniqueness of any wicked problem (Proposition 7 of Rittel and Webber), it is futile to attempt to apply as a template any retirement program that worked for some other organization, or for some other portion of a given organization at an earlier time with a different form of technical debt. But these four strategies, implemented carefully and communicated widely and effectively within the organization, can build organizational commitment to a long-term technical debt retirement program, even though retiring technical debt may be a super wicked problem.
References
[Brenner 2011] Richard Brenner. “Indicators of Lock-In: I,” Point Lookout 11:12, March 23, 2011.
Available: here; Retrieved: October 23, 2018.
[Cook 2016] John Cook, Naomi Oreskes, Peter T. Doran, William R.L. Anderegg, Bart Verheggen, Ed W. Maibach, J. Stuart Carlton, Stephan Lewandowsky, Andrew G. Skuce, Sarah A. Green, Dana Nuccitelli, Peter Jacobs, Mark Richardson, Bärbel Winkler, Rob Painting, and Ken Rice. “Consensus on consensus: a synthesis of consensus estimates on human-caused global warming,” Environmental Research Letters 11, 2016, 048002.
Available: here; Retrieved: October 23, 2018
[Gabriel 2018] Melissa Gabriel. “Hurricane Michael: Fate of costly stealth fighter jets at Tyndall Air Force Base still unknown,” USA Today: Pensacola News Journal, October 17, 2018.
Available: here; Retrieved: October 23, 2018
[Keizer 2018] Gregg Keizer. “Windows by the numbers: Windows 10 backtracks, Windows 7 remains resilient,” Computerworld, October 2, 2018.
Available: here; Retrieved: October 18, 2018
[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.
Available: here; Retrieved: October 17, 2018
[Phillips 2018a] Dave Phillips. “Tyndall Air Force Base a ‘Complete Loss’ Amid Questions About Stealth Fighters,” The New York Times, October 11, 2108.
Available: here; Retrieved: October 23, 2018
[Phillips 2018b] Dave Phillips. “Exposed by Michael: Climate Threat to Warplanes at Coastal Bases,” The New York Times, October 17, 2108.
Available: here; Retrieved: October 23, 2018
[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.
Available: here; Retrieved: October 16, 2018
- Glossary and Terminology
- Retiring technical debt can be a wicked problem
- Degrees of wickedness
- Nine indicators of wickedness
- Retiring technical debt from irreplaceable assets
- Legacy technical debt retirement decisions
[Shapiro 1998] Carl Shapiro and Hal R. Varian. Information rules: a strategic guide to the network economy. Harvard Business Press, 1998.
Other posts in this thread
- Managing technical debt
- Leverage points for technical debt management
- Undercounting nonexistent debt items
- Crowdsourcing debt identification
- Demodularization can help control technical debt
- Legacy debt incurred intentionally
- Metrics for technical debt management: the basics
- Accounting for technical debt
- Three cognitive biases
- The resilience error and technical debt
- Synergy between the reification error and confirmation bias
- Retiring technical debt can be a wicked problem
- Degrees of wickedness
- Retiring technical debt can be a super wicked problem
