policy Archives | Technical Debt for Policymakers

May 3, 2018July 9, 2021

Adopt an enterprise-wide definition of technical debt

Last updated on July 9th, 2021 at 11:25 am

Because effective technical debt management requires cooperation from almost everyone, an enterprise-wide definition of technical debt is essential. Absent a shared definition of technical debt, controversy can develop. Controversy is especially likely among those who have previously encountered the concept—namely, among technologists. Policymakers can make invaluable contributions to the design of the cultural transformation that will enable control of technical debt.

A physical expression of shared commitment, essential to adopting an enterprise-wide definition of technical debt — A physical expression of shared commitment. Effective technical debt management requires both a shared understanding of what it is and a shared commitment to do what’s required to control it.

Li et al. [Li 2015] found that defining what is and what isn’t technical debt remains an open question in software engineering. Even if we restrict the discussion to software constructed in-house, opinions about what constitutes technical debt do differ. The authors found that in the literature of technical debt, “The term ‘debt’ has been used in different ways by different people, which leads to ambiguous interpretation of the term.”

This finding is perhaps the most significant for policymakers. It suggests that controlling technical debt might require forging an organizational consensus about the meaning of the term technical debt. The people of most organizations come from many different backgrounds. Those with little knowledge of technical debt have few preconceptions. But those who are aware of the issue probably interpret the term technical debt in a variety of ways. Because some of those who do have awareness of the term are likely to have strong opinions about its meaning, one can anticipate a need to resolve these differences.

The effect of an absence of standards

Some technical terms, like RAID, byte, compiler, and kilowatt, have widely accepted standard definitions. Although the term technical debt has found wide use, it has no standard definition. What some people categorize as technical debt, others do not. Those who are accustomed to working with terms that have precise, widely accepted definitions might tend to assume that the term technical debt does have (or should have) one as well. This assumption can create some difficulty for people who don’t realize that others might have differing views of the definition of the term.

For example, there are those who believe that defects are not technical debt. And some believe that no element of a technological asset can constitute technical debt unless it is part of a product that a customer uses. Our definition differs with both of these views.

Last words

Policymakers must be aware that there is a lack of consensus about the definition of technical debt. Our definition, crafted specially for the use of policymakers, might seem unusually broad to technologists and engineers. For that reason alone, it’s advisable to become familiar with the various ways technologists use the term. Understanding their perspective is essential to formulating sound policy deserving of their respect.

References

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

April 25, 2018July 11, 2021

Malfeasance can lead to new technical debt

Last updated on July 11th, 2021 at 03:03 am

Elizabeth Holmes backstage at TechCrunch Disrupt San Francisco 2014. She was the founder, chairman, and CEO of Theranos, a startup that grew to a total valuation of $9 billion in 2015, and has since dramatically declined in value, now on the edge of its second bankruptcy. Theranos, through Holmes, claimed to have developed technology enabling blood testing with small amounts of blood. Theranos’s process supposedly required only 0.1% to 1% of the amount of blood conventional technologies require. These claims proved false. After a series of collisions with U.S. government agencies, the U.S. Securities and Exchange Commission sued Holmes and Theranos. In March 2018, a settlement was reached in which Holmes accepted severe financial penalties, loss of voting control of Theranos, and a ban from serving as an officer or director of any public company for ten years. Photo (cc) Max Morse for TechCrunch.

Although creating and deploying policies to manage technical debt is necessary, it isn’t always sufficient for achieving control. Even if training and communication programs are effective, intentional circumvention of technical debt management policy remains possible. Malfeasance can lead to new technical debt by circumventing any policy. And malfeasance can be an obstacle to retiring—or even identifying—existing technical debt. Moreover, indirect effects of forms of malfeasance seemingly unrelated to technical debt can incur technical debt or extend the lifetime of existing technical debt.

Examples of how malfeasance can lead to technical debt

Consider an example from software engineering. To save time, an engineer might intentionally choose a deprecated approach. When the malfeasance comes to light, a question naturally arises. Specifically, in what other places has this individual (or other individuals) been making such choices? In a conventional approach to controlling this form of technical debt, we might examine only the engineer’s current work. But a more comprehensive investigation might uncover a trail of malfeasance in the engineer’s previous assignments.

Allman relates a hardware-oriented example [Allman 2012]. He describes an incident involving the University of California at Berkeley’s CalMail system. It failed catastrophically in November 2011, when one disk in a RAID (Redundant Array of Inexpensive Disks) failed due to deferred maintenance. Allman regards this incident as traceable to the technical debt consisting of the deferred RAID maintenance. While this particular case isn’t an example of malfeasance, it’s reasonable to suppose that some decisions to defer maintenance on complex systems are arguably negligent.

History provides many clear examples of how malfeasance can lead to new technical debt indirectly. Consider the Brooklyn Bridge. Many of the suspension cables of the bridge contain substandard steel wire, which an unscrupulous manufacturer provided to the bridge constructors. When the bridge engineer discovered the malfeasance, he recognized that he couldn’t remove the faulty wire that had already been installed. So he compensated for the faulty wire by adding additional strands to the affected cables. For more, see “Nontechnical precursors of nonstrategic technical debt.”

What kinds of malfeasance deserve special attention and why

Malfeasance that leads to incurring technical debt or which extends the life of existing technical debt can have dire consequences. It has the potential to expose the enterprise to uncontrolled increases in operating expenses and unknown obstacles to revenue generation. The upward pressures on operating expenses derive from the MICs associated with technical debt. Although MICs can include obstacles to revenue generation, considering these obstacles separately helps to clarify of the effects of malfeasance.

Why malfeasance deserves special attention

Malfeasance deserves special attention because the financial harm to the enterprise can dramatically exceed the financial benefit the malfeasance confers on its perpetrators. This property of technical-debt-related malfeasance is what makes its correction, detection, and prevention so important.

For example, when hiring engineers, some candidates claim to have capabilities and experience that they do not possess. Once they’re on board, they expose the enterprise to the risk of technical debt creation through substandard work. That work can escape notice for indefinite periods. The malfeasance here consists of the candidate’s misrepresentation of his or her capabilities. Although the candidate, once hired, does receive some benefit arising from the malfeasance, the harm to the enterprise can exceed that benefit by orders of magnitude.

As a second example, consider the behavior of organizational psychopaths [Babiak 2007] [Morse 2004]. Organizational psychopathy can be a dominant factor to technical debt formation when the beneficiary of a proposal is the decision maker. An alternative beneficiary, just as harmful, is the advocate who takes credit for the short term effects of the decision. In either case, the beneficiary intends knowingly to move on to a new position or to employment elsewhere before the true long term cost of the technical debt becomes evident. This behavior is malfeasance of the highest order. And although it’s rare, its impact can be severe. For more, see “Organizational psychopathy: career advancement by surfing the debt tsunami.”

What’s required to control malfeasance

When a particular kind of malfeasance can incur technical debt or extend the life of existing technical debt, it merits special attention. Examples like those above suggest three necessary attributes of technical debt management programs that deal effectively with malfeasance.

Corrective measures

The organization can undertake corrective measures in a straightforward manner when inadvertent policy violations occur. For example, a technical debt retirement program might encounter unexpected difficulties in setting priorities when individual performance metrics conflict with the technical debt control program. Such conflicts can be inadvertent and collaborative resolution is feasible, if challenging.

But with regard to malfeasance, difficulties arise when policy violations come to light. When the violations are intentional, corrective action usually entails investigation of the means by which the infraction was achieved, and how it was concealed. When these activities involve many individuals attached to multiple business units, we need some means of allocating the cost of corrective action. Allocating the cost of corrections can also be difficult when one party has reaped extraordinary benefits by taking steps that led to incurring significant technical debt. In some cases, corrective measures might include punitive actions directed at individuals.

Detection measures

When intentional violations are covert, or those who committed the violations claim that they’re unintentional, only investigation can determine whether a pattern of violations exists. Technical debt forensic activities require resources. They need rigorous audits and robust record-keeping regarding the decisions that led to the formation or persistence of technical debt. Automated detection techniques might be necessary to control the cost of detection efforts, and to ensure reliable detection.

Preventative measures

Successful prevention of policy violations requires education, communication, and effective enforcement. The basis of effective policy violation prevention programs includes widespread understanding of the technical debt concept and technical debt management policies. Most important, it includes the certainty of discovery of intentional infractions. These factors require commitment and continuing investment.

Policy frameworks are at risk of decreased effectiveness if they pay too little attention to malfeasance and other forms of misconduct. Such misbehavior deserves special attention because it’s often accompanied both by attempts to conceal any resulting technical debt. Worse, perpetrators often try to mislead investigators and managers about the debt’s existence. These situations do arise, though rarely, and when they do, they must be addressed in policy terms.

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

How budget depletion leads to technical debt

Last updated on July 8th, 2021 at 01:38 pm

Some projects undergo budget depletion exercises after budget cuts. Or the exercises might occur when there’s evidence that the funds remaining won’t cover the work remaining. Formats vary, but the typical goal of these exercises is downscoping. We remove, relax, defer, or suspend some requirements. With limited funds, we execute downscoping in a manner that leads to technical debt.

A physical example

The accompanying photo shows the Old River Control Complex on the Mississippi River. The US Army Corps of Engineers (USACE) built it and operates it. It controls the flow from the Mississippi into the Atchafalaya River, a distributary. The Mississippi would otherwise have rerouted itself into the Atchafalaya, which has a steeper gradient to the ocean. Since that would have deprived New Orleans and its industrial facilities of water and navigational channels, USACE maintains flow control facilities.

The industrial facilities of the lower Mississippi constitute a technical debt. Their existence is no longer compatible with the “update” Nature is trying to deploy. But our national budget won’t support repositioning New Orleans and its industrial facilities. So we redirect the flow of water from Nature’s course to one more compatible with the industrial base. The Old River Control Complex, with levees, dredging projects, and gates throughout lower Louisiana, are the MICs we pay for the technical debt that is the outdated position of New Orleans and its industrial base. For more about Atchafalaya, see the famous New Yorker article by John McPhee [MacFee 1987].

A broad array of effects

Here’s an illustrative scenario. At the time downscoping begins, the work product might contain incomplete implementations of items that are due for removal from the list of objectives. This removal renders unnecessary a set of accommodations contained in surviving artifacts. They comprise a most insidious type of debt that’s difficult to detect. It’s difficult to detect because the affected system components appear to be merely overly complicated. Recognizing them as a residual of a cancelled capability requires knowledge of their history. Unless we document these artifacts at the time of the downscoping, that knowledge may be lost.

Other items of technical debt that arise from budget depletion include tests that no longer serve a purpose, or documentation that’s no longer consistent with the rest of the work product, or user interface artifacts no longer needed. When budgets become sufficiently tight, funds aren’t available for documenting these items of technical debt as debt. The enterprise might then lose track of them when team members move on to other work.

Sometimes, budget depletion takes effect even before the work begins. This happens, for example, when project champions unwittingly underestimate costs to gain approval for the work they have in mind. The unreasonableness of the budget becomes clear soon after the budget approval, and its effects take hold soon thereafter.

Budget depletion can also have some of the same effects as schedule pressure. When the team devises the downscoping plan, it must make choices about what to include in the revised project objectives. In some cases, the desire to include some work can bias estimates of the effort required to execute it. If the team underestimates the work involved, the result is increased pressure to perform that work. With increased pressure comes technical debt. See “With all deliberate urgency” for more.

Last words

A policy that could limit technical debt formation in response to budget depletion would require identifying the technical debt such action creates, and later retiring that debt. Because these actions do require resources, they consume some of the savings that were supposed to accrue from downscoping. In some cases, they could consume that amount in its entirety, or more. Most decision makers probably over-estimate the effectiveness of the downscoping strategy. Often, it simply reduces current expenses by trading them for increased technical debt, which raises future expenses and decreases opportunities in future periods.

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

Confirmation bias and technical debt

Last updated on July 8th, 2021 at 01:35 pm

Confirmation bias is a cognitive bias [Kahneman 2011]. It’s the human tendency to favor and seek only information that confirms our preconceptions. It also causes us to avoid information that disconfirms our preconceptions. For example, the homogeneity of cable news channel audiences is a result of confirmation bias. Another result is the alignment between preconceptions of the audience and the slant of the newscast for that channel.

How confirmation bias can lead to technical debt

Third stage ignition, sending the Mars Climate Orbiter to Mars in December, 1998 — Computer-generated image of the third stage ignition, sending the Mars Climate Orbiter (MCO) to Mars in December, 1998. The spacecraft eventually broke up in the Martian atmosphere due to what is now called the “metric mix-up.” The Lockheed Martin team that constructed and programmed the MCO used Imperial units. But the team at JPL that was responsible for flying the MCO used metric units. After the loss of the MCO, an investigation led by NASA uncovered the mix-up.
One of the many changes resulting from this loss was increased use of reviews and inspections. We don’t know why reviews and inspections weren’t as thorough before the loss of the MCO as they are now. But one possibility is the effects of confirmation bias in assessing the need for reviews and inspections. Image courtesy Engineering Multimedia, Inc., and U.S. National Aeronautics and Space Administration

Confirmation bias causes technical debt by biasing the information on which decision makers base decisions involving technical debt. Most people in these roles have objectives they regard as having priority over eliminating technical debt. This causes them to bias their searches for information about technical debt. The bias favors information that would support directly the achievement of those primary objectives. Decisions tend, for example, to discount warnings of technical debt issues. They also tend to underfund technical debt assessments, and set aside advice regarding avoiding debt formation in current projects.

An example

For example, anyone determined to find reasons to be skeptical of the need to manage technical debt need only execute a few Google searches. Searching for there is no such thing as technical debt yields about 300,000 results at this writing; technical debt is a fraud about 1.6 million; and technical debt is a bad metaphor about 3.7 million. Compare this to technical debt which yields only 1.6 million. A skeptic wouldn’t even have to read any of these pages to come away convinced that technical debt is at best a controversial concept. This is, of course, specious reasoning, if it’s reasoning at all. But it does serve to illustrate the potential for confirmation bias to contribute to preventing or limiting rational management of technical debt.

Last words

Detecting confirmation bias in oneself is extraordinarily difficult because confirmation bias causes us to (a) decide not to search for data that would reveal confirmation bias; and (b) if data somehow becomes available, to disregard or to seek alternative explanations for it if that data tends to confirm the existence of confirmation bias. Moreover, another cognitive bias known as the bias blind spot causes individuals to see the existence and effects of cognitive biases much more in others than in themselves [Pronin 2002].

Still, the enterprise would benefit from monitoring the possible existence and effects of confirmation bias in decisions with respect to allocating resources to managing technical debt. Whenever decisions are made, we must manage confirmation bias risk.

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

Nontechnical precursors of nonstrategic technical debt

Last updated on July 8th, 2021 at 01:09 pm

Nonstrategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce nonstrategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. In this thread I examine a variety of precursors of nonstrategic technical debt that aren’t directly related to technology. Sources of these precursors include:

Communication between and among people
Organizational policies relating to job assignments
Cognitive biases [Kahneman 2011]
Performance management policy
Incentive structures
Organizational structures
Contract language
Outsourcing
…and approaches to dealing with budget depletion.

Precursors vs. causes

The cables of the Brooklyn Bridge are an example of nonstrategic technical debt — Some of the suspension cables of the Brooklyn Bridge. Washington Roebling, the chief engineer, designed them to be composed of 19 “strands” of wire rope [McCullough 1972]. Each strand had 278 steel wires. Thus, the original design called for a total of 5,282 wires in each of the main cables. After the wire stringing began, the bridge company made an unsettling discovery. The wire supplier, J. Lloyd Haigh, had been delivering defective wire by circumventing the stringent inspection procedures. In all, Roebling estimated that 221 U.S. tons (200 metric tons) of rejected wire had been installed. This was a significant fraction of the planned total weight of 3,400 U.S. tons (3,084 metric tons). Because they couldn’t remove the defective wire, Roebling decided to add about 150 wires to each main cable. That extra wire would be provided at no charge by Haigh [Talbot 2011]. I can’t confirm this, but I suspect that Roebling actually added 152 wires, which would be eight wires for each of the 19 strands. That made a total of 286 wires per strand, or 5,434 wires. The presence of the defective wire in the bridge cables—which remains to this day—is an example of technical debt. The fraud perpetrated by Haigh illustrates how malfeasance can lead to technical debt.

I use the term precursor instead of cause because none of these conditions leads to technical debt inevitably. From the perspective of the policymaker, we can view these conditions as risks. It’s the task of the policymaker to devise policies that manage these risks.

McConnell has classified technical debt in a framework that distinguishes responsible forms of technical debt from other forms [McConnell 2008]. Briefly, we incur some technical debt strategically and responsibly. Then we retire it when the time is right. We incur other technical debt for other reasons, some of which are inconsistent with enterprise health and wellbeing.

The distinction is lost on many. Unfortunately, most technical debt is nonstrategic. We would have been better off if we had never created it. Or if we had retired it almost immediately. In any case we should have retired it long ago.

It’s this category of nonstrategic technical debt that I deal with in this thread. Although all technical debt is unwelcome, we’re especially interested in nonstrategic technical debt, because it’s usually uncontrolled. In these posts I explore the nontechnical mechanisms that lead to formation of nonstrategic technical debt. Schedule pressure is one exception. Because it’s so important, it deserves a thread of its own. I’ll address it later.

Last words

Here are some of the more common precursors of nonstrategic technical debt.

I’ll be adding posts on these topics, so check back often, or subscribe to receive notifications when they’re available.

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

Non-technical precursors of non-strategic technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

Nontechnical precursors of nonstrategic technical debt

December 11, 2017July 7, 2021

Organizational culture and technical debt

Last updated on July 7th, 2021 at 03:18 pm

A bonsai tree. We cannot control the growth of bonsai in detail. We can only make suggestions, and hope that the tree will follow our guidance. So it is with organizational culture.

Organizational culture affects—and is affected by—everything the organization does. It is complex beyond imagining. It creates leaders and leaders shape culture [Schein 2016]. Fortunately, for our narrow purposes—namely, improving how we manage technical debt—we need not explore every available model of culture. All we need to know is how to adjust cultures so that they’re more amenable to technical debt management efforts. Even so, that’s an ambitious objective.

Organizational culture is the collection of values, beliefs, and principles the people of the organization share. Some of these elements are within the awareness of the organization’s people. But some might not be. Culture excludes behavior; behavior is a result of the combined effects of personal choices and organizational culture.

With respect to technical debt, our objective is ambitious because it is at once specific and general. It is general in the sense that as a goal, we would like to devise prescriptions for all organizational cultures as they strive to be more effective in managing technical debt. It is specific in the sense that we’re concerned only with those cultural factors that affect technical debt management.

The case for considering cultural effects on technical debt

But aren’t there enough technical problems to keep us occupied before we get involved in all that culture change stuff? The answer to that, I believe, is yes. But we can make more rapid progress if we allow considerations of culture to guide our technical investigations.

Some of our best minds have been working on the technical debt problem for more than two decades, mostly from the technological perspective. True, we haven’t given the issue the attention and resources it deserves. That might be changing now, but one can reasonably ask a simple question: If some of the causes of the technical debt problem are cultural, and if we begin to make some technological progress in reducing the burden of technical debt, wouldn’t the cultural causes begin to dominate at some point? If that were to happen, the effectiveness of our technical solutions would appear to decline, even though they might be “right,” because technical solutions cannot address cultural problems.

It seems only prudent to prepare for that case by examining the role of culture in creating technical debt and preventing or delaying its retirement.

A reasonable goal might be to define a set of attributes an organizational culture should have if the organization wants to limit the contributions of culture to the formation and persistence of technical debt. I suspect that for most cultures, change would be required, because most organizations now carry more technical debt than they would care to acknowledge.

As with all culture change efforts, we seek, in the end, to modify the behavior of the people who live within that culture. We want to modify aspects of the culture so as to lead its people to be more likely to make choices that are consistent with prudent technical debt management.

Where to from here

With that goal in mind, I’m opening a thread in this blog that I hope will stimulate your thinking and mine about how organizational culture and technical debt interact. This post will contain an outline of these ruminations. Here’s what I have so far:

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

Non-technical precursors of non-strategic technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

Nontechnical precursors of nonstrategic technical debt

November 17, 2017July 8, 2021

Useful projections of MPrin might not be attainable

Last updated on July 8th, 2021 at 11:50 am

Summary

Expect the unexpected with technical debt retirement efforts. Technical debt retirement efforts can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Although these conflicts are technical in nature, resolving them can involve business priorities at any level. Planners must be aware of these potential conflicts, and coordinate with their leaders. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.

For planning purposes, it’s necessary from time to time to make projections of debt retirement costs (MPrin) for a given class of technical debt. The need arises when planning debt retirement, or when preparing debt retirement options for determining resource allocations. Although retiring some kinds of technical debt is straightforward, other kinds of debt can become intertwined with each other. Retiring still others might appear to be easy, but actual retirement efforts expose unanticipated entanglements. Moreover, debt retirement efforts can sometimes interact with other debt retirement efforts, operations, maintenance, cyberdefense, and new development in both expected and unexpected ways. For these reasons, making estimates of the MPrin with enough precision to be useful can be notoriously difficult.

Money is fungible; people are not

A tangle of cordage on board ship. Different kinds of technical debt can become entangled with each other. Untangling them can affect various other engineering efforts. Preparing an asset for a debt retirement effort by doing some preliminary untangling might be wise before trying to estimate the MPrin of any affected class of technical debt.

These considerations rarely arise when planning retirement of financial debts, because money is fungible. We might indeed have other uses for financial resources. But every unit of cash is equivalent to every other. That freedom isn’t necessarily available when planning resource allocations for technical debt retirement.

For example, not every engineer is equally capable of addressing every problem. Some people are particularly capable for certain kinds of work, and not very capable for other kinds. The problem of scheduling specialists is notorious for generating bottlenecks. And split assignments create even more trouble. People aren’t fungible.

Last words

Planning retirement of a particular set of technical debt classes can be complicated. Such planning requires knowledge of any efforts with which that retirement effort might interact. That information might not be available or might not be known. In general, preliminary work to decouple these activities—often called ”refactoring”—can

MPrin for missing or incomplete capability
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
MPrin when standards or regulations change
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
MPrin in a platform component upgrade
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
MPrin in a development project
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Examples of MPrin in four scenarios
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
The metaphorical principal of a technical debt
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.

November 15, 2017July 16, 2021

Glossary and Terminology

Last updated on July 16th, 2021 at 11:23 am

Even though technical debt has been with us for a very long time—probably since the time we began inventing technologies—the study of technical debt is relatively new. Ward Cunningham coined the term technical debt in 1992, and its meaning has evolved since then. Because universally accepted definitions for the term and associated concepts have not yet emerged, it seems necessary to have a page on this site that collects definitions.

Asset-exogenous technical debt

Exogenous technical debt is asset-exogenous when it’s brought about by an activity external to an asset, but internal to the enterprise. For example, a change in standards or regulations by some body within the enterprise can cause an asset to incur an asset-exogenous technical debt.

ATD

See Auxiliary technical debt.

Auxiliary technical debt

In the context of a Technical Debt Retirement Project (DRP) that has as an objective retiring from a specified set of assets a particular kind or particular kinds of technical debt, the ATD is the collection of instances of any other kinds of technical debt other than the kind that the DRP is trying to retire. More: “Auxiliary technical debt: Rules of engagement”

Class of technical debt

On occasion, we speak of classes of technical debt and instances of that class. This can be confusing, because the words class and instance have particular meanings in software engineering. That’s not the sense in which we use the terms here. In this blog, a class of technical debt is just a collection of instances of the same kind of debt. For example, consider the “ghost ramp” described in “Technical debt in the highway system.” It belongs to the class of ghost ramps. If we were maintaining the highway system of Massachusetts, it might be convenient to consider the class of ghost ramp technical debt if we want to let a contract to demolish all ghost ramps. Each ghost ramp would then be an instance of that class.

Cognitive bias

A cognitive bias is the human tendency to make systematic errors based not on evidence, but on factors related to the thought process. Psychologists have identified and demonstrated hundreds of cognitive biases, including several that could plausibly explain failures in priority setting for technical debt retirement projects.

Confirmation bias

Confirmation bias is a cognitive bias. It’s the human tendency to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. For example, the homogeneity of cable news channel audiences, and the alignment between preconceptions of the audience and the slant of the newscast for that channel, are results of confirmation bias. More: “Confirmation bias and technical debt”

Debt contagion

If a class of technical debt is allowed to remain outstanding, its volume can increase as a consequence of seemingly unrelated actions or decisions. Moreover, its existence can cause increases in the volume of other existing classes of technical debt, and its existence can lead to the formation of new classes of technical debt. This process is called debt contagion. More: “Debt contagion: how technical debt can create more technical debt”

DRP

A (technical) Debt-Bearing Asset

DRP

In this blog, I use the term DRP to mean a (technical) Debt Retirement Project. A DRP is a project that has as an objective retiring from a specified set of assets a particular kind of technical debt (or particular kinds of technical debt). Many projects have objectives of debt retirement, at some point or other. But DRPs differ from most, in that debt retirement is their primary objective—indeed, it might be their sole objective. More: “Nine indicators of wickedness”

Echo release

An echo release of an asset is a release version whose primary purpose is technical debt retirement. Typically, it’s created immediately following a release version that has created some incremental technical debt, hence the term “echo release.” The echo release is then executed to retire that incremental technical debt, and not to repair defects or add capability. More: “Accounting for technical debt”

Endogenous technical debt

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing). So endogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that directly involve the asset. More: “Exogenous technical debt”

Enterprise-exogenous technical debt

Exogenous technical debt is enterprise-exogenous when it’s brought about by an activity external to the enterprise. For example, a change in standards or regulations by some body outside the enterprise can cause an asset to incur an enterprise-exogenous technical debt.

Exogenous technical debt

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that don’t involve the asset directly. More: “Exogenous technical debt”

Ill-structured problem

An ill-structured problem is a problem that isn’t a well-structured problem [Simon 1973]. An example of an ill-structured problem is finding a definition for ill-structured problems. Another: designing a computer programming language. Still another, even more to the point: deciding when to retire a particular class of technical debt. NDM is more likely to be successful with ill-structured problems than is RDM.

Incremental technical debt

Incremental technical debt is either newly incurred exogenous technical debt, or technical debt that’s incurred in the course of work currently underway or just recently completed. For example, in an apartment building hallway renovation project, workmen did insert expansion joints in the sheetrock they replaced, but on the first three floors they completed, the joints were too widely separated. The remaining 22 floors were done correctly. Nine additional joints on each of the incorrect floors must be inserted eventually. The missing joints, which constitute incremental technical debt, will be inserted after the job is completed. More: “Controlling incremental technical debt”

Instance of technical debt

See “Class of technical debt”

Intertemporal choice

Intertemporal choice is the process by which people make decisions between options that occur at different points in time. Decisions involving intertemporal choice can be exceedingly complex, especially when options have effects that vary with time. For example, confronted with advice from technical experts regarding the urgent need to address the burden of enterprise technical debt, decision makers must consider an unpleasant possibility. To make resources available to retire the technical debt, it might be necessary to temporarily defer investment in some new products or enhancing some existing products. And if they make the recommended investments in technical debt retirement, customers won’t benefit in any visible way. So the choice reduces to one between new products and enhancements relatively sooner, versus retiring technical debt and only later attending to new products and enhancements of existing products. This dilemma is an example of what behavioral economists call intertemporal choice [Loewenstein 1992].

Key Performance Indicator (KPI)

A Key Performance Indicator (KPI) is a metric that provides meaningful insight that’s used to guide business decisions. All KPIs are metrics; not all metrics are KPIs. More: “Metrics for technical debt management: the basics”

Legacy technical debt

Legacy technical debt is technical debt associated with an asset, and which exists in any form prior to undertaking work on that asset. For example, in planning a project to renovate the hallways and common areas of a high-rise apartment building, Management discovers that beneath the existing carpeting is a layer of floor tile containing asbestos. Management has decided to remove the tile. In this context, the floor tile can be viewed as legacy technical debt. It isn’t directly related to the objectives of the current renovation, but removing it will enhance the safety of future renovations, enable certification of the building as asbestos-free, and reduce the cost of eventual demolition. More: “Exogenous technical debt”

Localizable technical debt

Localizable technical debt is technical debt that manifests itself as discrete chunks. Each instance is self-contained, and we can “point” to it as an instance of the debt in question. For example, if the organization regards Windows 10 as the current operating system for personal computers, and early versions of Windows as technical debt, the each computer that runs and earlier version of Windows is an instance of that technical debt. Each instance is discrete and localized. More: “Retiring localizable technical debt”

Measure

A measure is the result of determining the value of a quantifier. For example, we might use the quantifier’s definition to determine a measure of how much human effort has been expended on an asset in the past fiscal quarter. More: “Metrics for technical debt management: the basics”

Metric

A metric is an arithmetic formula expressed in terms of constants and a set of measures. One of the simpler metrics consists of a single ratio of two measures. For example, the metric that captures the average cost of acquiring a new customer in the previous fiscal quarter is the ratio of two measures, namely, the investment made in acquiring new customers, and the number of new customers acquired. More: “Metrics for technical debt management: the basics”

MICs, or metaphorical interest charges

MICs are the metaphorical interest charges associated with a technical debt. They aren’t interest charges in the financial sense; rather, the MICs of a technical debt represent the total of reduced revenue, lost opportunities, and increased costs of all kinds borne by the enterprise as a consequence of carrying that technical debt. Because the properties of MICs are very different from the properties of financial interest charges, we use the term MICs to avoid confusion with the term interest from the realm of finance. More: “How financial interest charges differ from interest charges on technical debt”

MPrin, or metaphorical principal

The MPrin of a technical debt at a give time T is the total cost of retiring that debt at time T. The total cost includes all cost factors: labor, equipment, service interruptions, revenue delays, anything. It even includes the ongoing costs of repairing defects introduced in the debt retirement process. More: “The metaphorical principal of a technical debt”

Naturalistic decision-making

Naturalistic decision-making (NDM) entails situation assessment and evaluation of a single option to select a satisfactory option. [Zannier 2007] Features that define naturalistic decision-making are “time pressure, high stakes, experienced decision makers, inadequate information (information that is missing, ambiguous, or erroneous), ill-defined goals, poorly defined procedures, cue learning, context (e.g., higher-level goals, stress), dynamic conditions, and team coordination.” [Klein 2017]

Nonstrategic technical debt

Nonstrategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce nonstrategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. And at times, it appears in the asset as a result of external events beyond the boundaries of the enterprise. More: “Nontechnical precursors of nonstrategic technical debt”

The planning fallacy

The planning fallacy is a cognitive bias that causes planners to underestimate costs and schedules, and over-promise benefits. They do this, in part, because they pay too little heed to past experience on similar efforts. They also rely too much on what they believe will happen on the effort they’re planning. First identified in a 1977 report by Daniel Kahneman and Amos Tversky [Kahneman 1977] [Kahneman 1979]. More: “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma”

Policy

Organizational policy is the framework of principles that guide policymakers, decision makers, and everyone in the organization as they carry out their responsibilities. Policy might be written or not, but written policy is more likely to consistently adhered to. Interestingly, the body of organizational policy is itself subject to accumulating technical debt. More: “What is policy?”

Policymaker

As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management. More: “Who are the policymakers?”

Quantifier

A quantifier is a specification for a measurement process designed to yield a numeric representation of some attribute of an asset or process. Quantifiers are used to obtain the values called measures, which in turn are used in computing metrics. More: “Metrics for technical debt management: the basics”

Rational decision-making

Rational decision-making (RDM) is an approach to making a choice of an option from among a set of options by selecting the option that is optimal with respect to a set of quantitative criteria. [Zannier 2007] Rational choice strategies generally follow this framework: (1) Identify a set of options; (2) Identify criteria for evaluating them; (3) Assign weight to each evaluation criterion; (4) Rate the options relative to the criteria; (5) Choose the option with the highest score. Many different frameworks for implementing this strategy are available, some specialized to specific subject domains [Thokala 2016].

Refactoring

Fowler defines refactoring as “the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure” [Fowler 1999]. Although refactoring is a term specific to software development processes, the concept applies to all technological development. For example, an automobile manufacturer’s decision to alter the design of one of their model vehicles to reduce manufacturing costs can be viewed as a form of refactoring. Refactoring is a practice essential to effective technical debt management. More: “Refactoring for policymakers”

Regression testing

Regression testing is a testing regimen that ensures that a previously developed and tested system still performs the same way after it has been altered or when it’s used in a new context. Regression testing is essential when we alter a system by retiring some of its technical debt.

The reification error

The reification error (also called the reification fallacy, concretism, or the fallacy of misplaced concreteness) is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing. Reification is useful in some applications, such as object-oriented programming and design. But when we use it in the domain of logical reasoning, troubles can arise. Specifically, we can encounter trouble when we think of “measuring” technical debt. Strictly speaking, we cannot measure technical debt. We can estimate the cost of retiring it, but estimates are only approximations. And in the case of technical debt, the approximations are usually fairly rough. To regard these estimates as measurements is to risk reifying them. Then when the actual cost of a debt retirement project is dramatically larger than the estimate, the consequences for enterprise budgets can be severe. We must always regard “measurements” of technical debt as estimates—estimates that are so prone to error that we must plan for error. The reification error is the dual of the resilience error. More: “Metrics for technical debt management: the basics.”

The resilience error

If the reification error is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing, the resilience error is an error of reasoning in which we treat an abstraction as if it were more flexible, resilient, and adaptable than it actually is. When we commit the resilience error with respect to an abstraction, we’re adopting a belief, usually without justification, and possibly outside our awareness. That belief is that the familiar properties of the abstraction can survive changes in the abstraction.

Specifically, if we make changes in the abstraction, we can be certain that the familiar properties of the abstraction we modified will apply in modified form. We hold this belief without fully investigating the consequences of the changes we made in the abstraction. Or we assume incorrectly that the abstraction will accommodate any changes we make to its environment. The resilience error is the dual of the reification error. We are at risk of making the resilience error when we refactor assets to reduce their burden of technical debt. More: “The resilience error and technical debt.”

Secured technical debt

A secured technical debt, like a secured financial debt, is one for which the enterprise has reserved the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. They might include particular staff, equipment, test beds, downtime, and financial resources. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes. More: “Using SMART goals for technical debt reduction”

Source and target components of a metaphor

In a metaphor of the form “A is B,” the source is the element whose attributes are being attributed to the target. For example, in “my son’s room is a war zone,” the source is the war zone, and the target is my son’s room. More: “The structure of metaphors”

Super wicked problem

A subset of wicked problems can be viewed as super wicked [Levin 2012]. Levin, et al. list the following four properties of super wicked problems: (1) Time is running out; (2) Those who cause the problem also seek to provide a solution or influence the solution; (3) The central authority needed to address the problem is weak, nonexistent, or chooses not to act effectively; (4) Policy responses discount the future irrationally. I’ve come to believe that some technical debt retirement project design can be a super wicked problem. More: “Retiring technical debt can be a super wicked problem”

Tame problems

A problem is a tame problem if it fails to meet at least one of the ten criteria established by Rittel and Webber [Rittel 1973] for wicked problems. Four of the criteria: it’s an ill-structured problem; it’s incompletely defined or internally contradictory; its solutions aren’t true-or-false, but good-or-bad; and there’s no way to exhaustively describe all solutions. I’m convinced that technical debt retirement project design can be a wicked problem. A tame problem is one that fails to meet at least one of the ten criteria for wickedness. Tame problems and wicked problems thus lie at opposite ends of a “Tame/Wicked” spectrum. Technical debt retirement project design problems fall somewhere on this spectrum. More: “Degrees of wickedness.”

Taylorism

Taylorism is an approach to management developed by Frederick Winslow Taylor in the early part of the twentieth century [Taylor 1913] [Kanigel 1997]. He proposed three principles of scientific management that could produce maximum efficiency. First, managers should select the person performing the work based on science. Second, organizations should decompose tasks based on scientific principles. Third, they must separate planning from execution. These principles are the basis of what became known in software engineering as the waterfall lifecycle. The approach works well for well-structured problems, but does not work well at all for ill-structured problems. Moreover, it depends for success on repeating solutions to problems already solved, which is why it proved so valuable in early manufacturing. The unsuitability of Taylorism for ill-structured problems is an important part of the basis for the Agile approach to problem solving.

TDIQ

In the context of a Technical Debt Retirement Project (DRP), we can define the Technical Debt In Question (TDIQ). If the DRP has as an objective retiring a kind of technical debt, that kind of technical debt is the TDIQ. More: “Retiring technical debt from irreplaceable assets”

Technical debt

Technical debt is any technological element that hampers development, maintenance, or enhancement efforts, through its existence or through its absence. It contributes to lower productivity or to a higher probability of defects. Or it can depress velocity in many other ways. That’s why we would like to revise, repair, replace, rewrite, create, or re-engineer it for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it. More: “A policymaker’s definition of technical debt”

Technological communication risk

Technological communication risk is the risk that, for whatever reason, knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. More: “Technological communication risk”

Temporal discounting

Temporal discounting is the human tendency to give greater value to a reward (or as economists would say, to assign greater utility to a good) the earlier it arrives. An analogous process affects perceptions of inconvenience or disutility: people assign more negative values to penalties and inconveniences the sooner they arrive. If the discount rate is constant, the discounting is termed exponential discounting or rational discounting. But other forms are possible. Hyperbolic discounting is one form of discounting at a rate that is higher for near-term arrivals than for distant-term arrivals [Laibson 1997]. Humans have been observed experimentally to favor a form of temporal discounting that is well modeled as hyperbolic discounting.

Terrifying opportunity

A terrifying opportunity arises when the organization rejects (or fails to recognize) a market opportunity because exploiting it would involve modifying an existing asset or product offering that harbors a heavy load of technical debt. The debt causes decision makers to assess that the probability of success is so low that the opportunity seems terrifying, and they therefore reject the opportunity. More: “MICs on technical debt can be difficult to measure”

Well-structured problem

As defined by Simon [Simon 1973], a well-structured problem is a problem that has some or all of six characteristics. The first is the existence of a definite criterion for testing any proposed solution, and a mechanizable process for applying that criterion. Second, there is at least one problem space in which we can represent the initial problem state, the goal state, and all states that can be reached or considered while solving the problem. There are four more criteria, but these are the biggies. An example of a well-structured problem is the game of chess. RDM is useful for attacking well-structured problems.

Wicked problem

A problem is a wicked problem if it meets the ten criteria established by Rittel and Webber [Rittel 1973]. Four of the criteria: it’s an ill-structured problem; it’s incompletely defined or internally contradictory; its solutions aren’t true-or-false, but good-or-bad; and there’s no way to exhaustively describe all solutions. I’m convinced that technical debt retirement project design can be a wicked problem. More: “Self-sustaining technical knowledge deficits during contract negotiations.”

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

Glossary and Terminology

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

Glossary and Terminology

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

Glossary and Terminology

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

Glossary and Terminology

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

Glossary and Terminology

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

Non-technical precursors of non-strategic technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

Nontechnical precursors of nonstrategic technical debt

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

Glossary and Terminology

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

Glossary and Terminology

November 15, 2017July 8, 2021

Policy implications of the properties of MPrin

Last updated on July 8th, 2021 at 11:48 am

Formulating sound policy vis-à-vis technical debt requires a thorough understanding of the distinction between the MPrin associated with a technical debt and the principal amount of a financial debt. The policy implications of the properties of technical debt arise from three fundamental differences between technical debt and financial debt.

MPrin can change spontaneously

For most financial debts, a formula determines the principal amount. Voluntary actions of the debtor can also affect the principal amount. For example, the debtor might make periodic payments on an installment loan, or new purchases on a credit card account. By contrast, MPrin of a technical debt can change absent any action by the “borrower.” For example, changes in regulations, standards, or technologies can all cause changes in MPrin. More: “How MPrin can change spontaneously”

Technical debt can create more technical debt

Technical debt left in place can create more technical debt without the knowledge or consent of the debtor organization. By contrast, the principal amount of a financial debt can grow, but law or regulation requires notification—and in some cases consent—of the debtor. More: “How MPrin can change spontaneously”

Projecting MPrin with useful precision might not be possible

The cost of retiring a technical debt can depend on how the asset bearing the debt has changed over the life of the debt. And it can depend on what other projects the enterprise is executing at debt retirement time. These factors are difficult to predict. By contrast, projecting the principal amount of a financial debt is formulaic. More: “Useful projections of MPrin might not be attainable”

A pole full of wires. Technical debt is everywhere.

The policy implications of these properties of MPrin can be profound. First, spontaneous change in MPrin implies a need for investments in market and technological intelligence. The intelligence of greatest value focuses specifically on potential effects on technical debt. Second, existing technical debt can lead to creating new instances of that debt or other debts. We can limit this “contagion” if we know what kinds of technical debt are most likely to exhibit this phenomenon. Finally, the difficulty of projecting MPrin implies that typical reliance on analytical modeling of enterprise asset evolution in preference to human judgment might be unwise. A wiser course might be investment in employee retention programs focused on the individuals who can provide the necessary wisdom.

This is just a sketch of the problems policymakers confront when dealing with the properties of MPrin. I’ll be addressing them in more detail in future posts.

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

Order from Amazon

Cited in:

Glossary and Terminology

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

Glossary and Terminology

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

Glossary and Terminology

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

Glossary and Terminology

Available: here; Retrieved: October 17, 2018

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

Glossary and Terminology

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

Non-technical precursors of non-strategic technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

Nontechnical precursors of nonstrategic technical debt

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

Glossary and Terminology

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

Available: here; Retrieved October 15, 2018

Cited in:

Glossary and Terminology

MPrin for missing or incomplete capability
In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
MPrin when standards or regulations change
The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
MPrin in a platform component upgrade
Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
MPrin in a development project
The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
Examples of MPrin in four scenarios
Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
The metaphorical principal of a technical debt
The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.

November 7, 2017July 8, 2021

What is policy?

Last updated on July 8th, 2021 at 11:46 am

DuBrin defines policies as “… general guidelines to follow when making decisions and taking action” [DuBrin 2016]. Some policies are written, some are unwritten. Some have names or identifiers, some don’t. For organizations seeking to gain control of technical debt, written policies are probably a good idea, for two reasons:

Many people affected by technical debt policies are probably unfamiliar with the technical debt concept. A written policy is more likely to be communicated uniformly to everyone.
The effort spent devising a written policy is likely to surface ambiguities, confusions, and differing needs. That’s one of the benefits of devising written policies. Resolving these issues during the policy formation process is better for the organization than resolving them during the deployment process.

A useful policy is clear. It uses terminology that’s simple and well-defined.

Properties of technical debt policy

Achieving these goals for technical debt policy formulation can present special problems. Much of the audience for the policy is unaware or incredulous of the connection between their own behavior and technical debt formation.

Specifically:

The policy must address an issue—technical debt—that has mainly technological manifestations
The policy must provide guidance for people as they make choices
Some of the choices that people make will produce technological manifestations
The connections between the choices and the technological manifestations can be obscure, especially
when the choices don’t appear to be technical
Some technical debt arises from phenomena unrelated to behavior of anyone within the organization

Said differently, technical debt policy must confront four issues:

Some people whose behavior we must modify are unaware of the consequences of their behavior
Some of those same people strongly believe that the technical debt problem results from malpractice by others
Current incentive structures play an important role in technical debt formation
Some technical debt arises from phenomena external to the organization

Effective technical debt policy must therefore include these elements:

An education component to help people see the connection between their choices and technical debt formation
A situational awareness component to alert the organization to internal and external developments that could cause technical debt formation
A means of allocating resources to technical debt management that holds accountable the business units whose actions contributed to technical debt formation
A means of committing future resources to technical debt retirement

References

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Also cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

An accessible and authoritative overview of organizational psychopathy. Order from Amazon

Cited in:

[DuBrin 2016] Andrew J. DuBrin. Essentials of Management, Tenth Edition. Indianapolis, Indiana: Wessex Press, 2016.

Order from Amazon

Cited in:

What is policy?

Order from Amazon

Cited in:

Glossary and Terminology

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

Glossary and Terminology

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

Glossary and Terminology

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

Glossary and Terminology

Available: here; Retrieved: October 17, 2018

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

Glossary and Terminology

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

How budget depletion leads to technical debt

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

Non-technical precursors of non-strategic technical debt

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

Confirmation bias and technical debt

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

Nontechnical precursors of nonstrategic technical debt

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

Glossary and Terminology

Available: here; Retrieved: 10/16/18

Cited in:

Glossary and Terminology

Available: here; Retrieved October 15, 2018

Cited in:

Glossary and Terminology

The effect of an absence of standards

Last words

Other posts in this thread

References

Share this:

Examples of how malfeasance can lead to technical debt

What kinds of malfeasance deserve special attention and why

Why malfeasance deserves special attention

What’s required to control malfeasance

Corrective measures

Detection measures

Preventative measures

References

Other posts in this thread

Share this:

A physical example

A broad array of effects

Last words

References

Other posts in this thread

Share this:

How confirmation bias can lead to technical debt

An example

Last words

References

Other posts in this thread

Share this:

Precursors vs. causes

Last words

References

Related posts

Share this:

The case for considering cultural effects on technical debt

Where to from here

References

Related posts

Share this:

Money is fungible; people are not

Last words

Related posts

Share this:

Asset-exogenous technical debt

ATD

Auxiliary technical debt

Class of technical debt

Cognitive bias

Confirmation bias

Debt contagion

DRP

DRP

Echo release

Endogenous technical debt

Enterprise-exogenous technical debt

Exogenous technical debt

Ill-structured problem

Incremental technical debt

Instance of technical debt

Intertemporal choice

Key Performance Indicator (KPI)

Legacy technical debt

Localizable technical debt

Measure

Metric

MICs, or metaphorical interest charges

MPrin, or metaphorical principal

Naturalistic decision-making

Nonstrategic technical debt

The planning fallacy

Policy

Policymaker

Quantifier

Rational decision-making

Refactoring

Regression testing

The reification error

The resilience error

Secured technical debt

Source and target components of a metaphor

Super wicked problem

Tame problems