Technological communication risk

Last updated on February 1st, 2018 at 07:32 am

Technologists must convey what they know about long-term technology trends to enterprise strategists and others. In addition to strategists, the interested parties include internal customers of technology, product owners, product managers, project sponsors, or senior management. Within the enterprise, technologists tend to be among those most knowledgeable regarding the relative alignment between enterprise technological assets and long-term technology trends. Yet technologists frequently fail to communicate this knowledge effectively to those who need it, and that can lead to non-strategic technical debt. I call this phenomenon technological communication risk.

See no evil, hear no evil, speak no evil
Hear no evil, see no evil, speak no evil — the iconic representation of communication failure. Technical debt can result from communication failures due to unwillingness to inform others of what you know, and unwillingness to receive information from others more knowledgeable.

Technological communication risk is the risk that knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. Policymakers can address this problem by working to define the roles of all involved to specify the need for this communication, and the need to be receptive to it.

A clear understanding of long-term technology trends is important in managing technical debt. Any significant misalignment between enterprise technological assets and long-term technology trends creates a risk of incurring new technical debt. As technologies evolve, enterprise assets must evolve with them. The gap between those assets and the state of the art is a source of lost productivity and depressed organizational agility, which is our definition of technical debt.

Some technologists are better informed about technology trends than are their internal customers, product owners, product managers, project sponsors, or senior management. Technologists often do attempt to communicate what they know on an informal basis, but unless such communication is expected and defined as an official duty, their superiors and internal customers don’t always welcome the information, especially if they haven’t heard it elsewhere, or if it conflicts with what they’ve learned elsewhere, or if its implications conflict with established strategic positions.

Many technologists are aware that their superiors might not welcome their observations about technological trends or technology-based strategic vulnerabilities or opportunities. For example, one might understand why a technologist might be reluctant to alert an unreceptive senior manager to a suddenly revealed cybersecurity risk that would be very expensive to mitigate. This mechanism is especially strong when deploying adequate cyberdefense would compete for resources with other initiatives already underway, or when the negative consequences of the vulnerability are unlikely to materialize. And some tend to question technologists’ credibility when they blame the technologists for the vulnerability itself.

Situations like these can lead to the formation of new non-strategic technical debt in circumstances such as the following:
  • Management directs the technologists to produce capabilities using approaches known to the technologists to be technological dead ends.
  • Management directs the technologists to implement capabilities that don’t exploit known approaches that could open new and vital lines of business.
  • Management directs the technologists to focus resources on initiatives that in the view of the technologists lack sufficient technological imperative.

Policymakers can mitigate technological communication risk by establishing internal standards that encourage knowledgeable technologists to share what they know with internal customers, project sponsors, or senior management. Similarly, those standards can encourage internal customers, project sponsors, product owners, product managers, and senior management to take heed when knowledgeable technologists do speak up.

Other posts in this thread

Failure to communicate the technical debt concept

Last updated on May 25th, 2018 at 09:46 am

The behavior of internal customers and users of enterprise technological assets can contribute to technical debt formation and persistence. Because of these contributions, introducing effective technical debt management practices requires widespread behavioral changes on the part of those internal customers and users. Accepting these changes, and the initiative and creativity they require, is possible only if people understand the technical debt concept. When they do, they can appreciate the benefits of controlling technical debt, and the consequences of failing to control it. Similarly, when they do not understand or accept the technical debt concept, progress toward effective technical debt management is unlikely. Policymakers can contribute to the planning and execution of the required organizational transformation.

Even when the engineering teams are aware of the technical debt concept, and when they do try to manage technical debt, they cannot make much progress unless they have the support and understanding of their own management, their internal customers, and their customers’ managements. Everyone must understand that controlling technical debt — and retiring it — is a necessary engineering activity that has a business purpose. Everyone must understand that technical debt arises as a result of everyone’s behavior — not just the behavior of technologists.

A tensegrity 3-prism
A tensegrity three-prism. . Read about tensegrity structures.
Image (cc) Bob Burkhardt courtesy Wikimedia.
Part of the job of Management is to see that engineers have what they need to avoid incurring technical debt unnecessarily, and that they have what they need to retire elements of legacy technical debt on a regular basis. Internal customers must understand that communicating their long-term business strategies to Engineering is essential for limiting unnecessary creation of artifacts that become non-strategic technical debt. Only by understanding the technical debt concept can internal customers learn to avoid the behaviors that lead to non-strategic technical debt, and adopt behaviors that limit new technical debt.

Tensegrity structures provide a metaphor for organizations that have mastered the technical debt concept. Tensegrity structures use isolated rigid components in compression, held by a network of strings or cables in tension. The rigid components are usually struts or masts, and they aren’t in contact with each other.

The struts correspond to the users or customers of technological assets. The cables correspond to the engineering activities required to support the customers. The organization is stable relative to technical debt only when the two kinds of elements (struts and cables) work together, each playing its own role, but each appreciating the role of the other.

Advocating for cultural transformation

Advocates of any change to organizational culture are often seen as acting in their own self-interest. That’s a common risk associated with cultural transformation. It’s a risk that can lead to failure when inserting practices related to technical debt management into the culture. The risk is greatest when advocates for change are drawn exclusively from the technical elements of the enterprise. The ideal advocates for these ideas and practices are the internal customers of the technical organizations, and senior management.

Other posts in this thread

Metaphors and the term technical debt

Last updated on December 11th, 2018 at 11:21 am

The technical debt metaphor is both powerful and perilous. Its power lies in its ability to communicate the concept that some technological assets regarded as operational might—and probably do—need further attention. The peril arises when we think of this metaphorical technical debt as if it were a financial debt.

Ward Cunningham, who coined the technical debt metaphor
Ward Cunningham, who coined the technical debt metaphor. Photo (cc) Carrigg Photography.

Ward Cunningham coined the technical debt metaphor in the context of developing a software asset [Cunningham 1992] [Cunningham 2011]. He observed that when the development process leads to new learning, re-executing the development project—or parts of the project—could lead to better results. For this reason, among others, newly developed software assets can contain, embody, or depend upon artifacts that, in hindsight, the developers recognize could be removed altogether, or could be replaced by more elegant, effective, or appropriate forms that can enhance maintainability, defensibility, and extensibility. To deploy the asset in pre-hindsight condition would entail an obligation to return to it in the future to implement the improvements revealed by that hindsight. That obligation is Cunningham’s conception of the asset’s technical debt.

This phenomenon is not restricted to newly developed software. It applies to all technological assets. And it applies to new development, maintenance, cyberdefense, and enhancement.

Cunningham was aware that he was using a metaphor to describe his situation, which is a common one in technological development projects—both software and hardware. He probably chose the debt metaphor because his audience was financially sophisticated. He was exploiting their knowledge of financial instruments to convey a concept that, from his perspective, properly belongs in the software engineering body of knowledge. Such uses of metaphors are not unusual.

For example, the concept of leverage, which originates in mechanics, captures the idea of mechanical advantage gained when one employs a rigid bar, resting on a fulcrum, to move a heavy or fixed object. When a lever is arranged so that A is the distance from the heavy weight to the fulcrum, and B is the distance from the fulcrum to the point of application of the force, one gains a force “multiplier” of B/A. This concept has been used in the world of finance, renamed financial leverage, to describe how borrowing can confer on the borrower a financial “force multiplier” by increasing the borrower’s total financial power. From there, the term leverage has spread into the general vocabulary of business. Indeed, we can now say that, “Cunningham leveraged his boss’s understanding of financial instruments to convey a concept that properly belongs in the software engineering body of knowledge.”

Metaphors are powerful.

And they are also dangerous. The danger arises when we rely on the audience to apply their experience to interpret the metaphor in the way we intend. But since the experience of every individual is different, we cannot be certain how the audience might interpret the metaphor. And that is where the trouble begins.

Before we undertake our exploration of the technical debt metaphor, we must investigate the structure of metaphors. This study will help us understand how the technical debt metaphor has evolved and how it continues to evolve. Even more important, a study of metaphors will help us understand and anticipate the communication problems that arise from the fact that the term technical debt is a metaphor. We’ll look at the structure of metaphors next time.

References

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Cunningham 2011] Ward Cunningham. “Ward Explains Debt Metaphor” (video; here; ).

Cited in:

Related posts

Glossary and Terminology

Last updated on October 8th, 2019 at 07:27 am

Even though technical debt has been with us for a very long time—probably since the time we began inventing technologies—the study of technical debt is relatively new. Ward Cunningham coined the term technical debt in 1992, and its meaning has evolved since then. Because universally accepted definitions for the term and associated concepts have not yet emerged, it seems necessary to have a page on this site that collects definitions.

Asset-exogenous technical debt

Exogenous technical debt is asset-exogenous when it’s brought about by an activity external to an asset, but internal to the enterprise. For example, a change in standards or regulations by some body within the enterprise can cause an asset to incur an asset-exogenous technical debt.

ATD

See Auxiliary technical debt.

Auxiliary technical debt

In the context of a Technical Debt Retirement Project (DRP) that has as an objective retiring from a specified set of assets a particular kind or particular kinds of technical debt, the ATD is the collection of instances of any other kinds of technical debt other than the kind that the DRP is trying to retire. More: “Rules of engagement for auxiliary technical debt

Class of technical debt

On occasion, we speak of classes of technical debt and instances of that class. This can be confusing, because the words class and instance have particular meanings in software engineering. That’s not the sense in which we use the terms here. In this blog, a class of technical debt is just a collection of instances of the same kind of debt. For example, consider the “ghost ramp” described in “Technical debt in the highway system.” It belongs to the class of ghost ramps. If we were maintaining the highway system of Massachusetts, it might be convenient to consider the class of ghost ramp technical debt if we want to let a contract to demolish all ghost ramps. Each ghost ramp would then be an instance of that class.

Cognitive bias

A cognitive bias is the human tendency to make systematic errors based not on evidence, but on factors related to the thought process. Psychologists have identified and demonstrated hundreds of cognitive biases, including several that could plausibly explain failures in priority setting for technical debt retirement projects.

Confirmation bias

Confirmation bias is a cognitive bias. It’s the human tendency to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. For example, the homogeneity of cable news channel audiences, and the alignment between preconceptions of the audience and the slant of the newscast for that channel, are results of confirmation bias. More: “Confirmation bias and technical debt

Debt contagion

If a class of technical debt is allowed to remain outstanding, its volume can increase as a consequence of seemingly unrelated actions or decisions. Moreover, its existence can cause increases in the volume of other existing classes of technical debt, and its existence can lead to the formation of new classes of technical debt. This process is called debt contagion. More: “How technical debt can create more technical debt

DRP

In this blog, I use the term DRP to mean a (technical) Debt Retirement Project. A DRP is a project that has as an objective retiring from a specified set of assets a particular kind of technical debt (or particular kinds of technical debt). Many projects have objectives of debt retirement, at some point or other. But DRPs differ from most, in that debt retirement is their primary objective—indeed, it might be their sole objective. More: “Nine indicators of wickedness

Echo release

An echo release of an asset is a release version whose primary purpose is technical debt retirement. Typically, it’s created immediately following a release version that has created some incremental technical debt, hence the term “echo release.” The echo release is then executed to retire that incremental technical debt, and not to repair defects or add capability. More: “Accounting for technical debt

Endogenous technical debt

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing).  So endogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that directly involve the asset. More: “Exogenous technical debt

Enterprise-exogenous technical debt

Exogenous technical debt is enterprise-exogenous when it’s brought about by an activity external to the enterprise. For example, a change in standards or regulations by some body outside the enterprise can cause an asset to incur an enterprise-exogenous technical debt.

Exogenous technical debt

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that don’t involve the asset directly. More: “Exogenous technical debt

Ill-structured problem

An ill-structured problem is a problem that isn’t a well-structured problem [Simon 1973]. An example of an ill-structured problem is finding a definition for ill-structured problems. Another: designing a computer programming language. Still another, even more to the point: deciding when to retire a particular class of technical debt. NDM is more likely to be successful with ill-structured problems than is RDM.

Incremental technical debt

Incremental technical debt is either newly incurred exogenous technical debt, or technical debt that’s incurred in the course of work currently underway or just recently completed. For example, in an apartment building hallway renovation project, workmen did insert expansion joints in the sheetrock they replaced, but on the first three floors they completed, the joints were too widely separated. The remaining 22 floors were done correctly. Nine additional joints on each of the incorrect floors must be inserted eventually. The missing joints, which constitute incremental technical debt, will be inserted after the job is completed. More: “Controlling incremental technical debt

Instance of technical debt

See “Class of technical debt

Intertemporal choice

Confronted with advice from technical experts regarding the urgent need to address the burden of enterprise technical debt, decision makers must consider an unpleasant possibility. To make resources available to retire the technical debt, it might be necessary to temporarily defer investment in some new products or enhancing some existing products. And if they make the recommended investments in technical debt retirement, customers won’t benefit in any visible way. So the choice reduces to one between new products and enhancements relatively sooner, versus retiring technical debt and only later attending to new products and enhancements of existing products. This dilemma is an example of what behavioral economists call intertemporal choice [Loewenstein 1992].

Key Performance Indicator (KPI)

A Key Performance Indicator (KPI) is a metric that provides meaningful insight that’s used to guide business decisions. All KPIs are metrics; not all metrics are KPIs. More: “Metrics for technical debt management: the basics

Legacy technical debt

Legacy technical debt is technical debt associated with an asset, and which exists in any form prior to undertaking work on that asset. For example, in planning a project to renovate the hallways and common areas of a high-rise apartment building, Management discovers that beneath the existing carpeting is a layer of floor tile containing asbestos. Management has decided to remove the tile. In this context, the floor tile can be viewed as legacy technical debt. It isn’t directly related to the objectives of the current renovation, but removing it will enhance the safety of future renovations, enable certification of the building as asbestos-free, and reduce the cost of eventual demolition. More: “Exogenous technical debt

Localizable technical debt

Localizable technical debt is technical debt that manifests itself as discrete chunks. Each instance is self-contained, and we can “point” to it as an instance of the debt in question. For example, if the organization regards Windows 10 as the current operating system for personal computers, and early versions of Windows as technical debt, the each computer that runs and earlier version of Windows is an instance of that technical debt. Each instance is discrete and localized. More: “Retiring localizable technical debt

Measure

A measure is the result of determining the value of a quantifier. For example, we might use the quantifier’s definition to determine a measure of how much human effort has been expended on an asset in the past fiscal quarter. More: “Metrics for technical debt management: the basics

Metric

A metric is an arithmetic formula expressed in terms of constants and a set of measures. One of the simpler metrics consists of a single ratio of two measures. For example, the metric that captures the average cost of acquiring a new customer in the previous fiscal quarter is the ratio of two measures, namely, the investment made in acquiring new customers, and the number of new customers acquired. More: “Metrics for technical debt management: the basics

MICs, or metaphorical interest charges

MICs are the metaphorical interest charges associated with a technical debt. They aren’t interest charges in the financial sense; rather, the MICs of a technical debt represent the total of reduced revenue, lost opportunities, and increased costs of all kinds borne by the enterprise as a consequence of carrying that technical debt. Because the properties of MICs are very different from the properties of financial interest charges, we use the term MICs to avoid confusion with the term interest from the realm of finance. More: “How financial interest charges differ from interest charges on technical debt

MPrin, or metaphorical principal

The MPrin of a technical debt at a give time T is the total cost of retiring that debt at time T. The total cost includes all cost factors: labor, equipment, service interruptions, revenue delays, anything. It even includes the ongoing costs of repairing defects introduced in the debt retirement process. More: “The metaphorical principal of a technical debt

Naturalistic decision-making

Naturalistic decision-making (NDM) entails situation assessment and evaluation of a single option to select a satisfactory option. [Zannier 2007] Features that define naturalistic decision-making are “time pressure, high stakes, experienced decision makers, inadequate information (information that is missing, ambiguous, or erroneous), ill-defined goals, poorly defined procedures, cue learning, context (e.g., higher-level goals, stress), dynamic conditions, and team coordination.”  [Klein 2017]

Non-strategic technical debt

Non-strategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce non-strategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. And at times, it appears in the asset as a result of external events beyond the boundaries of the enterprise. More: “Non-technical precursors of non-strategic technical debt

The planning fallacy

The planning fallacy is a cognitive bias that causes planners to underestimate costs and schedules, and over-promise benefits, because they pay too little heed to past experience on similar efforts, and rely too much on what they believe will happen on the effort they’re planning. First identified in a 1977 report by Daniel Kahneman and Amos Tversky [Kahneman 1977] [Kahneman 1979]. More: “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma

Policy

Organizational policy is the framework of principles that guide policymakers, decision makers, and everyone in the organization as they carry out their responsibilities. Policy might be written or not, but written policy is more likely to consistently adhered to. Interestingly, the body of organizational policy is itself subject to accumulating technical debt. More: “What is policy?

Policymaker

As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management. More: “Who are the policymakers?

Quantifier

A quantifier is a specification for a measurement process designed to yield a numeric representation of some attribute of an asset or process. Quantifiers are used to obtain the values called measures, which in turn are used in computing metrics. More: “Metrics for technical debt management: the basics

Rational decision-making

Rational decision-making (RDM) is an approach to making a choice of an option from among a set of options by selecting the option that is optimal with respect to a set of quantitative criteria. [Zannier 2007] Rational choice strategies generally follow this framework: (1) Identify a set of options; (2) Identify criteria for evaluating them; (3) Assign weight to each evaluation criterion; (4) Rate the options relative to the criteria; (5) Choose the option with the highest score. Many different frameworks for implementing this strategy are available, some specialized to specific subject domains [Thokala 2016].

Refactoring

Fowler defines refactoring as “the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure” [Fowler 1999]. Although refactoring is a term specific to software development processes, the concept applies to all technological development. For example, an automobile manufacturer’s decision to alter the design of one of their model vehicles to reduce manufacturing costs can be viewed as a form of refactoring. Refactoring is a practice essential to effective technical debt management. More: “Refactoring for policymakers

Regression testing

Regression testing is a testing regimen that ensures that a previously developed and tested system still performs the same way after it has been altered or when it’s used in a new context. Regression testing is essential when we alter a system by retiring some of its technical debt.

The reification error

The reification error (also called the reification fallacy, concretism, or the fallacy of misplaced concreteness) is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing. Reification is useful in some applications, such as object-oriented programming and design. But when we use it in the domain of logical reasoning, troubles can arise. Specifically, we can encounter trouble when we think of “measuring” technical debt. Strictly speaking, we cannot measure technical debt. We can estimate the cost of retiring it, but estimates are only approximations. And in the case of technical debt, the approximations are usually fairly rough. To regard these estimates as measurements is to risk reifying them. Then when the actual cost of a debt retirement project is dramatically larger than the estimate, the consequences for enterprise budgets can be severe. We must always regard “measurements” of technical debt as estimates—estimates that are so prone to error that we must plan for error.  The reification error is the dual of the resilience error. More: “Metrics for technical debt management: the basics.”

The resilience error

If the reification error is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing, the resilience erroris an error of reasoning in which we treat an abstraction as if it were more flexible, resilient, and adaptable than it actually is. When we commit the resilience error with respect to an abstraction, we adopt the belief—usually without justification, and possibly outside our awareness—that if we make changes in the abstraction without fully investigating the consequences of those changes, we can be certain that the familiar properties of the abstraction we modified will apply, suitably modified, to the new form of the abstraction.  Or we assume incorrectly that the abstraction will accommodate any changes we make to its environment. The resilience error is the dual of the reification error. We are at risk of making the resilience error when we refactor assets to reduce their burden of technical debt. More: “The resilience error and technical debt.”

Secured technical debt

A secured technical debt, like a secured financial debt, is one for which the enterprise has reserved the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. They might include particular staff, equipment, test beds, downtime, and financial resources. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes. More: “Using SMART goals for technical debt reduction

Source and target components of a metaphor

In a metaphor of the form “A is B,” the source is the element whose attributes are being attributed to the target. For example, in “my son’s room is a war zone,” the source is the war zone, and the target is my son’s room.  More: “The structure of metaphors

Super wicked problem

A subset of wicked problems can be viewed as super wicked [Levin 2012]. Levin, et al. list the following four properties of super wicked problems: (1) Time is running out; (2) Those who cause the problem also seek to provide a solution or influence the solution; (3) The central authority needed to address the problem is weak, non-existent, or chooses not to act effectively; (4) Policy responses discount the future irrationally. I’ve come to believe that some technical debt retirement project design can be a super wicked problem. More: “Retiring technical debt can be a super wicked problem

Taylorism

Taylorism is an approach to management developed by Frederick Winslow Taylor in the early part of the twentieth century [Taylor 1913] [Kanigel 1997]. He proposed three principles of scientific management could produce maximum efficiency: (1) scientific selection of the person performing the work; (2) scientific breakdown of tasks; and (3) separating planning from execution. These principles are the basis of what became known in software engineering as the waterfall lifecycle. The approach works well for well-structured problems, but does not work well at all for ill-structured problems. Moreover, it depends for success on repeating solutions to problems already solved, which is why it proved so valuable in early manufacturing. Its unsuitability for ill-structured problems is an important part of the basis for the Agile approach to problem solving.

TDIQ

In the context of a Technical Debt Retirement Project (DRP) that has as an objective retiring from a specified set of assets a particular kind of technical debt (or particular kinds of technical debt), the TDIQ is the Technical Debt In Question. More: “Retiring technical debt in irreplaceable assets

Technical debt

Technical debt is any technological element that contributes, through its existence or through its absence, to lower productivity or to a higher probability of defects during development, maintenance, or enhancement efforts, or which depresses velocity in some other way, and which we would therefore like to revise, repair, replace, rewrite, create, or re-engineer for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it. More: “A policymaker’s definition of technical debt

Technological communication risk

Technological communication risk is the risk that, for whatever reason, knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. More: “Technological communication risk

Temporal discounting

Temporal discounting is the human tendency to give greater value to a reward (or as economists would say, to assign greater utility to a good) the earlier it arrives. An analogous process affects perceptions of inconvenience or disutility: people assign more negative values to penalties and inconveniences the sooner they arrive. If the discount rate is constant, the discounting is termed exponential discounting or rational discounting. But other forms are possible. Hyperbolic discounting is one form of discounting at a rate that is higher for near-term arrivals than for distant-term arrivals [Laibson 1997]. Humans have been observed experimentally to favor a form of temporal discounting that is well modeled as hyperbolic discounting.

Terrifying opportunity

A terrifying opportunity arises when the organization rejects (or fails to recognize) a market opportunity because exploiting it would involve modifying an existing asset or product offering that harbors a heavy load of technical debt. The debt causes decision-makers to assess that the probability of success is so low that the opportunity seems terrifying, and they therefore reject the opportunity. More: “MICs on technical debt can be difficult to measure

Well-structured problem

As defined by Simon [Simon 1973], a well-structured problem is a problem that has some or all of six characteristics. The first is the existence of a definite criterion for testing any proposed solution, and a mechanizable process for applying that criterion. Second, there is at least one problem space in which we can represent the initial problem state, the goal state, and all states that can be reached or considered while solving the problem. There are four more criteria, but these are the biggies. An example of a well-structured problem is the game of chess. RDM is useful for attacking well-structured problems.

Wicked problem

A problem is a wicked problem if it meets the ten criteria established by Rittel and Webber [Rittel 1973]. Four of the criteria: it’s an ill-structured problem; it’s incompletely defined or internally contradictory; its solutions are not true-or-false, but good-or-bad; and there’s no way to exhaustively describe all solutions. I’m convinced that technical debt retirement project design can be a wicked problem. More: “Self-sustaining technical knowledge deficits during contract negotiations.”

References

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Cunningham 2011] Ward Cunningham. “Ward Explains Debt Metaphor” (video; here; ).

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

A policymaker’s definition of technical debt

Last updated on January 20th, 2019 at 01:54 pm

Policymakers have in mind the best interests of the entire enterprise. They need a definition of technical debt that is neutral relative to its causes and manifestations, because defining technical debt in terms of what caused it or where it lies in the enterprise could compromise that necessary neutrality.

Servers like the ones that made this page available to you
Servers like the ones that made this page available to you. Cybersecurity is concerned with defending servers like these, among others.

For example, if enterprise policy assumes that technical debt is something that lies only in software, and if the root causes of some instances of technical debt are new and developing threats in the cybersecurity environment, then enterprise policy vis-à-vis technical debt is likely to be ineffective. It might lead to decision makers focusing too much attention on the software development process and too little attention on the cybersecurity and threat intelligence processes.

Here’s a cause-neutral and manifestation-neutral definition of technical debt, what I call the policymaker’s definition [Brenner 2017a]:

Technical debt is any technological element that contributes, through its existence or through its absence, to lower productivity or to a higher probability of defects during development, maintenance, or enhancement efforts, or which depresses velocity in some other way, and which we would therefore like to revise, repair, replace, rewrite, create, or re-engineer for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it.

Extending the technical debt metaphor just a bit, people often talk about the principal and the interest charges associated with a technical debt, by analogy to the principal and interest charges associated with a financial debt. They’re convenient concepts, but the parallels between finance and technology aren’t real, and that’s where the trouble lies. Read more

There’s one other generalization contained in this definition of technical debt that differs from most other definitions. It’s in the phrase “or missing from.” Our policymaker’s definition doesn’t require that the technical debt item actually exist. That is, the absence of something can constitute technical debt. My favorite example of this was put forward by Ken Pugh, who defines acceptance test debt as “…the nonexistence or nonautomation of acceptance tests.” [Pugh 2010] If we seek to encompass all sources of reduced organizational responsiveness or unnecessary operating expense arising from technical debt, the definition of technical debt that we require must also address non-existence issues such as those identified by Pugh.

The definition above is workable for systems of all kinds. Consider two examples of “hardware”:

But the definition also applies to anything that is manifested in technological forms, including business plans, legislation, procedures, and microprocessor designs—almost anything.

References

[Brenner 2017a] Richard Brenner. “A Policy Maker’s Definition of Technical Debt,” Cutter Consortium Executive Update, February 27, 2017.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Cunningham 2011] Ward Cunningham. “Ward Explains Debt Metaphor” (video; here; ).

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

Related posts

Technical debt in a rail system

Last updated on June 14th, 2018 at 08:24 pm

Acela Express rounds a curve in Connecticut
Acela Express rounds a curve in Connecticut. Shown is the trailing power car of a southbound Acela Express and the front of a northbound Metro-North railcar.

Most definitions of technical debt require that the asset bearing the debt be software. From the policymaker’s perspective, this requirement is rather limiting. So for the purposes of this blog, I define technical debt as any property of a technological asset that we would like to revise, replace, or create, and which limits the ability of the enterprise to gain or maintain a dominant market leadership position.

Consider an example from the railroad industry. In the United States, the highest-speed rail line is Acela Express, which travels in the northeast corridor between Boston and Washington, D.C. Parts of the right-of-way, track, and catenary it uses were not originally designed for this application, and therefore trains cannot operate at their highest possible speed [Maloney 2000]. On the 231-mile section from Boston to New York’s Penn Station, Acela achieves an average speed of only 63 mph (101 km/h), even though the trains can operate safely on straight track at 150 mph (240 km/h). Yet, Acela still manages to capture a 54% share of the total air and rail market between these two cities.

That 54% share might be higher still if not for technical debt. To compensate for centrifugal forces as Acela Express rounds curves, its passenger cars are able to tilt the passenger spaces to enable the train to round the curves at higher speeds than would otherwise be comfortable for passengers. In effect, the cars “lean into” the curves, just as a running athlete leans when making a sudden change of direction. Although the cars were designed to be able to tilt by as much as 6.8º, the adjacent set of tracks is too close to permit this without risk of collision with trains on those tracks. The maximum tilt is therefore set at 4.2º, which, in turn, reduces the maximum speed consistent with passenger comfort that the trains can attain on curves. The technical debt manifested in the tracks Acela Express uses thus prevents it from offering a service that would be more competitive with alternative transport modes, especially airlines.

Note: In August 2016, Amtrak announced that it will be upgrading its trainsets and tracks to exploit new technologies, including active tilt technologies. All existing trainsets are due to be replaced in 2021-22.

References

[Brenner 2017a] Richard Brenner. “A Policy Maker’s Definition of Technical Debt,” Cutter Consortium Executive Update, February 27, 2017.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Cunningham 2011] Ward Cunningham. “Ward Explains Debt Metaphor” (video; here; ).

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Maloney 2000] Brenna Maloney and Don Phillips. “All Aboard AMTRAK’s Acela,” The Washington Post, November 30, 2000.

Available: here; Retrieved April 18, 2017.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

Related posts

The structure of metaphors

Last updated on December 17th, 2017 at 05:15 pm

A metaphor is a figure of speech that references one concept, object, or action, by identifying it with another, when that identification is not literally accurate. Metaphors help us understand and experience one thing, with which we might be unfamiliar, in terms of another, with which we are more familiar [Lakoff 1980].

A squirrel

For example, “my son’s room is a war zone,” identifies my son’s room as a war zone, when it is not literally a war zone. We mean by this that his room might be messy and disorganized, but we do not mean to imply that military ordnance or troops are involved. More examples:

  • True friends stab you in the front. — Oscar Wilde
  • A squirrel is just a rat in a cuter outfit. — “Carrie Bradshaw,” played by Sarah Jessica Parker in Sex in the City
  • A bank is a place where they lend you an umbrella in fair weather and ask for it back when it begins to rain. — variously attributed to Robert Frost, Mark Twain, and others

In these examples, Oscar Wilde is not saying that friends actually stab anyone; “Carrie Bradshaw” is not saying that squirrels are rats, or that they wear clothing; and Frost or Twain are not saying that banks actually lend umbrellas. Nevertheless these three statements do literally imply stabbings, squirrel clothing, and umbrella distribution. These metaphors make their points by being literally inaccurate. The literal but untrue assertion is the hallmark of the metaphor.

The fundamental structure of metaphors is “A is B.” Borrowing terminology from cognitive linguistics, A, the main entity referenced, is called the target of the metaphor; B, the entity alluded to, is called the source. Thus, the squirrel is the target; the rat in a cuter outfit is the source. The bank is the target; the perfidious umbrella lender is the source. For the technical debt metaphor, the needed technical work is the target; financial obligation or financial debt is the source. Metaphors serve to aid us in applying what we understand well in the domain of the source, to what we understand less well in the domain of the target.

DefinitionA metaphor is a figure of speech used to convey understanding of one concept, object, or action by identifying it with another that is well understood, even though the identification is not literally accurate. The well-understood concept is called the source. The less-well-understood concept is called the target. The metaphor is thus a statement that “<Target> is <Source>.” Although the identification of target with source is literally invalid, it provides a means of understanding some aspects of the target in terms of some of the properties or behavior of the source.

Because metaphors compel our minds to accept the identification between source and target in toto, they can cause us to make errors of thought. Those errors create risks for the enterprise as we attempt to manage technical debt. The risk arises because we begin to regard technical debt as a form of financial debt, when in reality it is not. This misidentification is an acceptable risk, if it is properly managed and understood, but that risk is often unrecognized, and therefore it remains unmitigated. A significant source of this risk is our inability to control which attributes of the metaphor’s source the reader or listener chooses to associate with the metaphor’s target. We can call this phenomenon unintended association.

Some of the unintended associations of the technical debt metaphor cause real problems for organizations as they try to manage their technical debt. We explore the unintended associations of the technical debt metaphor next time.

References

[Brenner 2017a] Richard Brenner. “A Policy Maker’s Definition of Technical Debt,” Cutter Consortium Executive Update, February 27, 2017.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Cunningham 2011] Ward Cunningham. “Ward Explains Debt Metaphor” (video; here; ).

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Lakoff 1980] George Lakoff and Mark Johnson. Metaphors We Live By. Chicago: The University of Chicago Press, 1980.

The classic and fundamental study of metaphor. Order from Amazon

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Maloney 2000] Brenna Maloney and Don Phillips. “All Aboard AMTRAK’s Acela,” The Washington Post, November 30, 2000.

Available: here; Retrieved April 18, 2017.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

Related posts

Show Buttons
Hide Buttons