Exogenous technical debt

Last updated on July 9th, 2021 at 04:58 pm

Exogenous technical debt is debt that arises from causes not directly related to the asset that bears the debt. Mastering understanding of exogenous technical debt is essential to controlling technical debt formation. Exogenous technical debt is particularly troublesome to those who work on the affected assets. They can’t control its formation, and they’re rarely responsible for creating it. But their internal customers and those who control resources often fail to understand this. Indeed, those who work on the affected assets often bear blame for the formation of exogenous technical debt even though they had no role in its formation, and could have done nothing to prevent its formation.

Exogenous technical debt and endogenous technical debt

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that don’t involve the asset directly.

Why we must track exogenous technical debt

Asbestos with muscovite.
Asbestos with muscovite. Asbestos is a family of minerals occurring naturally in fibrous form. The fibers are all known carcinogens. Until 1990, asbestos was a common ingredient of building materials, including insulation, plaster, and drywall joint compound. It’s now banned, but it’s present in existing homes and offices. The ban caused these structures to incur exogenous technical debt. Photo by Aramgutang courtesy Wikipedia.

Because so much technical debt arises indirectly, controlling its direct formation is insufficient to achieve control. To control technical debt formation, we must track which activities produce it. We must track both direct and indirect effects. Allocating technical debt retirement costs to the activities that brought that debt about is useful. It’s useful even if the allocation doesn’t affect budget authority for those activities. Knowledge about which past activities created technical debt, and how much, is helpful for long-term reduction in the rate of technical debt formation.

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries—or which will be carrying—the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing). So endogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that directly involve the asset.

More about endogenous technical debt in future posts. For now, let’s look more closely at exogenous technical debt, and its policy implications.

Examples of exogenous technical debt

In “Spontaneous generation,” I examined one scenario in which technical debt formation occurs spontaneously—that is, in the absence of engineering activity. Specifically, I noted how the emergence of the HTML5 standard led to the formation of technical debt in some (if not all) existing Web sites. This happened because those sites didn’t exploit capabilities that had become available in HTML5. Moreover, some sites needed rehabilitation to remove emulations of the capabilities of the new standard. Those emulations needed to be replaced with use of facilities in the HTML5 standard. All of these artifacts—including those that existed, and those that didn’t—comprised technical debt. This scenario thus led to the formation of exogenous technical debt.

In a second example, AMUFC, A Made-Up Fictitious Corporation, incurs technical debt when the vendor that supplies the operating system (OS) for AMUFC’s desktop computers announces the date of the end of extended support for the version of the OS in use at AMUFC. Because the end of extended support brings an end to security updates, AMUFC must retire that debt by migrating to the next version of that vendor’s OS before extended support actually ends.

In both examples, the forces that lead to formation of exogenous technical debt are external to the enterprise and the enterprise’s assets. But what makes technical debt exogenous is that the forces that led to its formation are unrelated the engineering work being performed on the asset. This restriction is loose enough to also include technical debt that arises from any change or activity external to the asset, but within the enterprise.

Exogenous technical debt arising from actions within the enterprise

Exogenous technical debt can arise from activities or decisions that take place entirely within the enterprise.

For example, consider the line of mobile devices of AMUFC (A Made-Up Fictitious Corporation). Until this past year, AMUFC has been developing ever more capable devices. These efforts extended its line of offerings at the high end—the more expensive and capable members of the line. But this past quarter, AMUFC developed a low-end member of the line.

As often happens, price constraints for the low-cost device led to innovations. Those innovations could produce considerable savings in manufacturing costs if used all across the line. In effect, the designs of the previously developed higher-end models have incurred exogenous technical debt. The debt is exogenous because the activity that led to debt formation wasn’t performed on the assets that carry the debt. The debt is real, even though the activity that led to debt formation occurred within the enterprise. This kind of exogenous technical debt is asset-exogenous. Exogenous technical debt of the kind that results from activity beyond the enterprise is enterprise-exogenous.

Exogeneity versus endogeneity

For asset-exogenous technical debt, ambiguity between endogeneity and exogeneity can arise. The example above regarding the line of mobile devices produced by AMUFC provides an illustration.

For convenience, call the team that developed one of the high-end devices Team High. Call the team that developed the low-end device Team Low. From the perspective of Team High, the technical debt due to the innovations discovered by Team Low is exogenous. But from the perspective of the VP Mobile Devices, that same technical debt might be regarded as endogenous. The debt can be endogenous at VP level because it’s possible to regard the entire product line as a single asset, and that might actually be the preferred perspective of VP Mobile Devices.

This ambiguity can lead to some nasty toxic conflict. Team High and VP Mobile Devices might attack each other as they try to defend themselves proactively against claims that they are incurring technical debt. Avoiding this kind of conflict requires educating everyone as to the origins of technical debt.

Exogeneity and legacy technical debt

The technical debt portfolio of a given asset can contain a mix a technical debt that arose from various past incidents. In assessing the condition of the asset, it’s useful to distinguish this existing debt from debt that’s incurred as a consequence of any current activity or decisions. Call this pre-existing technical debt legacy technical debt.

The legacy technical debt an asset carries is technical debt associated with the asset, and which existed in any form before undertaking work on that asset. For example, consider planning a project to renovate the hallways and common areas of a high-rise apartment building. Suppose workers discover beneath the existing carpeting a layer of asbestos floor tile. Then Management might decide to remove the tile. In this context, we can regard the floor tile as legacy technical debt. It isn’t directly related to the objectives of the current renovation. But removing it will enhance the safety of future renovations. It will also enable certification of the building as asbestos-free, increase the property value, and reduce the cost of eventual demolition. In this situation asbestos removal is retirement of legacy technical debt. Accounting for it as part of the common-area renovation would be misleading.

Exogeneity is relevant when allocating resources for legacy technical debt retirement efforts. If the debt in question is enterprise-exogenous, we can justifiably budget the effort from enterprise-level accounts. For other cases, other resources become relevant, depending on what actions created the debt. For example, suppose that the technical debt arose from a change in enterprise standards. Then we can justifiably allocate retirement costs to the standard-setting initiative. If the exogenous technical debt arose from innovations in other members of the asset’s product line, we can can justifiably allocate those debt retirement costs to the product line.

Policy insights

Understanding the properties of exogenous technical debt can be a foundation for policy innovations that enhance enterprise agility.

Culture transformation

Widespread understanding of the distinction between exogenous and endogenous technical debt is helpful in controlling interpersonal conflict. For example, it can reduce blaming behavior that targets the engineering teams responsible for developing and maintaining technological assets.

Understanding asset-exogenous technical debt helps non-engineers understand how their actions and decisions can lead to technical debt formation. The concept clarifies the import of their actions even when there is no apparent direct connection between those actions or decisions and the assets in question.

Resource allocation

Data about the technical debt creation effects of enterprise activities is helpful in allocating technical debt retirement costs. For example, suppose that we know all the implications of reorganization, including its impact on internal data about the enterprise itself. Then we can charge data-related activity to the reorganization instead of to general accounts of the Information Technology function. This helps the enterprise understand the true costs of reorganization.

Similarly, data about enterprise-exogenous technical debt helps planners understand how to deploy resources to gather external intelligence about trends that can affect internal assets. Such data is also useful for setting levels of support and participation in industrial standards organizations or in lobbying government officials.

Last words

Knowing the formation history of exogenous technical debt provides useful guidance for those charged with allocating the costs of retiring technical debt or preventing its formation.

Other posts in this thread

Spontaneous generation

Last updated on July 9th, 2021 at 12:36 pm

Technical debt needn’t result from anyone’s conscious decision. In some instances, technical debt seems to appear as if by spontaneous generation. And that creates problems for technical debt management programs that assume that technical debt results from employee decisions in the form of the intentions of engineers or others.

U.S. Army soldiers, along with volunteers from the community, install roof trusses for a Habitat for Humanity home in Brainerd, Minn., July 13, 2010
U.S. Army soldiers, along with volunteers from the community, install roof trusses for a Habitat for Humanity home in Brainerd, Minnesota, July 13, 2010. Hurricane ties are in place at the top of the wall as the roof trusses are being placed. The Florida state building code was strengthened in 2002 to require hurricane ties to strengthen the connection between the roofs and walls of buildings. Homes built before 2002, and which lack hurricane ties, are therefore carrying technical debt. Retiring that debt is difficult. It involves retrofitting hurricane ties which usually requires cutting holes in the home’s siding—one for each tie—and then repairing the holes. Photo by Sgt. Nicholas Olson, courtesy  Wikimedia Commons, where you can find a larger version of this image.
Although knowing author or engineer intention relative to technical debt artifacts can be helpful when organizations plan or execute technical debt retirement programs, sound technical debt management policy must address situations in which author or engineer intention wasn’t a contributing factor in creating the debt, or intention can’t be determined, or intention is concealed. Classifications of technical debt must therefore consider business strategy and resource availability as well as author intention.

This difference in priorities contributes to tension between technologists and policymakers with respect to their definitions of technical debt.

Accepted classification frameworks are technology­oriented

Within enterprises of significant size, classifying technical debt is an essential step in designing programs for reducing the cost of carrying technical debt. Although the choice of classification scheme depends on one’s objectives, most classification schemes explored so far in the technical debt literature are more suitable for use by technologists than by policymakers. But, unsurprisingly, the assistance they provide to policymakers relates mostly to policies that affect technologist behavior or resource allocation across technical activities.

An example might clarify the issue. Technologists tend to create classifications of technical debt that emphasize designer and implementor intentions. For example, Fowler has created a widely accepted two-dimensional [Lowy 2004] classification [Fowler 2009] that characterizes technical debt according to the Degree of Wisdom in incurring it (he calls this dimension Reckless/Prudent), and Degree of Intentionality in incurring it (he calls this dimension Deliberate/Inadvertent). See “Technical debt in software engineering” for more.

In the technical literature, this classification, and another due to McConnell [McConnell-slides 2013] are widely but not universally accepted. For example, some believe that no artifact is technical debt unless its presence (or absence) was the result of a conscious decision [Adobe Blogs 2014]. Some adherents of this view would reject all of Fowler’s “Inadvertent” technical debt.

This focus on engineering intention likely arises, in part, for two reasons. First, technologists tend to have good access to their own intentions, and to the intentions of other technologists. The second reason relates to developing plans for retiring particular classes of technical debt. Knowledge of the intentions of the people who created (or omitted) the artifacts in question is helpful in planning debt retirement projects..

Policymakers need a more inclusive classification framework

For policymakers, while both of these widely accepted classifications are helpful, they are inadequate. Intentionality with respect to technical debt formation is indeed a valuable consideration in developing technical debt policy. But technical debt can arise for reasons unrelated to engineers’ intentions. Indeed, it can arise for reasons unrelated to any enterprise activity. For these reasons intention-based classifications provide inadequate guidance for policy formation.

Consider technological advancement that arises from sources external to the enterprise. For example, with the emergence of the HTML5 standard, many Web sites became obsolete. They didn’t exploit capabilities that had become available. These sites were in need of updating to remain competitive in their markets. The new standard also created a ned to replace capabilities that emulated the new standard, but which exploited alternative technologies. All of these artifacts—including those that existed, and those that didn’t, comprise technical debt.

Relative to technical debt management, an enterprise that devotes resources to monitoring external technology trends would have an advantage over competitors that tend to focus solely on employee behavior.

Last words

Technological advancement that occurs outside the enterprise can thus create technical debt within the enterprise. This is just one example of spontaneous generation of technical debt. Thinking about technical debt this way, you can probably identify other sources of spontaneous generation. Together, they create a need for policies that can address their management.

Other posts in this thread

References

[Adobe Blogs 2014] Adobe Blogs. “What is Technical Debt?,” Adobe Blogs, September 8, 2014.

Available: here; Retrieved February 26, 2017.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Lowy 2004] Alex Lowy and Phil Hood. The Power of the 2x2 Matrix: Using 2x2 Thinking to Solve Business Problems and Make Better Decisions. Jossey-Bass, 2004.

Order from Amazon

Cited in:

[McConnell-slides 2013] Steve McConnell. “Managing Technical Debt”, ICSE 2013.

Available: here; Retrieved November 11, 2017

Cited in:

Adopt an enterprise-wide definition of technical debt

Last updated on July 9th, 2021 at 11:25 am

Because effective technical debt management requires cooperation from almost everyone, an enterprise-wide definition of technical debt is essential. Absent a shared definition of technical debt, controversy can develop. Controversy is especially likely among those who have previously encountered the concept—namely, among technologists. Policymakers can make invaluable contributions to the design of the cultural transformation that will enable control of technical debt.

A physical expression of shared commitment, essential to adopting an enterprise-wide definition of technical debt
A physical expression of shared commitment. Effective technical debt management requires both a shared understanding of what it is and a shared commitment to do what’s required to control it.

Li et al. [Li 2015] found that defining what is and what isn’t technical debt remains an open question in software engineering. Even if we restrict the discussion to software constructed in-house, opinions about what constitutes technical debt do differ. The authors found that in the literature of technical debt, “The term ‘debt’ has been used in different ways by different people, which leads to ambiguous interpretation of the term.”

This finding is perhaps the most significant for policymakers. It suggests that controlling technical debt might require forging an organizational consensus about the meaning of the term technical debt. The people of most organizations come from many different backgrounds. Those with little knowledge of technical debt have few preconceptions. But those who are aware of the issue probably interpret the term technical debt in a variety of ways. Because some of those who do have awareness of the term are likely to have strong opinions about its meaning, one can anticipate a need to resolve these differences.

The effect of an absence of standards

Some technical terms, like RAID, byte, compiler, and kilowatt, have widely accepted standard definitions. Although the term technical debt has found wide use, it has no standard definition. What some people categorize as technical debt, others do not. Those who are accustomed to working with terms that have precise, widely accepted definitions might tend to assume that the term technical debt does have (or should have) one as well. This assumption can create some difficulty for people who don’t realize that others might have differing views of the definition of the term.

For example, there are those who believe that defects are not technical debt. And some believe that no element of a technological asset can constitute technical debt unless it is part of a product that a customer uses. Our definition differs with both of these views.

Last words

Policymakers must be aware that there is a lack of consensus about the definition of technical debt. Our definition, crafted specially for the use of policymakers, might seem unusually broad to technologists and engineers. For that reason alone, it’s advisable to become familiar with the various ways technologists use the term. Understanding their perspective is essential to formulating sound policy deserving of their respect.

Other posts in this thread

References

[Adobe Blogs 2014] Adobe Blogs. “What is Technical Debt?,” Adobe Blogs, September 8, 2014.

Available: here; Retrieved February 26, 2017.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Lowy 2004] Alex Lowy and Phil Hood. The Power of the 2x2 Matrix: Using 2x2 Thinking to Solve Business Problems and Make Better Decisions. Jossey-Bass, 2004.

Order from Amazon

Cited in:

[McConnell-slides 2013] Steve McConnell. “Managing Technical Debt”, ICSE 2013.

Available: here; Retrieved November 11, 2017

Cited in:

Tension between policymakers and technologists

Last updated on July 11th, 2021 at 08:57 am

Surface tension holds raindrops together on a grapevine
Surface tension holds raindrops together on a grapevine. We often think of tension as a negative, destructive force. But tension—in the case of raindrops, surface tension—is what holds a raindrop together. Tension gives the raindrop structure and integrity. The tension between technologists and policymakers can also be constructive. It can ensure that the enterprise manages technical debt in ways that balance the political imperatives of technology and strategic health.

Tension between policymakers and technologists, which can manifest as misalignment of their respective priorities, is a significant contributor to uncontrolled growth of technical debt. In this thread, I explore sources of this tension. That exploration introduces concepts that can assist policymakers and technologists in their efforts to control technical debt growth.

Technical debt is not just a technical problem

The objective of technical debt management policy is effective, sustainable control of technical debt. In an enterprise that has achieved this objective, technical debt serves as a strategic tool. That tool assists in attaining and maintaining market leadership. In such an organization, technical debt does exist, and some legacy technical debt might remain in place. But the enterprise manages technical debt growth strategically, if growth occurs at all. The enterprise addresses any technical debt that carries significant MICs. And it pays special attention to technical debt that compromises productivity and enterprise agility. In short, the enterprise addresses technical debt not only as a technological issue, but also as a component of business strategy.

This stance is at odds with the historical position most enterprises have adopted vis-à-vis technical debt. In the historic view, technical debt is a technical problem, if it is recognized at all. Most enterprises relegate technical debt management to the technologists. Frequently, then, technologists view as interlopers any policymakers who enter the discussion about technical debt. Any such policymakers usually arrive late to the discussion. Technologists often view them as less-than-knowledgeable invaders attempting to seize control of a piece of the technologists’ domain. In this way tensions can arise between policymakers and technologists. Such tensions complicate the problem of managing technical debt.

Sources of tension

One possible source of this tension is apparent in a study of the literature of technical debt, which is evolving so rapidly that it has itself become a focus for research. Li et al. [Li 2015] have produced a review of the software engineering technical debt literature. From this review we can extract insights useful to policymakers. They studied only the literature relating to technical debt in software engineering. But their conclusions are, at least in part, applicable to any field in which the components of the finished product are executed within software tools before being committed to operational form. This covers a wide array of knowledge-intensive endeavors, including mechanical system design, electronic design, framing of legislation, process design, architecture, and even book authorship.

Last words

In this thread, I explore the sources of the tension between the modern reality of technical debt as an enterprise issue, and the historical situation of technical debt as a technological issue. This can serve as a guide for policymakers in reframing technical debt. The archaic view of technical debt is as a technological issue dependent for resolution on enterprise resources. The modern view is as an enterprise issue dependent for resolution on technological resources.

Other posts in this thread

References

[Adobe Blogs 2014] Adobe Blogs. “What is Technical Debt?,” Adobe Blogs, September 8, 2014.

Available: here; Retrieved February 26, 2017.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Lowy 2004] Alex Lowy and Phil Hood. The Power of the 2x2 Matrix: Using 2x2 Thinking to Solve Business Problems and Make Better Decisions. Jossey-Bass, 2004.

Order from Amazon

Cited in:

[McConnell-slides 2013] Steve McConnell. “Managing Technical Debt”, ICSE 2013.

Available: here; Retrieved November 11, 2017

Cited in:
Show Buttons
Hide Buttons