Tag: MPrin

Posted on

Exogenous technical debt

Last updated on July 9th, 2021 at 04:58 pm

Exogenous technical debt is debt that arises from causes not directly related to the asset that bears the debt. Mastering understanding of exogenous technical debt is essential to controlling technical debt formation. Exogenous technical debt is particularly troublesome to those who work on the affected assets. They can’t control its formation, and they’re rarely responsible for creating it. But their internal customers and those who control resources often fail to understand this. Indeed, those who work on the affected assets often bear blame for the formation of exogenous technical debt even though they had no role in its formation, and could have done nothing to prevent its formation.

Exogenous technical debt and endogenous technical debt

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that don’t involve the asset directly.

Why we must track exogenous technical debt

Asbestos with muscovite.
Asbestos with muscovite. Asbestos is a family of minerals occurring naturally in fibrous form. The fibers are all known carcinogens. Until 1990, asbestos was a common ingredient of building materials, including insulation, plaster, and drywall joint compound. It’s now banned, but it’s present in existing homes and offices. The ban caused these structures to incur exogenous technical debt. Photo by Aramgutang courtesy Wikipedia.

Because so much technical debt arises indirectly, controlling its direct formation is insufficient to achieve control. To control technical debt formation, we must track which activities produce it. We must track both direct and indirect effects. Allocating technical debt retirement costs to the activities that brought that debt about is useful. It’s useful even if the allocation doesn’t affect budget authority for those activities. Knowledge about which past activities created technical debt, and how much, is helpful for long-term reduction in the rate of technical debt formation.

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries—or which will be carrying—the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing). So endogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that directly involve the asset.

More about endogenous technical debt in future posts. For now, let’s look more closely at exogenous technical debt, and its policy implications.

Examples of exogenous technical debt

In “Spontaneous generation,” I examined one scenario in which technical debt formation occurs spontaneously—that is, in the absence of engineering activity. Specifically, I noted how the emergence of the HTML5 standard led to the formation of technical debt in some (if not all) existing Web sites. This happened because those sites didn’t exploit capabilities that had become available in HTML5. Moreover, some sites needed rehabilitation to remove emulations of the capabilities of the new standard. Those emulations needed to be replaced with use of facilities in the HTML5 standard. All of these artifacts—including those that existed, and those that didn’t—comprised technical debt. This scenario thus led to the formation of exogenous technical debt.

In a second example, AMUFC, A Made-Up Fictitious Corporation, incurs technical debt when the vendor that supplies the operating system (OS) for AMUFC’s desktop computers announces the date of the end of extended support for the version of the OS in use at AMUFC. Because the end of extended support brings an end to security updates, AMUFC must retire that debt by migrating to the next version of that vendor’s OS before extended support actually ends.

In both examples, the forces that lead to formation of exogenous technical debt are external to the enterprise and the enterprise’s assets. But what makes technical debt exogenous is that the forces that led to its formation are unrelated the engineering work being performed on the asset. This restriction is loose enough to also include technical debt that arises from any change or activity external to the asset, but within the enterprise.

Exogenous technical debt arising from actions within the enterprise

Exogenous technical debt can arise from activities or decisions that take place entirely within the enterprise.

For example, consider the line of mobile devices of AMUFC (A Made-Up Fictitious Corporation). Until this past year, AMUFC has been developing ever more capable devices. These efforts extended its line of offerings at the high end—the more expensive and capable members of the line. But this past quarter, AMUFC developed a low-end member of the line.

As often happens, price constraints for the low-cost device led to innovations. Those innovations could produce considerable savings in manufacturing costs if used all across the line. In effect, the designs of the previously developed higher-end models have incurred exogenous technical debt. The debt is exogenous because the activity that led to debt formation wasn’t performed on the assets that carry the debt. The debt is real, even though the activity that led to debt formation occurred within the enterprise. This kind of exogenous technical debt is asset-exogenous. Exogenous technical debt of the kind that results from activity beyond the enterprise is enterprise-exogenous.

Exogeneity versus endogeneity

For asset-exogenous technical debt, ambiguity between endogeneity and exogeneity can arise. The example above regarding the line of mobile devices produced by AMUFC provides an illustration.

For convenience, call the team that developed one of the high-end devices Team High. Call the team that developed the low-end device Team Low. From the perspective of Team High, the technical debt due to the innovations discovered by Team Low is exogenous. But from the perspective of the VP Mobile Devices, that same technical debt might be regarded as endogenous. The debt can be endogenous at VP level because it’s possible to regard the entire product line as a single asset, and that might actually be the preferred perspective of VP Mobile Devices.

This ambiguity can lead to some nasty toxic conflict. Team High and VP Mobile Devices might attack each other as they try to defend themselves proactively against claims that they are incurring technical debt. Avoiding this kind of conflict requires educating everyone as to the origins of technical debt.

Exogeneity and legacy technical debt

The technical debt portfolio of a given asset can contain a mix a technical debt that arose from various past incidents. In assessing the condition of the asset, it’s useful to distinguish this existing debt from debt that’s incurred as a consequence of any current activity or decisions. Call this pre-existing technical debt legacy technical debt.

The legacy technical debt an asset carries is technical debt associated with the asset, and which existed in any form before undertaking work on that asset. For example, consider planning a project to renovate the hallways and common areas of a high-rise apartment building. Suppose workers discover beneath the existing carpeting a layer of asbestos floor tile. Then Management might decide to remove the tile. In this context, we can regard the floor tile as legacy technical debt. It isn’t directly related to the objectives of the current renovation. But removing it will enhance the safety of future renovations. It will also enable certification of the building as asbestos-free, increase the property value, and reduce the cost of eventual demolition. In this situation asbestos removal is retirement of legacy technical debt. Accounting for it as part of the common-area renovation would be misleading.

Exogeneity is relevant when allocating resources for legacy technical debt retirement efforts. If the debt in question is enterprise-exogenous, we can justifiably budget the effort from enterprise-level accounts. For other cases, other resources become relevant, depending on what actions created the debt. For example, suppose that the technical debt arose from a change in enterprise standards. Then we can justifiably allocate retirement costs to the standard-setting initiative. If the exogenous technical debt arose from innovations in other members of the asset’s product line, we can can justifiably allocate those debt retirement costs to the product line.

Policy insights

Understanding the properties of exogenous technical debt can be a foundation for policy innovations that enhance enterprise agility.

Culture transformation

Widespread understanding of the distinction between exogenous and endogenous technical debt is helpful in controlling interpersonal conflict. For example, it can reduce blaming behavior that targets the engineering teams responsible for developing and maintaining technological assets.

Understanding asset-exogenous technical debt helps non-engineers understand how their actions and decisions can lead to technical debt formation. The concept clarifies the import of their actions even when there is no apparent direct connection between those actions or decisions and the assets in question.

Resource allocation

Data about the technical debt creation effects of enterprise activities is helpful in allocating technical debt retirement costs. For example, suppose that we know all the implications of reorganization, including its impact on internal data about the enterprise itself. Then we can charge data-related activity to the reorganization instead of to general accounts of the Information Technology function. This helps the enterprise understand the true costs of reorganization.

Similarly, data about enterprise-exogenous technical debt helps planners understand how to deploy resources to gather external intelligence about trends that can affect internal assets. Such data is also useful for setting levels of support and participation in industrial standards organizations or in lobbying government officials.

Last words

Knowing the formation history of exogenous technical debt provides useful guidance for those charged with allocating the costs of retiring technical debt or preventing its formation.

Other posts in this thread

Posted on

MPrin for missing or incomplete capability

Last updated on July 7th, 2021 at 02:53 pm

In some instances, the metaphorical principal (MPrin) of a technical debt is a missing or incompletely implemented capability. For example, absence of a fully automated regression test suite can create difficulties for testing a complex system. Defects can slip through. That would result in reduced productivity and velocity. In this case, the projected cost of implementing, testing, and documenting the test suite, and training its users, would constitute the initial MPrin of the outstanding technical debt. This definition differs from some definitions, because it includes testing, documenting, and initial training. In general, from the enterprise perspective, when identifying the MPrin associated with missing or incompletely implemented capabilities, we must include all artifacts necessary to eliminate reductions in productivity and velocity.

But even if we include these items in the conventional definition, MPrin at retirement can exceed the savings at the time we incurred the debt. For example, in the interval between origination and retirement the assets involved can change. Moreover, if retiring the debt causes a revenue stream interruption, the MPrin, which includes the lost revenue, can be significantly larger than the initial MPrin.

Unique problems of incompletely implemented capability

A concrete building under construction
A concrete building under construction. Concrete takes about a month to cure and reach full strength. If we waited for full curing before pouring the floor above, multistory concrete construction would be slow and expensive. So we start on the next floor after only a few days. Because the floors can’t support themselves for about a week, we shore them using lower floors. The shoring constitutes a technical debt resulting from the “incomplete implementation” (partial cure) of each floor. We retire that debt by removing the shoring as we go. More about shoring and re-shoring
Technical debt associated with incompletely implemented capability presents unique problems. We can retire it in three distinct ways. First, we can complete the implementation. The MPrin associated with this approach can grow beyond the initial cost of completion, for the usual reasons. Second, we can cancel the capability. If we do, retiring the debt completely would require removal of all artifacts that we no longer need. Finally, we can choose a middle path. In the middle path we adopt some parts that have been completed. But we reject other parts, and we add whatever is necessary to create a limited version of what we originally planned.

Special challenges of non-physical assets

Invisibility is an important attribute of non-physical assets such as software, procedures, legislation, regulations, and so on. Technical debt associated with incomplete implementation is difficult to manage in such assets. For example, the image above shows several levels of a concrete building under construction. The vertical members between the levels are part of a shoring system that supports the levels of the building. Shoring is necessary until the concrete floors cure well enough to support themselves. Shoring constitutes a kind of technical debt that must be “retired” before the building is complete. The teams constructing the building could never forget to remove the shoring because it obstructs installation of the windows and walls.

But things are very different with non-physical assets. It’s easy to forget to remove intermediate artifacts, or elements that were part of attempts that didn’t work out. Many non-physical assets are perfectly functional carrying that kind of technical debt. That debt becomes evident with time, as the asset becomes increasingly difficult to maintain, extend, or defend.

It’s this property of non-physical assets that makes technical debt management so much more difficult than it is with physical assets. Not more important, just more difficult.

Related posts

  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • Useful projections of MPrin might not be attainable
    Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

MPrin when standards or regulations change

Last updated on July 7th, 2021 at 11:15 am

The MPrin of technical debt that forms as a result of a change in standards or regulations is the cost of bringing affected assets into compliance. It matters not whether the standards in question are internal to your organization or external. The conventional definition of the MPrin for this kind of technical debt includes only the cost of aligning to the new standards or regs, the assets directly affected. But the conventional definition is incomplete. If we account for all work properly, the MPrin should also include ripple effects. Ripple effects are the changes in other assets that we must perform to maintain compatibility with the assets affected directly by the change in standards or regs.

The phrase standards or regs is beginning to bother even me. I’ll switch to standards when I mean standards or regulations (or regs) except when I say so explicitly.

Cost drivers of changes in standards

A view of the left side guard on a truck operated by the City of Cambridge, Massachusetts
A view of the left side guard on a truck operated by the City of Cambridge, Massachusetts. Side guards prevent vulnerable road users (pedestrians, bicyclists, and motorcyclists) from being swept under trucks and crushed (and often killed) by the truck’s rear wheels. Cambridge has a pilot program affecting city-operated trucks, but Boston is requiring all contractors to install side guards. This change in regulations creates a technical debt for all truck operators whose vehicles lack side guards. [Volpe 2017] City of Cambridge photo courtesy U.S. Department of Transportation.

Aligning existing assets to new standards can have expensive consequences. We must include all costs in the calculation of MPrin. Unfortunately, some costs are often overlooked or accounted for in other ways. For example, testing might require a service interruption or product availability delays or interruptions. And that could entail a revenue stream delay or interruption. That lost revenue is certainly a consequence of the debt retirement effort.

Deferring retirement of this class of technical debt can expose the enterprise to the risk of MPrin growth in two ways. First, when we defer debt retirement, the number of instances of violations of the new standards can increase as we develop new assets in compliance with the obsolete standards. Second is the potential for increases in the number of ripple effect instances when we defer debt retirement. These instances arise from increases in the number of artifacts that require updating. The issue isn’t that they aren’t compliant with the new standards. Rather, it is that we must align them with the components we modify to comply with the new standards. In this way, MPrin at debt retirement time can greatly exceed the savings we realized when we first incurred the debt.

Last words

However, as with most technical debts, deferring retirement of this class of debt can sometimes be wise. For example, if the assets that bear the debt are about to be retired, the debt they carry vanishes when we retire those assets.

References

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • Useful projections of MPrin might not be attainable
    Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

MPrin in a platform component upgrade

Last updated on July 12th, 2021 at 03:05 am

An aisle in the stacks of a library, where marking the books with RFID tags would be a platform component upgrade
An aisle in the stacks of a library. Some libraries are upgrading their book tagging systems from barcodes to RFID tags. This is essentially a platform component upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day [Boss 2011]. It’s a big job.

The MPrin of technical debt that forms as a consequence of a platform component upgrade depends on how we incur the debt. If we incur the debt by installing the upgrade, and then perform only some of the work made necessary by the upgrade, then the MPrin is the total cost of performing the deferred work. If we incur the debt by deferring the upgrade, then the conventional definition of the MPrin has two components. The first is the cost of the upgrade, and the second is the cost of any work made necessary by the upgrade, but not performed.

Hold on—it’s not so simple

In this latter instance, MPrin can increase over time. Increases can occur if the following three-step sequence happens for either maintenance or enhancements. In step 1, we perform work in the environment of the obsolete platform component, but after deferring the upgrade. Step 2 is performance of the upgrade. In step 3, we must repeat the work we performed in step 1 because the step 1 version isn’t compatible with the upgrade. This situation can be even worse if we discover the need for step 3 as a result of operational failure after the upgrade. In that case, maintainers must investigate the failure first. And the failure might cause database contamination, which would also need remedying. These additional costs are actually part of the debt retirement effort for the debt incurred by deferring the upgrade, but we usually account for them—mistakenly—as routine operational expense.

Last words

Advance knowledge of what can go wrong is always a nice-to-have. Most of us try to acquire this knowledge before or as we plan our projects. And most of us can do better. Before you consider a plan complete, ask yourself if anyone else might have already tried something similar. If you can guess who that might be, contact him or her to find out how it went. No point repeating someone else’s mistakes.

References

[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.

Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017

Cited in:

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • Useful projections of MPrin might not be attainable
    Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

MPrin in a development project

Last updated on July 7th, 2021 at 11:12 am

The "basket bridge" of the Los Angeles Metro system
The “basket bridge” of the Los Angeles Metro system. Constructed by the Metro Gold Line Foothill Extension Construction Authority, and opened in 2012, it carries the Los Angeles to Pasadena Metro Gold Line light rail across the eastbound lanes of the I-210 Freeway. The rail line provides transport service to a ridership that had been driving or using bus service. Although the bus and rail aren’t exact duplicates of each other, there is enough overlap that bus ridership dropped significantly after the rail line opened. That drop created a technical debt in the bus system. Retiring that debt requires reducing, rescheduling and re-routing bus service [Broverman 2017].

We usually regard the MPrin of new technical debt associated with a development project as the difference between the cost of implementing it sustainably and the cost of simply making it work. The effort saved by choosing the latter over the former is the initial MPrin of the technical debt.

For example, consider an enhancement project for an existing asset. After we achieve an operational capability, we might notice that we’ve duplicated some of the asset’s pre-existing functionality. The responsible debt-free approach has three stages. First, we eliminate the new and unnecessary duplicated capability. Next, we modify the asset to use the previously existing capability. Finally, we re-test the asset. The approach that generates new debt involves leaving the duplication in place.

Other generators of technical debt

In a closely related example, we might recognize that the duplicated functionality in the newly developed portion of the asset is superior to the pre-existing form elsewhere in the asset. We’d like to remove the pre-existing form and replace instances of that form with instances of the newly developed functionality. But that work is clearly outside the scope of the new development, and it must await budgetary authority before it can be undertaken. Consequently, it becomes technical debt for the larger asset.

As time passes, and the enterprise undertakes other development projects, the implementations of previous projects can accumulate additional technical debt. The more obvious mechanisms by which this occurs include defect discovery, customer requests for enhancements, the need to enhance cyberdefenses in response to new threats, or changes in law or regulation. Less obvious, but no less significant, are changes in markets, customer needs, and underlying technologies. All can contribute to technical debt formation

A final example

An example of a less obvious process might be insights gained in marketing one product (call it P1). Suppose those insights reveal an opportunity to introduce other related products—P2, P3, and P4—to form a suite. The latter products could employ some assets developed for P1, if the latter products had been constructed slightly differently. The changes required in P1 therefore constitute technical debt, because we would have been able to develop P2, P3, and P4 much more rapidly if we had recognized the opportunity earlier. The P1 changes then become technical debt. And if we pursue P2, P3, or P4 without first retiring that debt, the debt probably expands, because the subsequent products manifest it.

New product (or service) developments often generate these situations.

References

[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.

Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017

Cited in:

[Broverman 2017] Neal Broverman. “The Success of the Gold and Expo Lines Has Taken a Toll on Bus Ridership,” Los Angeles Magazine, March 30, 2017.

Available: here; Retrieved: November 21, 2017

Cited in:

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • Useful projections of MPrin might not be attainable
    Expect the unexpected with technical debt retirement efforts, because they can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

Examples of MPrin in four scenarios

Last updated on June 15th, 2021 at 01:57 pm

Some examples might help to clarify the differences between the principal of financial debts and the MPrin of a technical debt. The examples to come in the next four posts illustrate the unique properties of MPrins of technical debts.

Technical debt can arise as a result of:
  • Changes internally within the enterprise
  • External environmental changes that directly affect existing assets
  • Changes in the competitive environment
  • Insights and changes in perception that lead to changes in objectives
  • Existing technical debt
  • Deferring decisions about almost anything

By contrast, we incur financial debt only as a result of decisions to incur financial debt.

The examples below illustrate some of these phenomena. For each one, the full post contains a more complete explanation.

Development projects

This example illustrates how technical debt can develop as a result of unanticipated insight regarding a marketing opportunity for a new product line. It shows how technical debt can arise independent of any decision made within the enterprise, and without any changes to assets of any kind. More: “MPrin in a development project”

Platform component upgrades

We’ve already provided an example of technical debt arising from a platform upgrade. In this example, we show how deferring a platform upgrade creates new complications not present in the previous example, by illustrating how total MPrin can increase after the debt first forms. More: “MPrin in a platform component upgrade”

Standards or regulatory changes

Changes in standards or regulations, whether internal, industry-wide, or governmental, can create technical debt. In some cases, the enterprise might not even be aware of the new debt. More: “MPrin when standards or regulations change”

Missing or incompletely implemented capability

When a capability is incompletely implemented, it’s clear that the part left undone constitutes technical debt. What is less clear is what happens when the capability implementation is halted or rescinded. Trying to avoid new technical debt can actually be the cause of new technical debt, albeit of a different kind. More: “MPrin for missing or incomplete capability”

Whether or not any similar scenarios have happened in your organization, these discussions are helpful for gaining insight into what kinds of technical debt policies can assist your organization in managing its technical debt. Let me know if you have experience with situations in which problems can be traced, even if only in part, to treating technical debt as if it were financial debt.

Posted on

Useful projections of MPrin might not be attainable

Last updated on July 8th, 2021 at 11:50 am

SummaryExpect the unexpected with technical debt retirement efforts. Technical debt retirement efforts can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Although these conflicts are technical in nature, resolving them can involve business priorities at any level. Planners must be aware of these potential conflicts, and coordinate with their leaders. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.

For planning purposes, it’s necessary from time to time to make projections of debt retirement costs (MPrin) for a given class of technical debt. The need arises when planning debt retirement, or when preparing debt retirement options for determining resource allocations. Although retiring some kinds of technical debt is straightforward, other kinds of debt can become intertwined with each other. Retiring still others might appear to be easy, but actual retirement efforts expose unanticipated entanglements. Moreover, debt retirement efforts can sometimes interact with other debt retirement efforts, operations, maintenance, cyberdefense, and new development in both expected and unexpected ways. For these reasons, making estimates of the MPrin with enough precision to be useful can be notoriously difficult.

Money is fungible; people are not

A tangle of cordage
A tangle of cordage on board ship. Different kinds of technical debt can become entangled with each other. Untangling them can affect various other engineering efforts. Preparing an asset for a debt retirement effort by doing some preliminary untangling might be wise before trying to estimate the MPrin of any affected class of technical debt.
These considerations rarely arise when planning retirement of financial debts, because money is fungible. We might indeed have other uses for financial resources. But every unit of cash is equivalent to every other. That freedom isn’t necessarily available when planning resource allocations for technical debt retirement.

For example, not every engineer is equally capable of addressing every problem. Some people are particularly capable for certain kinds of work, and not very capable for other kinds. The problem of scheduling specialists is notorious for generating bottlenecks. And split assignments create even more trouble. People aren’t fungible.

Last words

Planning retirement of a particular set of technical debt classes can be complicated. Such planning requires knowledge of any efforts with which that retirement effort might interact. That information might not be available or might not be known. In general, preliminary work to decouple these activities—often called ”refactoring”—can

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

Debt contagion: how technical debt can create more technical debt

Last updated on July 7th, 2021 at 11:08 am

Decisions to defer technical debt retirement must take into account phenomena that if left unattended can lead to three undesirable outcomes. First, deferring technical debt retirement can increase the volume of the deferred class of technical debt. Second, deferring technical debt retirement can increase the volume of other existing classes of technical debt. And third, deferring technical debt retirement can generate new classes of technical debt.

An example of debt contagion

Suppose Working at a computerwe have a fleet of desktop computers, running a mix of operating systems. A few of these systems are running Windows 8 and the rest are running Windows 10. We’d like to upgrade the Windows 8 machines to Windows 10, but we can’t, because some of their users need access to a (fictional) scriptable application called CRUSH. CRUSH isn’t available for Windows 10. CRUSH for Windows 10 is promised “shortly.”

Instead of asking our CRUSH users to find an alternative to CRUSH, we defer the Windows 8 upgrade. We’re hoping that CRUSH for Windows 10 will soon arrive. Meanwhile, other Windows 8 users are happy to continue using Windows 8. Some of them have acquired—and have grown fond of using—another similar (fictional) scriptable package called REMOTE. REMOTE is also unavailable for Windows 10. Worse, the CRUSH user community is continuing to grow.

Thus, by deferring the Windows 8 upgrade, we’ve made space for additional problems preventing the upgrade to Windows 10. The technical debt associated with the Windows 8 upgrade now includes Windows 8 itself, and all the scripts, documents, and knowledge that are accumulating for both CRUSH and REMOTE.

Lessons learned from this example

The lesson here isn’t to ban scriptable applications. Nor is the lesson to compel desktop users to adhere to an enterprise standard. Both options create numerous problems. The main lesson this example provdes is that deferring debt retirement can enable formation of new instances of existing technical debt. In this example, the new debt includes the growth of the CRUSH user community with the assets they continue to develop, New debt also includes unrelated debt connected with the introduction of REMOTE. Thus, if we defer retirement of one class of technical debt, we must consider all costs of such deferment. Those additional costs can include expansion of the total volume of technical debt, and all its consequences, as expressed as metaphorical interest charges and MPrin.

Some of the new technical debt that forms when we leave existing debt in place is closely related to the existing debt. For example, once we’ve implemented some part of an asset in a way that we now acknowledge contains a form of technical debt, we tend to apply that same approach when we undertake extensions or enhancements, rather than using what everyone might acknowledge is a superior approach.

Introduction to debt contagion

Martini and Bosch have identified a phenomenon they call debt contagion [Martini 2015]. Because of debt contagion, creating new system elements in forms compatible with elements already identified as debt causes debt propagation. This practice helps us maintain some degree of uniformity in the asset, recognizing that in doing so we’re increasing the MPrin of a given class of technical debt. These future expansions of MPrin can be difficult to predict at the time we first incur the debt, or at any time.

However, some forms of technical debt are far less discriminating with respect to the kinds of technical debt they spawn. Debt with this property tends to be associated with the processes used to develop or maintain technological assets. In “A policymaker’s definition of technical debt,” we cite Pugh’s example of acceptance test debt as a form of technical debt [Pugh 2010].

Acceptance test debt can reduce the ability of the organization to retire other forms of technical debt. Absent automated acceptance tests, testing system components from which technical debt has recently been removed is less efficient and reliable than it would be if automated acceptance tests were available. This depressed efficiency and reliability retards debt retirement activity. It might even prevent the organization from attempting debt retirement in some circumstances. In a future post, I’ll describe how a deficient regime of reviews and inspections can also lead to incurring new technical debt, or to elevated levels of legacy technical debt.

Last words

Our final example illustrates how hardware and software interfaces can propagate technical debt. This is ironic, because interfaces were conceived to insulate one portion of an asset from others. Given a system S composed of modules, suppose that module M of S provides services other modules of S. M does contain technical debt, of a form whose retirement would simplify M’s interface. Because that change would require changes to the modules that use M’s services, we decide to defer retiring M’s debt. Meanwhile, other projects are introducing new modules into S, and, of course, they must use M’s existing interface. The MPrin of the technical debt associated with M’s interface thus expands.

Unless we provide an alternate version of M (call it M’) or an alternate interface to M, MPrin expansion can repeat whenever new modules appear in S. But even if we do provide alternatives, engineers must consciously adopt them. Although some will, others might not. Some are under severe schedule pressure. Some cannot or will not learn the new approaches. And some receive direct orders not to use the new approaches. The MPrin associated with M can thus continue to expand, albeit perhaps at a reduced rate.

Technical debt, left in place, can grow and spawn new forms of technical debt. Make technical debt retirement a priority.

References

[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.

Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017

Cited in:

[Broverman 2017] Neal Broverman. “The Success of the Gold and Expo Lines Has Taken a Toll on Bus Ridership,” Los Angeles Magazine, March 30, 2017.

Available: here; Retrieved: November 21, 2017

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

Glossary and Terminology

Last updated on July 16th, 2021 at 11:23 am

Even though technical debt has been with us for a very long time—probably since the time we began inventing technologies—the study of technical debt is relatively new. Ward Cunningham coined the term technical debt in 1992, and its meaning has evolved since then. Because universally accepted definitions for the term and associated concepts have not yet emerged, it seems necessary to have a page on this site that collects definitions.

Asset-exogenous technical debt

Exogenous technical debt is asset-exogenous when it’s brought about by an activity external to an asset, but internal to the enterprise. For example, a change in standards or regulations by some body within the enterprise can cause an asset to incur an asset-exogenous technical debt.

ATD

See Auxiliary technical debt.

Auxiliary technical debt

In the context of a Technical Debt Retirement Project (DRP) that has as an objective retiring from a specified set of assets a particular kind or particular kinds of technical debt, the ATD is the collection of instances of any other kinds of technical debt other than the kind that the DRP is trying to retire. More: “Auxiliary technical debt: Rules of engagement”

Class of technical debt

On occasion, we speak of classes of technical debt and instances of that class. This can be confusing, because the words class and instance have particular meanings in software engineering. That’s not the sense in which we use the terms here. In this blog, a class of technical debt is just a collection of instances of the same kind of debt. For example, consider the “ghost ramp” described in “Technical debt in the highway system.” It belongs to the class of ghost ramps. If we were maintaining the highway system of Massachusetts, it might be convenient to consider the class of ghost ramp technical debt if we want to let a contract to demolish all ghost ramps. Each ghost ramp would then be an instance of that class.

Cognitive bias

A cognitive bias is the human tendency to make systematic errors based not on evidence, but on factors related to the thought process. Psychologists have identified and demonstrated hundreds of cognitive biases, including several that could plausibly explain failures in priority setting for technical debt retirement projects.

Confirmation bias

Confirmation bias is a cognitive bias. It’s the human tendency to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. For example, the homogeneity of cable news channel audiences, and the alignment between preconceptions of the audience and the slant of the newscast for that channel, are results of confirmation bias. More: “Confirmation bias and technical debt”

Debt contagion

If a class of technical debt is allowed to remain outstanding, its volume can increase as a consequence of seemingly unrelated actions or decisions. Moreover, its existence can cause increases in the volume of other existing classes of technical debt, and its existence can lead to the formation of new classes of technical debt. This process is called debt contagion. More: “Debt contagion: how technical debt can create more technical debt”

DRP

A (technical) Debt-Bearing Asset

DRP

In this blog, I use the term DRP to mean a (technical) Debt Retirement Project. A DRP is a project that has as an objective retiring from a specified set of assets a particular kind of technical debt (or particular kinds of technical debt). Many projects have objectives of debt retirement, at some point or other. But DRPs differ from most, in that debt retirement is their primary objective—indeed, it might be their sole objective. More: “Nine indicators of wickedness”

Echo release

An echo release of an asset is a release version whose primary purpose is technical debt retirement. Typically, it’s created immediately following a release version that has created some incremental technical debt, hence the term “echo release.” The echo release is then executed to retire that incremental technical debt, and not to repair defects or add capability. More: “Accounting for technical debt”

Endogenous technical debt

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing).  So endogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that directly involve the asset. More: “Exogenous technical debt”

Enterprise-exogenous technical debt

Exogenous technical debt is enterprise-exogenous when it’s brought about by an activity external to the enterprise. For example, a change in standards or regulations by some body outside the enterprise can cause an asset to incur an enterprise-exogenous technical debt.

Exogenous technical debt

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activity or decisions that don’t involve the asset directly. More: “Exogenous technical debt”

Ill-structured problem

An ill-structured problem is a problem that isn’t a well-structured problem [Simon 1973]. An example of an ill-structured problem is finding a definition for ill-structured problems. Another: designing a computer programming language. Still another, even more to the point: deciding when to retire a particular class of technical debt. NDM is more likely to be successful with ill-structured problems than is RDM.

Incremental technical debt

Incremental technical debt is either newly incurred exogenous technical debt, or technical debt that’s incurred in the course of work currently underway or just recently completed. For example, in an apartment building hallway renovation project, workmen did insert expansion joints in the sheetrock they replaced, but on the first three floors they completed, the joints were too widely separated. The remaining 22 floors were done correctly. Nine additional joints on each of the incorrect floors must be inserted eventually. The missing joints, which constitute incremental technical debt, will be inserted after the job is completed. More: “Controlling incremental technical debt”

Instance of technical debt

See “Class of technical debt”

Intertemporal choice

Intertemporal choice is the process by which people make decisions between options that occur at different points in time. Decisions involving intertemporal choice can be exceedingly complex, especially when options have effects that vary with time. For example, confronted with advice from technical experts regarding the urgent need to address the burden of enterprise technical debt, decision makers must consider an unpleasant possibility. To make resources available to retire the technical debt, it might be necessary to temporarily defer investment in some new products or enhancing some existing products. And if they make the recommended investments in technical debt retirement, customers won’t benefit in any visible way. So the choice reduces to one between new products and enhancements relatively sooner, versus retiring technical debt and only later attending to new products and enhancements of existing products. This dilemma is an example of what behavioral economists call intertemporal choice [Loewenstein 1992].

Key Performance Indicator (KPI)

A Key Performance Indicator (KPI) is a metric that provides meaningful insight that’s used to guide business decisions. All KPIs are metrics; not all metrics are KPIs. More: “Metrics for technical debt management: the basics”

Legacy technical debt

Legacy technical debt is technical debt associated with an asset, and which exists in any form prior to undertaking work on that asset. For example, in planning a project to renovate the hallways and common areas of a high-rise apartment building, Management discovers that beneath the existing carpeting is a layer of floor tile containing asbestos. Management has decided to remove the tile. In this context, the floor tile can be viewed as legacy technical debt. It isn’t directly related to the objectives of the current renovation, but removing it will enhance the safety of future renovations, enable certification of the building as asbestos-free, and reduce the cost of eventual demolition. More: “Exogenous technical debt”

Localizable technical debt

Localizable technical debt is technical debt that manifests itself as discrete chunks. Each instance is self-contained, and we can “point” to it as an instance of the debt in question. For example, if the organization regards Windows 10 as the current operating system for personal computers, and early versions of Windows as technical debt, the each computer that runs and earlier version of Windows is an instance of that technical debt. Each instance is discrete and localized. More: “Retiring localizable technical debt”

Measure

A measure is the result of determining the value of a quantifier. For example, we might use the quantifier’s definition to determine a measure of how much human effort has been expended on an asset in the past fiscal quarter. More: “Metrics for technical debt management: the basics”

Metric

A metric is an arithmetic formula expressed in terms of constants and a set of measures. One of the simpler metrics consists of a single ratio of two measures. For example, the metric that captures the average cost of acquiring a new customer in the previous fiscal quarter is the ratio of two measures, namely, the investment made in acquiring new customers, and the number of new customers acquired. More: “Metrics for technical debt management: the basics”

MICs, or metaphorical interest charges

MICs are the metaphorical interest charges associated with a technical debt. They aren’t interest charges in the financial sense; rather, the MICs of a technical debt represent the total of reduced revenue, lost opportunities, and increased costs of all kinds borne by the enterprise as a consequence of carrying that technical debt. Because the properties of MICs are very different from the properties of financial interest charges, we use the term MICs to avoid confusion with the term interest from the realm of finance. More: “How financial interest charges differ from interest charges on technical debt”

MPrin, or metaphorical principal

The MPrin of a technical debt at a give time T is the total cost of retiring that debt at time T. The total cost includes all cost factors: labor, equipment, service interruptions, revenue delays, anything. It even includes the ongoing costs of repairing defects introduced in the debt retirement process. More: “The metaphorical principal of a technical debt”

Naturalistic decision-making

Naturalistic decision-making (NDM) entails situation assessment and evaluation of a single option to select a satisfactory option. [Zannier 2007] Features that define naturalistic decision-making are “time pressure, high stakes, experienced decision makers, inadequate information (information that is missing, ambiguous, or erroneous), ill-defined goals, poorly defined procedures, cue learning, context (e.g., higher-level goals, stress), dynamic conditions, and team coordination.”  [Klein 2017]

Nonstrategic technical debt

Nonstrategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce nonstrategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. And at times, it appears in the asset as a result of external events beyond the boundaries of the enterprise. More: “Nontechnical precursors of nonstrategic technical debt”

The planning fallacy

The planning fallacy is a cognitive bias that causes planners to underestimate costs and schedules, and over-promise benefits. They do this, in part, because they pay too little heed to past experience on similar efforts. They also rely too much on what they believe will happen on the effort they’re planning. First identified in a 1977 report by Daniel Kahneman and Amos Tversky [Kahneman 1977] [Kahneman 1979]. More: “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma”

Policy

Organizational policy is the framework of principles that guide policymakers, decision makers, and everyone in the organization as they carry out their responsibilities. Policy might be written or not, but written policy is more likely to consistently adhered to. Interestingly, the body of organizational policy is itself subject to accumulating technical debt. More: “What is policy?”

Policymaker

As I use the term in this blog, a policymaker is someone who is responsible for developing, revising, or approving organizational policies that affect technical debt management. More: “Who are the policymakers?”

Quantifier

A quantifier is a specification for a measurement process designed to yield a numeric representation of some attribute of an asset or process. Quantifiers are used to obtain the values called measures, which in turn are used in computing metrics. More: “Metrics for technical debt management: the basics”

Rational decision-making

Rational decision-making (RDM) is an approach to making a choice of an option from among a set of options by selecting the option that is optimal with respect to a set of quantitative criteria. [Zannier 2007] Rational choice strategies generally follow this framework: (1) Identify a set of options; (2) Identify criteria for evaluating them; (3) Assign weight to each evaluation criterion; (4) Rate the options relative to the criteria; (5) Choose the option with the highest score. Many different frameworks for implementing this strategy are available, some specialized to specific subject domains [Thokala 2016].

Refactoring

Fowler defines refactoring as “the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure” [Fowler 1999]. Although refactoring is a term specific to software development processes, the concept applies to all technological development. For example, an automobile manufacturer’s decision to alter the design of one of their model vehicles to reduce manufacturing costs can be viewed as a form of refactoring. Refactoring is a practice essential to effective technical debt management. More: “Refactoring for policymakers”

Regression testing

Regression testing is a testing regimen that ensures that a previously developed and tested system still performs the same way after it has been altered or when it’s used in a new context. Regression testing is essential when we alter a system by retiring some of its technical debt.

The reification error

The reification error (also called the reification fallacy, concretism, or the fallacy of misplaced concreteness) is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing. Reification is useful in some applications, such as object-oriented programming and design. But when we use it in the domain of logical reasoning, troubles can arise. Specifically, we can encounter trouble when we think of “measuring” technical debt. Strictly speaking, we cannot measure technical debt. We can estimate the cost of retiring it, but estimates are only approximations. And in the case of technical debt, the approximations are usually fairly rough. To regard these estimates as measurements is to risk reifying them. Then when the actual cost of a debt retirement project is dramatically larger than the estimate, the consequences for enterprise budgets can be severe. We must always regard “measurements” of technical debt as estimates—estimates that are so prone to error that we must plan for error.  The reification error is the dual of the resilience error. More: “Metrics for technical debt management: the basics.”

The resilience error

If the reification error is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing, the resilience error is an error of reasoning in which we treat an abstraction as if it were more flexible, resilient, and adaptable than it actually is. When we commit the resilience error with respect to an abstraction, we’re adopting a belief, usually without justification, and possibly outside our awareness. That belief is that the familiar properties of the abstraction can survive changes in the abstraction.

Specifically, if we make changes in the abstraction, we can be certain that the familiar properties of the abstraction we modified will apply in modified form. We hold this belief without fully investigating the consequences of the changes we made in the abstraction. Or we assume incorrectly that the abstraction will accommodate any changes we make to its environment. The resilience error is the dual of the reification error. We are at risk of making the resilience error when we refactor assets to reduce their burden of technical debt. More: “The resilience error and technical debt.”

Secured technical debt

A secured technical debt, like a secured financial debt, is one for which the enterprise has reserved the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. They might include particular staff, equipment, test beds, downtime, and financial resources. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes. More: “Using SMART goals for technical debt reduction”

Source and target components of a metaphor

In a metaphor of the form “A is B,” the source is the element whose attributes are being attributed to the target. For example, in “my son’s room is a war zone,” the source is the war zone, and the target is my son’s room.  More: “The structure of metaphors”

Super wicked problem

A subset of wicked problems can be viewed as super wicked [Levin 2012]. Levin, et al. list the following four properties of super wicked problems: (1) Time is running out; (2) Those who cause the problem also seek to provide a solution or influence the solution; (3) The central authority needed to address the problem is weak, nonexistent, or chooses not to act effectively; (4) Policy responses discount the future irrationally. I’ve come to believe that some technical debt retirement project design can be a super wicked problem. More: “Retiring technical debt can be a super wicked problem”

Tame problems

A problem is a tame problem if it fails to meet at least one of the ten criteria established by Rittel and Webber [Rittel 1973] for wicked problems. Four of the criteria: it’s an ill-structured problem; it’s incompletely defined or internally contradictory; its solutions aren’t true-or-false, but good-or-bad; and there’s no way to exhaustively describe all solutions. I’m convinced that technical debt retirement project design can be a wicked problem. A tame problem is one that fails to meet at least one of the ten criteria for wickedness. Tame problems and wicked problems thus lie at opposite ends of a “Tame/Wicked” spectrum. Technical debt retirement project design problems fall somewhere on this spectrum. More: “Degrees of wickedness.”

Taylorism

Taylorism is an approach to management developed by Frederick Winslow Taylor in the early part of the twentieth century [Taylor 1913] [Kanigel 1997]. He proposed three principles of scientific management that could produce maximum efficiency. First, managers should select the person performing the work based on science. Second, organizations should decompose tasks based on scientific principles. Third, they must separate planning from execution. These principles are the basis of what became known in software engineering as the waterfall lifecycle. The approach works well for well-structured problems, but does not work well at all for ill-structured problems. Moreover, it depends for success on repeating solutions to problems already solved, which is why it proved so valuable in early manufacturing. The unsuitability of Taylorism for ill-structured problems is an important part of the basis for the Agile approach to problem solving.

TDIQ

In the context of a Technical Debt Retirement Project (DRP), we can define the Technical Debt In Question (TDIQ). If the DRP has as an objective retiring a kind of technical debt, that kind of technical debt is the TDIQ. More: “Retiring technical debt from irreplaceable assets”

Technical debt

Technical debt is any technological element that hampers development, maintenance, or enhancement efforts, through its existence or through its absence. It contributes to lower productivity or to a higher probability of defects. Or it can depress velocity in many other ways. That’s why we would like to revise, repair, replace, rewrite, create, or re-engineer it for sound engineering reasons. It can be found in—or it can be missing from—software, hardware, processes, procedures, practices, or any associated artifact, acquired by the enterprise or created within it. More: “A policymaker’s definition of technical debt”

Technological communication risk

Technological communication risk is the risk that, for whatever reason, knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. More: “Technological communication risk”

Temporal discounting

Temporal discounting is the human tendency to give greater value to a reward (or as economists would say, to assign greater utility to a good) the earlier it arrives. An analogous process affects perceptions of inconvenience or disutility: people assign more negative values to penalties and inconveniences the sooner they arrive. If the discount rate is constant, the discounting is termed exponential discounting or rational discounting. But other forms are possible. Hyperbolic discounting is one form of discounting at a rate that is higher for near-term arrivals than for distant-term arrivals [Laibson 1997]. Humans have been observed experimentally to favor a form of temporal discounting that is well modeled as hyperbolic discounting.

Terrifying opportunity

A terrifying opportunity arises when the organization rejects (or fails to recognize) a market opportunity because exploiting it would involve modifying an existing asset or product offering that harbors a heavy load of technical debt. The debt causes decision makers to assess that the probability of success is so low that the opportunity seems terrifying, and they therefore reject the opportunity. More: “MICs on technical debt can be difficult to measure”

Well-structured problem

As defined by Simon [Simon 1973], a well-structured problem is a problem that has some or all of six characteristics. The first is the existence of a definite criterion for testing any proposed solution, and a mechanizable process for applying that criterion. Second, there is at least one problem space in which we can represent the initial problem state, the goal state, and all states that can be reached or considered while solving the problem. There are four more criteria, but these are the biggies. An example of a well-structured problem is the game of chess. RDM is useful for attacking well-structured problems.

Wicked problem

A problem is a wicked problem if it meets the ten criteria established by Rittel and Webber [Rittel 1973]. Four of the criteria: it’s an ill-structured problem; it’s incompletely defined or internally contradictory; its solutions aren’t true-or-false, but good-or-bad; and there’s no way to exhaustively describe all solutions. I’m convinced that technical debt retirement project design can be a wicked problem. More: “Self-sustaining technical knowledge deficits during contract negotiations.”

References

[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.

Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017

Cited in:

[Broverman 2017] Neal Broverman. “The Success of the Gold and Expo Lines Has Taken a Toll on Bus Ridership,” Los Angeles Magazine, March 30, 2017.

Available: here; Retrieved: November 21, 2017

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:
Posted on

Policy implications of the properties of MPrin

Last updated on July 8th, 2021 at 11:48 am

Formulating sound policy vis-à-vis technical debt requires a thorough understanding of the distinction between the MPrin associated with a technical debt and the principal amount of a financial debt. The policy implications of the properties of technical debt arise from three fundamental differences between technical debt and financial debt.

MPrin can change spontaneously

For most financial debts, a formula determines the principal amount. Voluntary actions of the debtor can also affect the principal amount. For example, the debtor might make periodic payments on an installment loan, or new purchases on a credit card account. By contrast, MPrin of a technical debt can change absent any action by the “borrower.” For example, changes in regulations, standards, or technologies can all cause changes in MPrin. More: “How MPrin can change spontaneously”

Technical debt can create more technical debt

Technical debt left in place can create more technical debt without the knowledge or consent of the debtor organization. By contrast, the principal amount of a financial debt can grow, but law or regulation requires notification—and in some cases consent—of the debtor. More: “How MPrin can change spontaneously”

Projecting MPrin with useful precision might not be possible

The cost of retiring a technical debt can depend on how the asset bearing the debt has changed over the life of the debt. And it can depend on what other projects the enterprise is executing at debt retirement time. These factors are difficult to predict. By contrast, projecting the principal amount of a financial debt is formulaic. More: “Useful projections of MPrin might not be attainable”

A pole full of wires
A pole full of wires. Technical debt is everywhere.
The policy implications of these properties of MPrin can be profound. First, spontaneous change in MPrin implies a need for investments in market and technological intelligence. The intelligence of greatest value focuses specifically on potential effects on technical debt. Second, existing technical debt can lead to creating new instances of that debt or other debts. We can limit this “contagion” if we know what kinds of technical debt are most likely to exhibit this phenomenon. Finally, the difficulty of projecting MPrin implies that typical reliance on analytical modeling of enterprise asset evolution in preference to human judgment might be unwise. A wiser course might be investment in employee retention programs focused on the individuals who can provide the necessary wisdom.

This is just a sketch of the problems policymakers confront when dealing with the properties of MPrin. I’ll be addressing them in more detail in future posts.

References

[Boss 2011] Richard W. Boss, “RFID Technology for Libraries,” American Library Association, 2011.

Some libraries are upgrading their book tagging systems from barcodes to RFID tags—what is essentially a platform upgrade. When they do convert, every item in their collections becomes an instance of technical debt until it’s tagged with an RFID. A tagging technician can process about 1,000 items per day. It’s a big job. Available: here; Retrieved: November 21, 2017

Cited in:

[Broverman 2017] Neal Broverman. “The Success of the Gold and Expo Lines Has Taken a Toll on Bus Ridership,” Los Angeles Magazine, March 30, 2017.

Available: here; Retrieved: November 21, 2017

Cited in:

[Fowler 1999] Martin Fowler, Kent Beck (Contributor), John Brant (Contributor), William Opdyke, Don Robert, Erich Gamma (Foreword). Refactoring: Improving the Design of Existing Code. Boston: Addison-Wesley Professional; first edition (July 8, 1999).

Order from Amazon

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kanigel 1997] Robert Kanigel. The one best way: Frederick Winslow Taylor and the enigma of efficiency. Viking Penguin, 1997.

Order from Amazon

Cited in:

[Klein 2017] Gary Klein. Sources of Power: How People Make Decisions, 20th Anniversary Edition. Cambridge, Massachusetts: The MIT Press, 1999.

Order from Amazon

Cited in:

[Laibson 1997] David Laibson. “Golden eggs and hyperbolic discounting,” Quarterly Journal of Economics 112:2, 1997, 443-477.

Available: here; Retrieved: October 25, 2018

Cited in:

[Levin 2012] Kelly Levin, Benjamin Cashore, Steven Bernstein, and Graeme Auld. “Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change,” Policy Science 45, 2012, 123–152.

Available: here; Retrieved: October 17, 2018

Cited in:

[Loewenstein 1992] George Loewenstein and Drazen Prelec. “Anomalies in Intertemporal Choice: Evidence and an Interpretation,” Quarterly Journal of Economics, 57:2, 1992, 573-598.

Available: here; Retrieved: October 12, 2018

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Simon 1973] Herbert A. Simon. “The Structure of Ill Structured Problems,” Artificial Intelligence 4, 1973, 181-201.

Available: here; Retrieved: 10/16/18

Cited in:

[Taylor 1913] Frederick Winslow Taylor. The Principles of Scientific Management. New York: Harper & Brothers, 1913.

Available: here; Retrieved: October 16, 2018 Order from Amazon

Cited in:

[Thokala 2016] Praveen Thokala, Nancy Devlin, Kevin Marsh, Rob Baltussen, Meindert Boysen, Zoltan Kalo, Thomas Longrenn et al. “Multiple Criteria Decision Analysis for Health Care Decision Making—An Introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task Force,” Value in Health 19:1, 2016, 1-13.

Available: here; Retrieved: 10/16/18

Cited in:

[Volpe 2017] Volpe National Transportation Systems Center. “Truck Side Guards Resource Page,” October 2017.

Available: here; Retrieved: November 22, 2017

Cited in:

[Zannier 2007] Carmen Zannier, Mike Chiasson, and Frank Maurer. “A model of design decision making based on empirical results of interviews with software designers,” Information and Software Technology 49, 2007, 637-653.

Available: here; Retrieved October 15, 2018

Cited in:

Related posts

  • MPrin for missing or incomplete capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in a platform component upgrade
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Show Buttons
Hide Buttons