Tag: metaphorical principal

Posted on

The resilience error and technical debt

Last updated on October 4th, 2018 at 02:09 pm

I’ve mentioned the reification error in a previous post (see “Metrics for technical debt management: the basics”), but I haven’t explored its dual, the resilience error. Let me correct that oversight now.

The future USS Zumwalt (DDG 1000) in sea trials, December 2015
The future USS Zumwalt (DDG 1000) is underway for the first time conducting at-sea tests and trials in the Atlantic Ocean Dec. 7, 2015. The first of the Zumwalt class of US Navy guided missile destroyers, it is designed to be stealthy, and to be supported by a minimal crew. After the program experienced explosive cost growth, the class has been downsized from 32 ships to three, and complement increased from 95 to over 140 to reduce capital costs. The three vessels on order now have significantly reduced missions. As one might expect, the causes of these troubles are much debated. But it’s possible that the resilience error plays a role. Before the first of a new class of ships goes to sea, it exists as an abstraction—a collection of concepts, plans, promises, and technologies, tried and untried. Many elements of this collection have never inter-operated with other elements. The first ship represents the first opportunity to see how all the elements work together. Although troubles often appear even before the ship is fully assembled, anticipating all troubles is extraordinarily difficult.

Reification risk is the risk that an error of reasoning known as the reification error might affect decisions—in this case, decisions regarding technical debt. The reification error [Levy 2009] [Gould 1996] (also called the reification fallacy, concretism, or the fallacy of misplaced concreteness [Whitehead 1948]) is an error of reasoning in which we treat an abstraction as if it were a real, concrete, physical thing. Reification is useful in some applications, such as object-oriented programming and design.

But when we reify in the domain of logical reasoning, troubles can arise. For example, we can encounter trouble when we think of “measuring” technical debt. Strictly speaking, we cannot measure technical debt. It isn’t a real, physical thing that can be measured. What we can do is estimate the cost of retiring technical debt, but estimates are only approximations. And in the case of technical debt, the approximations are usually fairly rough—they have wide uncertainty bands. That’s one way for trouble to enter the scene. When we regard the estimate as if it were a measurement, we tend to think of it as more certain than it actually is. Technical debt retirement projects then overrun their budgets and schedules, and chaos reigns.

For example, if we think we’ve measured the MPrin of a class of technical debt, rather than that we’ve estimated it, we’re more likely to believe that one measurement will suffice, and that it will be valid for a long time (or indefinitely). On the other hand, if we think we’ve estimated the MPrin of a class of technical debt, we’re more likely to believe that obtaining a second independent estimate would be wise, and that the estimate we do have might not be valid for long. These are just some of the consequences of the reification error.

The resilience error

If the reification error is risky because it entails regarding an abstraction as a real, physical thing, we might postulate the existence of a resilience error that’s risky because it entails regarding an abstraction as more resilient, pliable, adaptable, or extensible than it actually is.

When we commit the resilience error with respect to an abstraction, we adopt the belief—usually without justification, and possibly outside our awareness—that if we make changes in the abstraction without fully investigating the consequences of those changes, we can be certain that the familiar properties of the abstraction we modified will apply, suitably modified, to the new form of the abstraction.  Or we assume incorrectly that the abstraction will accommodate any changes we make to its environment.

Sometimes we benefit when we modify abstractions; usually we encounter unintended and unpleasant consequences. For example, unless we examine our modifications carefully, it’s possible that the implications of a modification might conflict with one or more of the fundamental assumptions of the abstraction.

Examples of the resilience error

Perhaps a (ahem) concrete example will illustrate. Consider the steel hull of an ocean liner. We can manufacture it more cheaply if we can devise a way to use less steel. So one approach to that goal is to remove a small portion of the bottom of the hull, say, a circular hole one meter in diameter. We send some people into the ship to do the work, and they return with panicky reports of water coming in. But the ship seems fine, so we reject the reports. Even a day later, all seems well. But by the end of the second day, the trouble is obvious. The ship is sinking.

The problem in our example is that the circular hole in the hull violated a fundamental assumption about how ship hulls work: they work by keeping all water out of the ship. We had extended the idea of hull to make it lighter, but in doing so, we encountered some unintended consequences because our extension violated a fundamental property of hulls.

Now for a less fanciful example.

Consider the fictitious company Alpha Properties LLC, which manages small condominium associations (from 25 to 100 units). Things have been going swimmingly at Alpha Properties, and they’ve decided to expand to handle large condominium associations. Their financial accounting software  has worked well, and their employees have become quite expert in its use. Alpha management has heard good reports from other management companies that deal with large client associations. So Alpha decides to use the same software for its larger accounts too. But things don’t work out so well.

The software is fine, but the processes used by the staff are cumbersome and slow. For example, setting up a new association requires too much manual data entry. For a 100-unit association, client setup wasn’t a burden, but for a 900-unit association the problem is just unmanageable.

This is a fine example of the resilience error. When we make this error, we fail to appreciate how an abstraction can encapsulate assumptions that make for difficulties when we try to extend it or apply it in a new or altered context. In this example, Alpha’s data flow processes are the abstraction. The context is signing up a new client association. When the context (signing up a large new client) is different, it violates an internal assumption of the abstraction (the data flow process for signing up a new client).

How the resilience error leads to technical debt

In many cases, the resilience error is at the heart of the causes of technical debt. It works like this. We have an asset that works perfectly well for one set of applications or in one set of contexts. We want to apply that asset in a new way, which might (or might not) require some minor extensions. When we try it, we find that the asset incorporates some assumptions about the application or the context, and one or more of those assumptions are violated by the new application or the new context. Scrambling, we find some quick fixes that can get things working again, but those fixes usually aren’t well designed or easily maintained. The result is a trail of technical debt.

Acquiring companies is like that. Before the acquisition, we think we’ll be able to merge the IT operations to save some expenses in operations. When we actually try it, though, merging them proves to be far more expensive than we imagined. Ah, the resilience error.

What makes this situation so difficult is that often we’re unable to anticipate what assumptions we might be about to violate. That’s why we make the resilience error.

Spotting difficulties with adapting to new applications and new contexts isn’t so difficult with physical entities. For example, we can see in advance that a square peg won’t fit into a round hole. But with abstractions, we can’t always see the problems in advance. Piloting, prototypes, games, and simulations can help us avoid some trouble, but not all.

References

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Other posts in this thread

Posted on

Accounting for technical debt

Last updated on September 16th, 2018 at 03:28 pm

With all the talk of technical debt these days, it’s a bit puzzling why there’s so little talk in the financial community about how to go about accounting for technical debt [Conroy 2012]. Perhaps one reason for this is the social gulf that exists between the financial community and the community most keenly aware of the effects of technical debt — the technologists. But another possibility is the variety of mechanisms compelling technologists to leave technical debt in place and move on to other tasks.

Accounting for technical debt isn’t the same as measuring it
Accounting for technical debt isn’t the same as measuring it. We usually regard our accounting system as a way of measuring and tracking the financial attributes of the enterprise. As such, we think of those financial attributes as representations of money. Technical debt is different. It isn’t real, and it isn’t a representation of money — it’s a representation of resources. Money is just one of those resources. Money is required to retire technical debt, and money is consumed when we carry technical debt, but other kinds of resources are also required. Sometimes we forget that when we account for technical debt.

Here’s an example. One common form of technical debt is the kind first described by Cunningham [Cunningham 1992]. Essentially, when we complete a project, we often find that we’ve advanced our understanding of what was actually needed to accomplish our goals. And we’ve advanced our understanding to such an extent that we recognize that we should have taken a different approach. Fowler described this kind of technical debt as, “Now we know how we should have done it.” [Fowler 2009] At this point, typically, we disband the team and move on to other things, leaving the technical debt outstanding, and often, undocumented and soon to be forgotten.

A (potentially) lower-cost approach involves immediate retirement of the debt and a re-release of the asset — an “echo release” — in which the asset no longer carries the technical debt we just incurred and immediately retired. But because echo releases usually offer no immediate, evident advantage to the people and assets that interact with the asset in question, decision-makers have difficulty allocating resources to echo releases.

This problem is actually due, in part, to the effects of a shortcoming in management accounting systems. Most enterprise management accounting systems track effectively the immediate costs associated with technical debt retirement projects. They do a much less effective job of representing the effects of failing to execute echo releases, or failing to execute debt retirement projects in general. The probable cause of this deficiency is the distributed nature of the MICs — the metaphorical interest charges associated with carrying a particular technical debt. The MICs appear in multiple forms: lower productivity, increased time-to-market, loss of market share, elevated voluntary turnover in the ranks of technologists, and more (see “MICs on technical debt can be difficult to measure”). These phenomena are poorly represented in enterprise accounting systems.

Decision-makers then adopt the same bias that afflicts the accounting system. In their deliberations regarding resource allocation, they emphasize only the cost of debt retirement, often omitting from consideration altogether any mention of the cost of not retiring the debt, which can be ongoing.

If we do make long-term or intermediate-term projections of costs related to carrying technical debt or costs related to debt retirement releases, we do so in the context of a cost/benefit computation as part of the proposal to retire the debt. Methods vary from proposal to proposal. Many organizations lack a standard method for making these projections. And because there’s no standard method for estimating costs, comparing the benefits of different debt retirement proposals is difficult. This ambiguity and variability further encourages decision-makers to base their decisions solely on current costs, omitting consideration of projected future benefits.

Dealing with accounting for technical debt

Relative to technical debt, the accounting practice perhaps most notable for its absence is accounting for outstanding technical debts as liabilities. We do recognize outstanding financial debt as such, but few balance sheets — even those for internal use only — mention outstanding technical debt. Ignorance of the liabilities imposed by outstanding technical debt can cause decision-makers to believe that the enterprise has capacity and resources that it doesn’t actually have. Many of the problems associated with high levels of technical debt would be alleviated more readily if we began to track our technical debts as liabilities — even if we did so for internal purposes only.

But other shortcomings in accounting practices can create additional problems almost as severe.

Addressing the technical-debt-related shortcomings of accounting systems requires adopting enterprise-wide patterns for proposals, which gives decision-makers meaningful comparisons between different technical debt retirement options and between technical debt retirement options and development or maintenance options. One area merits focused and immediate attention: estimating MPrin and estimating MICs.

Standards for estimating MPrin are essential for estimating the cost of retiring technical debt. Likewise, standards for estimating MICs, at least in the short term, are essential for estimating the cost of not retiring technical debt. Because both MPrin and MICs can include contributions from almost any enterprise component, merely determining where to look for contributions to MPrin or MICs can be a complex task. So developing a checklist of potential contributions can help proposal writers develop a more complete and consistent picture of the MICs or MPrin associated with a technical debt. Below are three suggestions of broad areas worthy of close examination.

Revenue stream disruption

Technical debt can disrupt revenue streams either in the course of being retired, or when defects in production systems need attention. When those systems are taken out of production for repairs or testing, revenue capture might undergo short disruptions. Burdens of technical debt can extend those disruptions, or increase their frequency.

For example, a technical debt consisting of the absence of an automated test can lengthen a disruption while the system undergoes manual tests. A technical debt consisting of a misalignment between a testing environment and the production environment can allow defects in a repair or enhancement to slip through, creating new disruptions as those new defects get attended to. Even a short disruption of a high-volume revenue stream can be expensive.

Some associations between classes of technical debt and certain revenue streams can be discovered and defined in advance of any debt retirement effort. This knowledge is helpful in estimating the contributions to MICs or MPrin from revenue stream disruption.

Extended time-to-market

Although technologists are keenly aware of productivity effects of technical debt, these effects can be small compared to the costs of extended time-to-market. In the presence of outstanding technical debt, time-to-market expands not only as a result of productivity reduction, but also from resource shortages and resource contention. Extended time-to-market can lead to delays in realizing revenue potential, and persistent and  irreparable reductions in market share. To facilitate comparisons between different technical debt retirement proposals, estimates of these effects should follow standard patterns.

Data flow disruption

All data flow disruptions are not created equal. Some data flow processes can detect their own disruptions and backfill when necessary. For these flows, the main consequence that might contribute to MICs or MPrin is delay. And the most expensive of these are delays in receipt of orders, delays in processing orders, and delays in responding to anomalous conditions. Data flows that cannot detect disruptions are usually less critical, but they nevertheless have costs too. All of these consequences can be modeled and estimated, and we can develop standard packages for doing so that we can apply repeatedly to MICs or MPrin estimates for different kinds of technical debt.

Last words

Estimates of MICs or MPrin are helpful in estimating the costs of retiring technical debt. They’re also helpful in estimating the costs of not retiring technical debt. In either case, they’re only estimates, and as such, they have error bars and confidence limits. The accounting systems we now use have no error bars. That, too, is a shortcoming that must be addressed.

References

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Other posts in this thread

Posted on

The Principal Principle: Focus on MICs

Last updated on December 11th, 2018 at 06:56 am

When some organizations first realize that their performance is being limited by technical debt, they begin by chartering a “technical debt inventory.” Their goals are to determine just how much technical debt they’re carrying, where it is, how much retiring it all will cost, and how fast they can retire it. That’s understandable. It’s not too different from how one would approach an out-of-control financial debt situation. Understandable, but in most cases, ineffective. With technical debt we need a different approach, because technical debt is different from financial debt. With technical debt, we must be guided by what I call the Principal Principle, which is:

The Metaphorical Principal (MPrin) of a technical debt, which is the cost of retiring it, isn’t what matters most. What usually matters most is the Metaphorical Interest Charges—the MICs.

The door to a bank vault
The door to a bank vault. One way to know that technical debt differs from financial debt is that banks aren’t involved in any way. Treating technical debt as if it had anything in common with financial debt—beyond our own sense of obligation—is a shortcut to real trouble. Remember the Principal Principle.

With technical debt, MICs can vary dramatically. For assets that aren’t being maintained or enhanced, the MICs can be Zero for extended periods. For retiring assets, their technical debt can vanish when the asset is retired. For other assets, MICs can be dramatically higher—beyond the total cost of replacing the asset.

Most people regard MICs as being restricted to productivity problems among engineers. I take a different approach. I include in MICs anything that depresses net income—lost or delayed revenue, increased expenses, anything. For instance, if technical debt causes a two-month delay in reaching a market, its effect on revenues can be substantial for years to come. I regard all of that total effect as contributing to MICs.

So the Principal Principle is that a focus on Principal can be your undoing. Focus on MICs. Drive them to Zero and keep them there.

Other posts in this thread

Posted on

Technical debt use disorder

The American Psychiatric Association has identified a disorder they call Substance Use Disorder (SUD), which includes alcoholism, drug addiction, and other patterns related to substance use [APA 2013]. The research they’ve done can serve well as a model for understanding organizational behavior related to technical debt. In this post I’ll show how to use that model to describe a disorder of organizations that we could call Technical Debt Use Disorder (TDUD). It’s a pattern in which the organization can’t seem to retire much of its technical debt, or can’t stop incurring increasing burdens of technical debt, even though everyone in the organization realizes—or almost everyone realizes—that technical debt is harming the organization.

A ball and chain, with shackle
A ball and chain, with shackle.  Attaching this device to the legs of prisoners or slaves limits their ability to run, just as technical debt limits the ability of organizations to exploit new opportunities, or even to maintain their current market positions.

A brief description of the American Psychiatric Association’s publication, DSM-5®, might help explain the connection between SUD and TDUD. DSM-5 is the fifth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM). It’s used by health care professionals in the United States and much of the rest of the world as a guide to diagnosing mental disorders, and as a framework for further research. First published in 1952, the current revision, DSM-5, was released in 2013.

So what does DSM-5 have to do with technical debt?

What distinguishes responsible use of technical debt from irresponsible use is a topic that has generated many papers, conference presentations, and hallway debates over the years. Although there is consensus about the distinction in many cases, the debate continues. Sometimes, though, research in one field suggests paths forward in seemingly unrelated fields. So much thought and study has been invested in DSM-5 that it’s worth a look to see if the technical debt community can harvest something useful from the research in psychiatry.

I looked at DSM because I noticed that organizations that carry significant volumes of technical debt seem to have difficulty retiring it. In some cases, they also have difficulty halting accumulation of technical debt, or even slowing the rate of accumulation. This struck me as similar to the substance use problems some people encounter. I began to wonder whether there might be parallels between the substance use disorders that afflict people—alcoholism, drug addiction, and so on—and the technical debt problems that afflict organizations.

In the table below is one set of parallels I’ve found. The left column of the table is the list of diagnostic criteria for Substance Use Disorder provided in the DSM. The right column is my rewording of those criteria in an attempt to make them apply to how organizations deal with technical debt. I had thought initially that the rewording exercise might be difficult—that it might be a stretch. And here and there, it was a bit of a stretch. But overall, the SUD framework is a very good fit.

Diagnosing technical debt use disorder

Have a look at the table, and then check out the comments below it about how health care professions use the criteria.

DSM-5 Criteria for Substance Use Disorder (SUD)Criteria for Enterprise Technical Debt Use Disorder (TDUD)
1. Taking the substance in larger amounts or for longer than you’re meant to.
1. Incurring technical debt in larger amounts than you intended and carrying it for longer than you intended.
2. Wanting to cut down or stop using the substance but not managing to.2. Wanting to retire your technical debt or reduce the rate of incurring it but not managing to.
3. Spending a lot of time getting, using, or recovering from use of the substance.3. Spending a lot of time dealing with the consequences of the technical debt you’ve already incurred.
4. Cravings and urges to use the substance.4. Insistent demands on precious resources, causing the enterprise to incur “just a little more” technical debt.
5. Not managing to do what you should at work, home, or school because of substance use.5. Not managing to attend to the needs of existing products, services, or technological infra­structure because of the demands resulting from metaphorical interest charges on technical debt.
6. Continuing to use, even when it causes problems in relationships.6. Continuing to carry technical debt, or continuing to incur yet more technical debt, even though it causes toxic conflict among employees, and problems in customer relationships and strategic partnerships.
7. Giving up important social, occupational, or recreational activities because of substance use.7. Giving up developing important new products or services, or upgrading critical infrastructure, or pursuing new initiatives because of resource deficits traceable to technical debt service.
8. Using substances again and again, even when it puts you in danger.8. Incurring technical debt again and again, even when it puts the enterprise in fiscal danger or danger of losing market position.
9. Continuing to use, even when you know you have a physical or psychological problem that could have been caused or made worse by the substance.9. Continuing to incur technical debt, even when you know you have a fiscal or market leadership problem that could have been caused or made worse by technical debt.
10. Needing more of the substance to get the effect you want (tolerance).10. Needing to incur more technical debt to get the fiscal effect you need—a product delivered or a contract completed.
11. Development of withdrawal symptoms, which can be relieved by taking more of the substance.11. Upon attempting to retire existing technical debt, or halting incurring yet more technical debt, fiscal or market position problems develop in short order, which can be relieved only by incurring yet more debt.

In health care, two or three symptoms indicate a mild substance use disorder; four or five symptoms indicate a moderate substance use disorder, and six or more symptoms indicate a severe substance use disorder. Have a look at the right-hand column. How would you score your organization? Can we categorize the severity of an organization’s problem with technical debt using a scale similar to the one health care professionals use for SUD?

Conclusion

Technical debt isn’t inherently evil. Its existence among technological assets isn’t proof of engineering malpractice. For example, we can decide—responsibly—to deliver a system that carries technical debt, provided that we’ve carefully weighed the consequences of incurring that debt against the consequences of delayed delivery, and provided that we have a workable plan for retiring that debt, or for carrying the burden of its MICs.

But organizations can nevertheless become trapped in a cycle of technical debt, unable to make much progress in reducing it. In some cases, business as usual won’t work. Drastic action is required.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Other posts in this thread

Posted on

Using SMART goals for technical debt reduction

Last updated on May 9th, 2019 at 02:17 pm

Attempting to reduce the enterprise burden of technical debt by setting so-called “SMART goals” in the obvious way can often produce disappointing results. SMART, due to George T. Doran [Doran 1981], is a widely used approach for expressing management goals. “SMART” is an acronym for “Specific, Measurable, Attainable, Realistic, and Time-boxed,” though the last three words (the “ART”) are chosen in various alternative ways. Doran himself used “assignable, realistic, and time-related.”

Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives
Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives. Watch a 2010 interview of Prof. Doran at YouTube.
SMART is so embedded in management culture that many assume without investigation that expressing technical debt reduction goals directly using the SMART pattern will produce the desired results. Also embedded in management culture is the aphorism, “You get what you measure.” [Ariely 2010]  [Bouwers 2010] Combining these two ideas in a straightforward way, one might express the technical debt reduction goal as, “Reduce the burden of technical debt by 20% per year for each of the next five years.”

There is ample support for a claim that this “direct” approach to applying the SMART technique will be ineffective. The fundamental issue is that so much of employee behavior affects technical debt indirectly that it overwhelms the effects of employee behaviors that affect technical debt directly. The result is that although the direct approach does cause some employees to adopt desirable behaviors, their impact is not significant enough compared to the effects of the behaviors of employees who see little connection between their own activities and the burden of technical debt, or who are subject to competing constraints on their behaviors that then cause them to act in ways that increase technical debt.

That’s why it’s necessary for management to develop a series of SMART goals that affect behaviors that have indirect effects on technical debt. In the first part of this post, “Setting a direct SMART goal for technical debt reduction is problematic,” I explore the problems inherent in the direct approach. In the second part, “How to set SMART goals for technical debt,” I provide examples of SMART goals that touch on behaviors that have indirect effects on technical debt.

Setting a direct SMART goal for technical debt reduction is problematic

Let’s begin by exploring some of the problems with the direct approach. In this section, I assume that management has set a SMART goal for the enterprise in the form, “Reduce the burden of technical debt by 20% per year for each of the next five years.” But there’s nothing special about the numbers. My comments below apply to the form of the goal, rather than the specific numbers.

The direct approach assumes measurability

To attain a goal of a 20% reduction in technical debt in a given year, we must be able to measure the level of technical debt at the beginning of the year and the level at the end of the year, presumably with confidence in the 90% range or better. Such a measurement with the precision required might not be possible. Moreover, in most cases the probability that such a measurement is possible is low. For these reasons, setting periodic goals for total technical debt is not a useful management tool.

Consider a simple example. One form of technical debt—and it’s a common form—is missing or incompletely implemented capability. In some instances, the metaphorical principal (MPrin) of a given instance of this debt in the current year can change spontaneously to a dramatically larger value in the following year (or even the following week), due to changes in the underlying asset unrelated to the technical debt, or due to debt contagion, or due to any number of other reasons. When this happens, the technical debt retirement effort for that year can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well.

The direct approach assumes a static principal

With most financial debts, the principal amount is specified at the time of loan origination. Moreover, we can compute the principal at any time given the mathematical formulas specified in the loan agreement.

By contrast, in many cases, the metaphorical principal amount of a technical debt might be neither fixed nor readily computable. We can estimate the MPrin of a given kind of technical debt at a given time, and we can even make forward projections of those estimates. But they are merely estimates, and their error bars can be enormous. See “Policy implications of the properties of MPrin” and “Useful projections of MPrin might not be attainable.”

The direct approach focuses on MPrin, not MICs

Objectives expressed in terms of the volume of technical debt—the total MPrin—are at risk of missing the point. Total MPrin is not what matters most. What matters is MICs—the total cost of carrying the debt. Even more important is the timing of arrival of the MICs.

And like MPrin, MICs can vary in wild and unpredictable ways. For example, the MICs for a piece of technical debt borne by an asset that isn’t undergoing maintenance or enhancement can be zero; in a later time period, when that asset is undergoing enhancement, the MICs can be very high indeed. See  “MICs on technical debt can be unpredictable” for a detailed discussion.

Priority setting for technical debt retirement is most effective when it takes into account the timing of MICs. For example, if we know that we must enhance a particular asset by FY21 Q3, and if we know that it bears technical debt that adds to the cost of the enhancement, retiring that debt in FY20 would be advisable. On the other hand, if that form of technical debt has no effect on MICs for the foreseeable future, retiring that technical debt might not be worth the effort.

The direct approach fails to distinguish legacy technical debt from incremental technical debt

Unless policies are already in place governing the formation of new technical debt—what I call incremental technical debt—technical debt retirement programs might encounter severe difficulty meeting their goals. The technical debt retirement program might simply be unable to keep up with the formation of new technical debt resulting from new development or from ongoing maintenance and enhancement of existing assets.

The direct approach fails to anticipate the formation of enterprise-exogenous technical debt

Technical debt can sometimes form as a result of innovations, changes in standards, or changes in regulations that occur entirely external to the enterprise. I call such technical debt enterprise-exogenous. When this happens, the technical debt retirement effort can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well. Before initiating a technical debt reduction program, it’s wise to first deploy a program that’s capable of retiring technical debt at a pace that at least equals the pace of formation of enterprise-exogenous technical debt.

Incurring technical debt is sometimes the responsible thing to do

At times, incurring technical debt is prudent. In some situations, accepting the debt you’ve incurred—even for the long term—might be called for. Because strict goals about total technical debt can lead to reluctance to incur debt that has a legitimate  business purpose, whatever goals are set for total technical debt must be nuanced enough to deal with these situations. Goals for total technical debt that adhere strictly to the SMART goal pattern sometimes lack the necessary level of nuance.

How to set SMART goals for technical debt

SMART goals can work for technical debt management, but we must express them in ways that are more closely related to behavioral choices. Here are some examples of SMART goals that can be effective elements of the technical debt management program. Some of these examples are admittedly incomplete. For example,  I offer no proof of assignability, attainability, or realism, because they can vary from organization to organization, or because the goal in question must be distributed across multiple organizational elements in ways peculiar to the organization.

At least 30% of incremental technical debt will be secured technical debt at the end of Year 1; 60% by the end of Year 2

Incremental technical debt is technical debt that’s incurred in the course of work currently underway or just recently completed. Because it’s so well understood, its MPrin can be estimated with higher precision than is usually possible with legacy technical debt. That precision is needed for defining the collateral and resources used to secure the debt.

A secured technical debt, like a secured financial debt, is one for which the enterprise reserves the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. Beyond financial resources, they might include particular staff, equipment, test beds, and downtime. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes.

Within one year, at least 50% of all incremental technical debt will be retired within one year of its origination; 70% within 18 months

This goal also exploits the fact that incremental technical debt can be estimated with relatively high precision. As a goal, it builds on the goal above by requiring that the resources pledged to retire incremental debts actually be expended as intended.

Within one year, all engineers and their direct supervisors will be educated in basic technical debt concepts

The educational materials will be developed in the next five months and piloted with 10% of the technical staff within seven months. The material will include an online proficiency test that 90% of engineers will have successfully passed within a year.

Within one year, 90% of project plans will include projections of the MPrin of the incremental technical debt they expect to generate for each delivery cycle

This information is useful for making forward projections of resources needed to secure incremental technical debt. Tracking the accuracy of these projections helps project planners improve their estimates.

Within one year, initiate a practice of identifying the top five forms of legacy technical debt, ranked by the volume of the contagion

Debt contagion is the propagation of a given form of technical debt by creating new system elements or assets in forms compatible with elements already identified as technical debt. By examining the body of incremental technical debt created enterprise-wide in a given time period (say, by fiscal quarter), we can determine the portion of that incremental debt that results from contagion, for each type of contagious legacy technical debt. This data is needed to identify the most contagious forms of legacy technical debt. They are prime candidates for debt retirement.

Within one year, initiate an industrial intelligence practice that is responsible for projecting the formation of enterprise-exogenous technical debt

This group must have a sophisticated grasp of the technologies in use within the enterprise that already bear enterprise-exogenous technical debt, or which could be subject to the formation of enterprise-exogenous technical debt. Its responsibilities cover enterprise products and services, as well as enterprise infrastructure. It issues advisories as needed, and an annual forecast. The group is also responsible for recommending and monitoring participation in industrial standards organizations. The group reports to the CIO or CTO.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Other posts in this thread

Posted on

Useful projections of MPrin might not be attainable

Last updated on November 28th, 2017 at 11:32 am

SummaryExpect the unexpected with technical debt retirement efforts. Technical debt retirement efforts can conflict with ongoing operations, maintenance of existing capabilities, development of new capabilities, cyberdefense, or other technical debt retirement efforts. Although these conflicts are technical in nature, resolving them can involve business priorities at any level. Planners must be aware of these potential conflicts, and coordinate with their leaders. Policymakers can make important contributions to the enterprise mission if they can devise guidelines and frameworks for resolving these conflicts as closely as possible to the technical level.

For planning purposes, it’s necessary from time to time to project the size of the MPrin for given . The need arises when planning debt retirement, or when preparing debt retirement options for determining resource allocations. Although retiring some kinds of technical debt is straightforward, other kinds of debt can be intertwined with each other. Still others might appear to be easily retired, but actual retirement efforts expose unanticipated entanglements. Moreover, debt retirement efforts can sometimes interact with other debt retirement efforts, operations, maintenance, cyberdefense, and new development in both expected and unexpected ways. For these reasons, making estimates of the MPrin with enough precision to be useful can be notoriously difficult.

A tangle of cordage
A tangle of cordage on board ship. Different kinds of technical debt can be tangled with each other, and untangling them can affect various other engineering efforts. Preparing an asset for a debt retirement effort by doing some preliminary untangling might be wise before trying to estimate the MPrin of any affected class of technical debt.

These considerations rarely arise when planning retirement of financial debts, because money is fungible. We might indeed have other uses for financial resources, but every unit of cash is equivalent to every other. That freedom is not necessarily available when planning resource allocations for technical debt retirement.

For example, not every engineer is equally qualified to address every problem. Some people are particularly capable for certain kinds of work, and not very qualified for other kinds. The problem of scheduling specialists is notorious for generating bottlenecks. And split assignments create even more trouble. People are not fungible.

Planning retirement of a particular set of technical debt classes requires knowledge of any efforts with which that retirement effort might interact. That information might not be available or might not be known. In general, preliminary work to decouple these activities — often called refactoring — can greatly simplify technical debt retirement planning. Even before undertaking refactoring, gathering information about the entanglements of different classes of technical debt can be very helpful. Because allocating resources to such efforts can be difficult in feature-oriented cultures, policymakers can take the lead in raising the priority of such efforts.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Related posts

  • MPrin for missing or incompletely implemented capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in platform component upgrades
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

How technical debt can create more technical debt

Last updated on November 28th, 2017 at 11:29 am

SummaryAlthough the MPrin of a given can increase or decrease even if instances of that debt remain untouched, the total volume of all other technical debt can change as well. If a class of technical debt is allowed to remain outstanding, its volume can increase as a consequence of seemingly unrelated actions or decisions. Moreover, its existence can cause increases in the volume of other existing classes of technical debt, and its existence can lead to the formation of new classes of technical debt. When formulating technical debt management policy, it's essential to understand these mechanisms and risks.

Decisions to defer technical debt retirement must take into account phenomena that, if left unattended, can lead to (a) increasing volume of a given , (b) increasing volume of other existing classes of technical debt, and (c) generating new classes of technical debt. An example can illustrate this behavior.

Suppose Working at a computerwe have a fleet of desktop computers, running a mix of operating systems. A few of these systems are running Windows 8 and the rest are running Windows 10. We’d like to upgrade the Windows 8 machines to Windows 10, but we cannot, because some of their users need access to a (fictional) scriptable application called CRUSH that is not available for Windows 10. CRUSH for Windows 10 is promised “shortly.” Instead of asking our CRUSH users to find an alternative to CRUSH, we defer the Windows 8 upgrade, in the hope that CRUSH for Windows 10 will soon arrive. Meanwhile, other Windows 8 users are happy to continue using Windows 8, and some of them have acquired — and have grown fond of using — another similar (fictional) scriptable package called REMOTE, which is also unavailable for Windows 10. Worse, the CRUSH user community is continuing to grow. Thus, by deferring the Windows 8 upgrade, we have made space for additional problems preventing the upgrade to Windows 10. The technical debt associated with the Windows 8 upgrade now includes Windows 8 itself, and all the scripts, documents, and knowledge that are accumulating for both CRUSH and REMOTE.

The lesson here is not to ban scriptable applications, nor to compel desktop users to adhere to an enterprise standard. Both options create numerous problems. The point of the example is merely that deferring debt retirement can enable formation of new instances of existing technical debt (the growth of the CRUSH user community and the assets they continue to develop) and new, unrelated debt (the introduction of REMOTE). Thus, if we make a decision to defer retirement of a class of technical debt, we must consider all costs of such deferment, including expansion of the total volume of technical debt, and all its consequences, as expressed as metaphorical interest charges and MPrin.

Some of the new technical debt that forms when we leave existing debt in place is closely related to the existing debt. For example, once we’ve implemented some part of an asset in a way that we now acknowledge contains a form of technical debt, we tend to apply that same approach when we undertake extensions or enhancements, rather than using what everyone might acknowledge is a superior approach. Martini and Bosch have identified a phenomenon they call debt contagion [Martini 2015], whereby creating new system elements in forms compatible with elements already identified as debt effectively causes debt propagation. This practice helps us maintain some degree of uniformity in the asset, recognizing that in doing so we’re increasing the MPrin of a given class of technical debt. These future expansions of MPrin can be difficult to predict at the time we first incur the debt, or at any time.

However, some forms of technical debt are far less discriminating with respect to the kinds of technical debt they spawn. Debt with this property tends to be associated with the processes used to develop or maintain technical assets. In “A policymaker’s definition of technical debt,” we cite Pugh’s example of acceptance test debt as a form of technical debt [Pugh 2010].

But acceptance test debt can reduce the ability of the organization to retire technical debt. In the absence of automated acceptance tests, testing system components from which technical debt has recently been removed is less efficient and reliable than it would be if automated acceptance tests were available, which retards debt retirement activity and which might even prevent the organization from attempting debt retirement in some circumstances. In a future post, we shall describe how a deficient regime of reviews and inspections can also lead to incurring new technical debt, or to elevated levels of legacy technical debt.

Our final example illustrates how interfaces — which, ironically, were conceived to insulate one portion of an asset from others — can act so as to propagate technical debt. This example could apply to either hardware or software. Given a system S composed of several modules, suppose that module M of S provides services of some kind to several other modules of S. M does contain some technical debt, of a form whose retirement would simplify M’s interface. Because that change would require changes to the modules that use M’s services, retiring the debt has been deferred. Meanwhile, new modules are being introduced into S, and, of course, they must use M’s existing interface. The MPrin of the technical debt associated with M’s interface thus expands.

Unless we provide an alternate version of M (call it M’) or an alternate interface to M, this process of MPrin expansion can repeat whenever new modules are introduced into S. But even if we do provide M’ or an alternate interface to M, engineers must consciously refrain from using the older approaches. Some will refrain, but some might not. Those who are under severe schedule pressure, or those who cannot or will not learn the new approaches, or those who are directed not to use the new approaches, might continue to use the familiar approaches as long as they are able. The MPrin associated with M can thus continue to expand, albeit perhaps at a reduced rate.

Technical debt, left in place, can grow and spawn new forms of technical debt. Make technical debt retirement a priority.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Related posts

  • MPrin for missing or incompletely implemented capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in platform component upgrades
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

Policy implications of the properties of MPrin

Last updated on May 21st, 2019 at 05:38 pm

Formulating sound policy vis-à-vis technical debt requires a thorough understanding of the distinction between the MPrin associated with a technical debt and the principal amount of a financial debt. There are three fundamental differences between them.

MPrin can change spontaneously

For most financial debts, a formula determines the principal amount. Voluntary actions of the debtor can also affect the principal amount. For example, the debtor might make periodic payments on an installment loan, or new purchases on a credit card account. By contrast, MPrin of a technical debt can change absent any action by the “borrower.” For example, changes in regulations, standards, or technologies can all cause changes in MPrin. More: “How MPrin can change spontaneously”

Technical debt can create more technical debt

Technical debt left in place can create more technical debt without the knowledge or consent of the debtor organization. By contrast, the principal amount of a financial debt can grow, but law or regulation requires notification—and in some cases consent—of the debtor. More: “How MPrin can change spontaneously”

Projecting MPrin with useful precision might not be possible

The cost of retiring a technical debt can depend on how the asset bearing the debt has changed over the life of the debt. And it can depend on what other projects the enterprise is executing debt retirement time. These factors are difficult to predict. By contrast, projecting the principal amount of a financial debt is formulaic. More: “Useful projections of MPrin might not be attainable”

A pole full of wires
A pole full of wires. Technical debt is everywhere.

The policy implications of these properties of MPrin can be profound. The possibility of spontaneous change in MPrin implies a need for investments in market and technological intelligence focused specifically on potential effects on technical debt. Moreover,  existing technical debt can cause the creation of new instances of that debt or other debts. This “contagion” implies a need for awareness of what kinds of technical debt are most likely to exhibit this phenomenon. Finally, the difficulty of projecting MPrin implies that typical reliance on analytical modeling of enterprise asset evolution in preference to human judgment may be misplaced. A wiser course might be investment in employee retention programs focused on the individuals who can provide the necessary wisdom.

This is just a sketch of the problems policymakers confront when dealing with the properties of MPrin. I’ll be addressing them in more detail in future posts.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Related posts

  • MPrin for missing or incompletely implemented capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in platform component upgrades
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

The metaphorical principal of a technical debt

Last updated on November 21st, 2017 at 09:05 am

The “principal” amount of a technical debt is not like the principal amount of a financial debt. In fact, the two are so different that policymakers can make real trouble for their organizations if they fail to take the differences into account.

Accomac Debtors’ Prison in Accomac, Virginia
Accomac Debtors’ Prison in Accomac, Virginia. Built in 1783 as a jailer’s house, it served as a debtors’ prison from 1824 to 1849. Debtors imprisoned there were required to work to earn their keep and to retire their debt, if they could not find other means to do so. Today, bankruptcy laws facilitate seizure of property to retire debts, a much more enlightened approach. Requiring the product or service engineering functions to fund technical debt retirement efforts through expense budgets is analogous to debtors’ prison. A more enlightened approach is needed.  Photo (cc) Ser Amantio di Nicolao, courtesy Wikimedia Commons.

In the field of finance, the principal amount of a loan at the time of origination is the amount that was borrowed. Over time, as the debtor makes payments according to the terms of typical loan agreements, the principal either remains constant, in which case it’s repaid in a lump sum at a specified date, or it declines gradually, in increments, with each payment. This is the widely accepted layperson’s definition, and it’s the basis of the association people make with respect to the metaphorical principal amount of a technical debt.

That’s unfortunate. Because the metaphorical principal amounts of most technical debts behave very differently from the principal amounts of financial debts, using the term principal to refer to the metaphorical principal associated with a given kind of technical debt is risky. The risk arises from confusing financial principal, which is typically fixed or slowly declining, with the metaphorical principal of technical debt, which can exhibit sudden and dramatic fluctuations. These confusions arise because of unintended associations of the technical debt metaphor.

Using an alternative term that makes the metaphor obvious can limit this risk. One such term is metaphorical principal, or for convenience, MPrin.

Although the distinction between the initial principal amount of a technical debt and MPrin is not universally recognized [Seaman 2013], some have addressed it. For example, Avgeriou et al. define the initial principal of a particular class of technical debt as the savings realized by incurring the debt, and the current principal as the resources required to deploy a different or better solution now [Avgeriou 2016].

The initial principal concept of Avgeriou et al. is what we call in this blog the MPrin at the time the debt is incurred, or initial MPrin. Although the initial MPrin does have some value for decision makers at the time the debt is incurred, it’s most valuable when deciding whether or not to incur the debt, if, indeed, one has an opportunity to make such a decision. However, once the debt has been incurred, the current MPrin is what matters; initial MPrin becomes irrelevant.

Policymakers must keep clearly in mind that the MPrin of a given kind of technical debt is the total cost of retiring that debt, at the time it is retired, including all cost sources.

We’ll have a look at the policy implications of the properties of MPrin next time.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.

Available: here; Retrieved: March 10, 2017.

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.

Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Related posts

  • MPrin for missing or incompletely implemented capability
    In some instances, technical debt is actually a missing or incompletely implemented capability. If we retire the debt by completing the implementation, the MPrin is the cost of that effort, plus any training, testing, and lost revenue. If we retire the debt by halting or withdrawing the capability, the MPrin is the total cost of removal, plus testing and lost revenue.
  • MPrin when standards or regulations change
    The Metaphorical Principal of a technical debt that’s incurred as a result of a change in standards or regulations, internal or external, is the cost of bringing all affected assets into full compliance. Properly accounted for, however, the MPrin should include ripple effects, which are the changes in other assets that are required to keep them compatible with the assets that are directly affected.
  • MPrin in platform component upgrades
    Platform component upgrades often trigger the need to make changes in whatever sits atop the platform, to maintain compatibility with the platform. Those changes obviously contribute to MPrin. But less obvious are the contributions that arise from deferring the upgrade.
  • MPrin in a development project
    The MPrin of an asset that is subjected to new development or enhancement has some special characteristics. For an existing asset, new development can lead to duplication of capability. For new assets, unanticipated opportunities can transform into technical debt components that were not viewed as technical debt, without ever changing them in any way.
  • Examples of MPrin in four scenarios
    Some examples might help to clarify the differences between the principal of financial debts and the Metaphorical Principal of a technical debt. The examples to come in the next four posts are designed to illustrate the unique properties of MPrins of technical debts.
Posted on

Case 3: Help desk operations

Last updated on November 21st, 2017 at 08:34 am

This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.

Background

A tech support person at workDuring the troubles following release of UGI’s StrawIntoGold 1.0 product, the Help Desk operated by UGI’s Customer Service Department was inundated with calls for assistance. Customer Service alerted Engineering, which provided an explanation and an estimated repair date for the customer service representatives to pass on to callers, but in the crush and panic, neither Engineering management nor Customer Service management provided a script for the representatives to use when the calls came in. Consequently, calls took approximately 15% longer to handle than they would have if a carefully worded script had been available. Further, the message conveyed to customers was not always clear or consistent, which resulted in some customers calling again with the same issue.

Discussion

The decision not to provide customer service representatives with a script can be viewed as incurring a technical debt. The extra time handling calls, the extra calls that resulted from the absence of a script, and even a few lost customers, can be viewed as the metaphorical interest charges on that technical debt. Because of the singular nature of this incident, it’s doubtful that a script will ever be written, but if it were, the cost of doing so, and the cost of distributing it and training all customer service representatives would be the metaphorical principal of this technical debt.

The policymaker’s perspective

UGI doesn’t have a means of making those who incurred the debt accountable for the metaphorical interest charges on that debt. In this case, the Customer Service function incurs additional operating expenses because the Engineering and Customer Service, together, elected not to develop a script for the customer service representatives.

Another component of the metaphorical interest charges is the total of lost sales, damage to UGI’s reputation, and possible loss of market share. Marketing could have stepped in to assist with limiting that damage, but because they viewed the problem as technical, they did not participate. A whole-enterprise perspective on managing the technical debt might have led to a collaboration between Engineering, Marketing, and Customer Support to build better relationships with the customers who were affected by the incident.

Accounting properly for the metaphorical interest charges associated with technical debt can lead to a better understanding of the effects of technical debt.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

The Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

This article is probably the source of the adage “You are what you measure.” Personally, I believe it’s overstated. That is, it’s true in the large, perhaps, but not in detail. Moreover, there are some things that we are that can’t be measured. But it’s important to understand the content of this article because so many people take it as dogma. Available: here; Retrieved: June 4, 2018

Cited in:

[Avgeriou 2016] Paris Avgeriou, Philippe Kruchten, Ipek Ozkaya, and Carolyn Seaman, eds. “Managing Technical Debt in Software Engineering,” Dagstuhl Reports, 6:4, 110–138, 2016.

Available: here; Retrieved: March 10, 2017.

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gould 1996] Stephen Jay Gould. The mismeasure of man (Revised & Expanded edition). W. W. Norton & Company, 1996.

Order from Amazon

Cited in:

[Levy 2009] David A. Levy, Tools of Critical Thinking: Metathoughts for Psychology (second edition). Long Grove, Illinois: Waveland Press, Inc., 2009.

Order from Amazon

Cited in:

[Martini 2015] A. Martini and J. Bosch. “The danger of architectural technical debt: Contagious debt and vicious circles,” Working IEEE/IFIP Conf. Softw. Arch., 2015.

Cited in:

[Pugh 2010] Ken Pugh. “The Risks of Acceptance Test Debt,” Cutter Business Technology Journal, October 2010, 25-29.

Cited in:

[Seaman 2013] Carolyn Seaman. “Measuring and Monitoring Technical Debt” 27 March 2013. Slides.

Defines technical debt as the gap between just making it work and doing it right. This is the initial principal approach to the definition. Considers known defects not fixed to be technical debt.

Cited in:

[Whitehead 1948] Alfred North Whitehead. Science and the Modern World. New York: Pelican Mentor (MacMillan), 1948 [1925].

Order from Amazon

Cited in:

Related posts

  • Case 2: New software Development
    This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue is not actually technical debt.
  • Case 1: A platform upgrade
    This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker's view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.
  • Technical debt in the highway system
    Here’s an example of using the technical debt metaphor in civil engineering. In this case, it helps us understand why, in some cases, leaving some technical debt in place can be a better choice than retiring it.
  • Technical debt in a rail system
    Here’s an example of using the technical debt metaphor to think about how elements of a hardware asset can limit the ability of the enterprise to attain and maintain market leadership. The asset in this example is the trackbed of Amtrak’s Acela Express.
Show Buttons
Hide Buttons