Nine indicators of wickedness

Last updated on December 1st, 2018 at 09:04 am

Several properties of the problem of designing technical debt retirement projects tend to make those design problems more likely to be wicked problems—that is, more likely to satisfy all ten of the criteria of Rittel and Webber [Rittel 1973]. I call these properties indicators of wickedness.

The interchange between Interstate 35 and U.S. Route 30, just outside Ames, Iowa
The interchange between Interstate 35 and U.S. Route 30, just outside Ames, Iowa. The new flyover ramp is indicated in red. It replaces the cloverleaf ramp in the upper right quadrant of the cloverleaf. A construction error has forced a delay, while the piers of the flyover ramp bridge are corrected. The cloverleaf, which was designed with curves a bit too tight, was a high-accident area, and thus constituted a technical debt, with the ongoing vehicle accidents comprising the metaphorical interest charges. The construction error in the flyover ramp piers necessitated a rollback. Rollbacks often indicate wickedness in the projects to design technical debt retirement projects, but in this case, the indicated wickedness is not much greater than was anticipated by the project designers. Construction drawing by Iowa Department of Transportation  [Iowa DOT 2016].
Although we usually have some notion of the degree of wickedness of a given design effort for a technical debt retirement project, actually executing the debt retirement project can reveal unanticipated issues and complexity. Some of what’s revealed can cause us to adjust our estimate of the degree of wickedness of the design effort. If we know in advance what kinds of revelations are most likely to cause such adjustments, we can reduce the incidence of unanticipated revelations.

As I noted in my post, “Degrees of wickedness,” we can regard all problems as lying on a Tame/Wicked spectrum, with wicked problems lying at the extreme Wicked end of the spectrum, and the tamest of the tame lying at the opposite end. As for the ten criteria of wickedness developed by Rittel and Webber, I proposed that they could be satisfied in degrees, with the most wicked problems satisfying all ten criteria absolutely.

As a quick review, here are the attributes of wicked problems as Rittel and Webber see them [Rittel 1973], rephrased for brevity:

  1. There is no clear problem statement
  2. There’s no way to tell when you’ve “solved” it
  3. Solutions aren’t right/wrong, but good/bad
  4. There’s no ultimate test of a solution
  5. You can’t learn by trial-and-error
  6. There’s no way to describe the set of possible solutions
  7. Every problem is unique
  8. Every problem can be seen as a symptom of another problem
  9. How you explain the problem determines what solutions you investigate
  10. The planner (or designer) is accountable for the consequences of trying a solution

Below is a sample of conditions or situations that tend to increase the wickedness of the problem of designing a technical debt retirement project. I have no data to support these conjectured effects. But the principles I used to generate them are three:

  • If a phenomenon expands the set of stakeholders in a debt retirement project, it tends to enhance the wickedness of the design problem.
  • If a phenomenon increases the number or heterogeneity of the assets or processes that must be considered, it tends to enhance the wickedness of the design problem.
  • If a phenomenon creates a need for a rollback of work performed as part of the debt retirement project, and that rollback creates a need to re-design the debt retirement project, it tends thereby to enhance the wickedness of the design problem.

In what follows, I use the term “DRP” to indicate the Debt Retirement Project itself, as distinguished from the effort to design the DRP, which I refer to as “DDRP.” The problem whose wickedness we’re considering is not the DRP itself, but the DDRP. Also, let DBA (for debt-bearing assets) be the set of assets undergoing modification in the context of the DRP, and let IA (for interacting assets)  be the set of assets, excluding the DBA assets, that interact directly or indirectly with assets in the DBA.

With all this in mind, I offer the following nine examples of indicators of wickedness of the DDRP.

1.   A previous attempt to retire this debt was abandoned

Perhaps the most significant indicator of the wickedness of the DDRP is either the failure of a previous attempt to execute a DRP with similar objectives, or the failure of a previous attempt to execute a DDRP for a DRP with those objectives. There are two reasons why such failures are significant indicators of wickedness.

First, it’s reasonable to assume that these previous attempts weren’t founded on any recognition of the wickedness of the DRP or the DDRP. Few such efforts are.  (A Google search for the two phrases “technical debt” and “wicked problem” yields less than 1000 results) (update 12 Nov 2018: 1160 results) Consider first the DDRP. If it is a wicked problem, proceeding as if it were not would very likely fail. If the designers of the previous DDRP did assume that it was a wicked problem, investigating their approach could prove invaluable, and save much time and effort. An analogous argument applies for the DRP itself.

Second, if the previous attempt to execute a DRP with similar objectives has left traces of itself in the DBA, and if those traces must be taken into account while executing the DDRP, they might complicate the DRP, and they might be incompletely addressed in the DDRP. To the extent that these conditions prevail, Criterion 5 is satisfied, and the DDRP exhibits wickedness.

2.   Some revenue streams need to be interrupted

If the work of the DRP entails temporary interruption of revenue streams, executing the DRP can have significant and long lasting effects on the organization. In estimating the cost of the DRP, it’s clearly necessary to account for the financial impact of any revenue shifted into the future, and any revenue irretrievably lost as well. And in some cases, market share might also suffer. All of these factors tend to increase the wickedness of the DDRP.

When these effects are expected, political opposition to the DRP can develop. Senior management can prevent this opposition from halting the DDRP inappropriately by requiring that the business case for the DRP include these financial factors and demonstrate clearly the need to proceed despite them. Involving potential political opponents of the DRP in business case development can be an effective means of ensuring the strength of the business case.

The ability to model all these financial effects is an important organizational asset that can be developed and maintained, for deployment across multiple DDRPs. The organization can monitor DRPs, gathering actual experience data for comparison to the effects projected in the respective business cases of the DRPs. Those comparisons are useful for enhancing the modeling capability.

3.   Some assets not directly touched need to be re-tested

The DDRP is more likely to be a wicked problem if, as a result of the changes executed in the DRP, any of the assets in IA need to be re-tested after or during DRP execution. The need to re-test any assets in IA typically arises when there’s some risk that the DRP’s changes in the DBA could somehow affect the performance of the assets in IA, and when the consequences of such a risk event are severe.

This scenario enhances of the wickedness of the DDRP for at least five possible reasons.

  • Baseline testing of IA is necessary to enable the DRP team to recognize the effects of the DRP on IA behavior. But this baseline testing can reveal pre-existing and unaddressed faults. Because leaving those faults in place can seriously complicate interpretation of anomalies that appear in IA assets after DRP work has begun, the DDRP team might insist that the owners of the IA assets in question address some of these faults. With regard to these issues, political differences between the DDRP team and the owners of IA assets are possible.
  • The additional testing of IA assets tends to expand dramatically the set of stakeholders affected by the DRP, to include the owners, users, and maintainers of the IA assets.
  • The additional testing of IA assets can increase the need to interrupt revenue streams temporarily, and increase the number, duration, and frequency of such interruptions.
  • The additional testing of IA assets can require expertise and staffing beyond the DRP project team, which can disrupt other elements of the organization as the people needed are temporarily assigned to IA testing.
  • The additional testing of IA assets can reveal unanticipated consequences of the DRP alterations, which can trigger re-planning or re-design of the DRP during its execution. That re-planning or re-design, in turn, can trigger alterations in the DDRP.

The need to re-test assets not directly touched in the DRP is more likely when the DRP alters the external behavior of any of the DBA assets. The goal of many DRPs is improvement of the internals of assets without altering their external behavior, except possibly for performance improvements. This goal is desirable because it limits the need for re-testing and re-certification of IA assets. However, some kinds of technical debt appear in the externals of the DBA assets, including their architecture, behavior, appearance, or interfaces. Compared to DRPs that do not alter the externals of DBA assets, retiring the kinds of technical debt that alter the externals of DBA assets is inherently more difficult and more risky because it requires more extensive re-testing and re-certification of both DBA and IA assets.

4.   Multiple sites are directly touched

When the DRP entails modification of technological assets of geographically dispersed organizations, one consequence is a tendency to increase the wickedness of the DDRP. This comes about because of factors including the following:

  • Sites might be dispersed not only geographically, but they might also be separated by language boundaries, legal jurisdictions, cultural divides, time zones, and much more. The required work can vary from site to site for technical reasons and because of variations arising from these non-technical factors.
  • The multiple sites might have different landlords, with different lease agreements governing the organization’s occupancy of the property. This is just one of many factors that increase the numbers of stakeholders and exacerbate their heterogeneity. And the leases might constrain the kind of work that can be performed according to the day of the week or time of day.
  • If local vendors provide services such as communications or Internet connections to some of the sites, and if the work of the DRP involves these technologies and the local vendors, the task of coordinating all the different players can be complex and riddled with unanticipated obstacles.

For example, if the work involves networking hardware and software, work that we might prefer to perform at night or on a weekend might need to be carried out during business hours for some of the sites. For a global enterprise, there might not be a time of day when no sites are conducting regular business.

As a second example, consider a network upgrade for the retail branch offices of a global bank. If that upgrade requires trenching for new cable connections, the project design must take into account local regulations governing the trenches, including how they must be permitted, dug, re-filled, covered, and marked while still open. These regulations vary with national and sometimes local jurisdiction. The complexity causes most organizations to rely on local vendors, but even then, the vendor selection process must include reliable vendor assessment and evaluation. Scheduling becomes a complex and risky endeavor.

For these reasons, a DDRP that involves technological assets housed at multiple sites geographically dispersed has an elevated probability of exhibiting the properties of a wicked problem.

5.   Government agencies and/or industrial standards organizations must re-certify assets

Another driver of stakeholder expansion is the need for re-certification of assets after they’ve been modified in the course of executing the DRP. The certification agencies can range from local and municipal regulators to national regulators and pan-industrial standards organizations. The sheer number of possibilities is itself a contributor to increased wickedness, but the nature of the operating style of these organizations merits special notice.

For the most part, these agencies operate without competitors, whether they are government elements or private organizations. Perhaps for this reason, “customer service” might not be their strength, and gaining timely cooperation from them might be a challenging undertaking. Even though re-certification might be a small part of the DRP, it can easily become a blocking obstacle. Researching these requirements and their associated lead times, and maintaining a current knowledge base about them, can be a non-trivial task of the DDRP.

6.   Non-technical stakeholders must change their behavior

Generally, people don’t like to change how they do their jobs. There are exceptions, of course, if they recognize a benefit that arrives in some immediate and visible way. But unless there is a recognizable benefit, requiring people to change their work patterns as part of a DRP is likely to increase the wickedness of the DDRP. And the difficulty is more problematic if the people affected are technically unsophisticated, because they’re less likely to appreciate the value of managing technical debt, and less likely to accept explanations of that value when those explanations are offered.

DDRP wickedness increases in this case because, in addition to retiring the technical debt, it must address the tasks of motivating and training the affected population. That requires preparing materials, scheduling and accounting for the time spent in training, and monitoring training effectiveness. The business case must also address these issues, but in addition, it must provide evidence required to defuse any political opposition that might otherwise develop.

7.   Discovery of major unanticipated complexity triggers re-design

Unanticipated complexity happens in almost every project of almost any kind. But for DDRPs, unanticipated complexity significant enough to trigger significant adjustment or re-design of the DRP during execution of the DRP is another matter. Such a discovery can mean that the DBA assets or their connections to the IA assets have changed since the plan was developed. Or it can mean that the design team had an incomplete or incorrect understanding of the problem they were trying to solve. These events can occur for a number of reasons.

Technical causes are perhaps more easily imagined, so I’ll focus on non-technical causes, which can actually be more serious. For example, suppose that a political alliance enabled the VP of Sales and the VP of Engineering to reach a deal that enabled the DRP team to work on some DBA assets critical to the Sales function, by taking them off line for defined periods. If that political alliance weakens, or if the deal between the two VPs collapses for some other reason, the scheduled downtime of those assets might vanish or be shortened dramatically. This pattern is more likely to arise in situations in which the DDRP team is not a party to such agreements. The DDRP team must be a party to any agreements regarding access to assets by the DRP team.

As a second example, consider what happens when the enterprise undertakes an acquisition that wasn’t revealed to the DDRP team during their design effort. Because chances are good that the DDRP would have a significant amount of re-work to do in such cases, the DDRP team must be kept informed of any organizational changes that could affect the DRP, for the active life of the DRP.

Re-designing the DDRP can take time. Elements of the DDRP that have short shelf lives must be revisited during re-design. And the need to re-design can also indicate gaps in the DDRP team’s understanding of the problem.

All of these conditions tend to move the DDRP in the direction of increased wickedness.

8.   Weekend or middle-of-the-night work periods are required

The need to perform critical operations on weekends or in nighttime hours suggests three things. First, the work is risky in the sense that undetected faults that go into production can lead to costly operational errors. Second, the organization lacks a simulated operating environment that emulates the actual operating environment faithfully enough to enable detection of errors before deployment. Third, and finally, the organization lacks a rapid rollback mechanism that can restore the original state of an asset if the new modified state proves problematic when deployed.

These last two factors—the lack of a simulated operating environment and the lack of a rapid rollback mechanism—should be corrected if multiple DRPs are anticipated. Cost is usually the blocking issue. However, that cost must be compared against the cost of retarding all future DRPs, and the cost of any operational failures arising from deploying faulty systems.

Continued refusal to provide a simulated operating environment with rapid rollback increases the wickedness of this and any future DDRPs.

9.   Rollback(s) of attempted changes triggers re-design

A rollback is an incident in which, in the course of executing the DRP, it has become necessary to revert some (or all) of the work that has already been performed. Minor rollbacks do happen. But a major rollback, or a rollback the necessity of which is discovered long after completion of the work in question, could be an indication of a deep misunderstanding of the consequences of the work involved. Because that misunderstanding could have consequences not yet recognized, such a rollback could suggest that the wickedness if the DDRP had been underestimated.

Let DBAf (faulty DBA) be the set of assets in the DBA that formerly contained some of the debt being retired, and which had been altered as part of the DRP, and whose alterations contained or led to exposure of some kind of fault(s) that forced a rollback after they were deployed. Let DBAfw (wicked-faulty DBA) represent the subset of DBAf for which that rollback did trigger a re-design of the DDRP. Then wickedness of the DDRP is correlated with the size of DBAfw and the extent of the DDRP re-design that the rollback triggered.

For example, let Efw be a member of DBAfw. And suppose that Efw is a modular element of a system that monitors the click-through behavior of users of a Web site. It records data for later analysis, and because of the fault it does so incorrectly. When the errors are discovered, the module is withdrawn and replaced by the original, unaltered, debt-bearing form. Because Efw contaminated the original database, data rollback is impossible. The error was discovered, but there is no way to re-capture the data that has been lost. That’s why the DDRP (and after that, possibly the DRP) must be re-designed. This scenario is an example of Criterion 5. If there are political consequences for the loss of data, it could be an example of Criterion 10.

This example suggests how the frequency of incidents that trigger re-design of the DDRP can be an indicator of the wickedness of the DDRP.

Example of a non-indicator: the I-35 SR-30 interchange near Ames, Iowa

Just outside Ames, Iowa, is an interchange between Interstate 35 (a four-lane, divided, limited-access roadway) and U.S. Route 30 (also four-lane, divided, but not limited-access). The interchange is a conventional cloverleaf design. The “leaves” are rather tight, though, and consequently, there have been numerous rollovers and crashes at this interchange. We can regard these tight cloverleaf ramps as technical debt in the highway system, and the rollovers and crashes as metaphorical interest charges on that debt.

In 2016, construction began on a new “flyover” exit ramp from northbound Interstate 35 onto westbound U.S. Route 30. The objective was to reduce the number of accidents at the interchange by replacing the current tight-curvature cloverleaf ramp with a flyover exit ramp with a longer radius of curvature. We can regard this project as a Debt Retirement Project (DRP). The project that planned that DRP was an effort to Design a Debt Retirement Project (DDRP).

Completion of the DRP was scheduled for November 2018. When completed, the new ramp will replace the northeast leaf of the cloverleaf. Like most civil engineering projects, this project does have some elements of wickedness, but they were dealt with effectively. Nevertheless, a construction error is delaying completion [Magel 2018] [Iowa DOT 2018]. The error involves the height and position of the bolt anchors where steel bridge beams will connect to the concrete piers of the new flyover ramp. Six piers have been constructed to support the flyover. Those piers are being corrected by jackhammering the concrete tops, leaving the steel reinforcement in place. Then the beam anchors are positioned correctly, and concrete re-poured. The length of the delay in completion hasn’t yet been announced.

This effort, which constitutes a rollback and re-deployment, is a significant project in itself. It requires scheduling the work to be performed, but it also requires scheduling highway lane closures and lane shifts, working around high-volume traffic periods, and possibly pouring concrete in winter conditions. And after the piers are corrected, the bridge beam placement and bridge roadbed work must proceed on a new schedule.

Consequently, the construction error triggered a redesign of the flyover project’s DRP. But it probably did not trigger a significant re-design of the DDRP. The construction error is therefore unlikely to be an indicator of significant additional wickedness for the DDRP.

Last words

You can become better managers of the risk of unanticipated wickedness. If your organization is embarking upon a long-term program of technical debt retirement, you’ll be executing many DDRPs and DRPs. Gathering data about incidents of unanticipated wickedness in DDRPs can be a useful practice, if you use that data when you design new technical debt retirement projects.

References

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Accounting for technical debt

Last updated on September 16th, 2018 at 03:28 pm

With all the talk of technical debt these days, it’s a bit puzzling why there’s so little talk in the financial community about how to go about accounting for technical debt [Conroy 2012]. Perhaps one reason for this is the social gulf that exists between the financial community and the community most keenly aware of the effects of technical debt — the technologists. But another possibility is the variety of mechanisms compelling technologists to leave technical debt in place and move on to other tasks.

Accounting for technical debt isn’t the same as measuring it
Accounting for technical debt isn’t the same as measuring it. We usually regard our accounting system as a way of measuring and tracking the financial attributes of the enterprise. As such, we think of those financial attributes as representations of money. Technical debt is different. It isn’t real, and it isn’t a representation of money — it’s a representation of resources. Money is just one of those resources. Money is required to retire technical debt, and money is consumed when we carry technical debt, but other kinds of resources are also required. Sometimes we forget that when we account for technical debt.

Here’s an example. One common form of technical debt is the kind first described by Cunningham [Cunningham 1992]. Essentially, when we complete a project, we often find that we’ve advanced our understanding of what was actually needed to accomplish our goals. And we’ve advanced our understanding to such an extent that we recognize that we should have taken a different approach. Fowler described this kind of technical debt as, “Now we know how we should have done it.” [Fowler 2009] At this point, typically, we disband the team and move on to other things, leaving the technical debt outstanding, and often, undocumented and soon to be forgotten.

A (potentially) lower-cost approach involves immediate retirement of the debt and a re-release of the asset — an “echo release” — in which the asset no longer carries the technical debt we just incurred and immediately retired. But because echo releases usually offer no immediate, evident advantage to the people and assets that interact with the asset in question, decision-makers have difficulty allocating resources to echo releases.

This problem is actually due, in part, to the effects of a shortcoming in management accounting systems. Most enterprise management accounting systems track effectively the immediate costs associated with technical debt retirement projects. They do a much less effective job of representing the effects of failing to execute echo releases, or failing to execute debt retirement projects in general. The probable cause of this deficiency is the distributed nature of the MICs — the metaphorical interest charges associated with carrying a particular technical debt. The MICs appear in multiple forms: lower productivity, increased time-to-market, loss of market share, elevated voluntary turnover in the ranks of technologists, and more (see “MICs on technical debt can be difficult to measure”). These phenomena are poorly represented in enterprise accounting systems.

Decision-makers then adopt the same bias that afflicts the accounting system. In their deliberations regarding resource allocation, they emphasize only the cost of debt retirement, often omitting from consideration altogether any mention of the cost of not retiring the debt, which can be ongoing.

If we do make long-term or intermediate-term projections of costs related to carrying technical debt or costs related to debt retirement releases, we do so in the context of a cost/benefit computation as part of the proposal to retire the debt. Methods vary from proposal to proposal. Many organizations lack a standard method for making these projections. And because there’s no standard method for estimating costs, comparing the benefits of different debt retirement proposals is difficult. This ambiguity and variability further encourages decision-makers to base their decisions solely on current costs, omitting consideration of projected future benefits.

Dealing with accounting for technical debt

Relative to technical debt, the accounting practice perhaps most notable for its absence is accounting for outstanding technical debts as liabilities. We do recognize outstanding financial debt as such, but few balance sheets — even those for internal use only — mention outstanding technical debt. Ignorance of the liabilities imposed by outstanding technical debt can cause decision-makers to believe that the enterprise has capacity and resources that it doesn’t actually have. Many of the problems associated with high levels of technical debt would be alleviated more readily if we began to track our technical debts as liabilities — even if we did so for internal purposes only.

But other shortcomings in accounting practices can create additional problems almost as severe.

Addressing the technical-debt-related shortcomings of accounting systems requires adopting enterprise-wide patterns for proposals, which gives decision-makers meaningful comparisons between different technical debt retirement options and between technical debt retirement options and development or maintenance options. One area merits focused and immediate attention: estimating MPrin and estimating MICs.

Standards for estimating MPrin are essential for estimating the cost of retiring technical debt. Likewise, standards for estimating MICs, at least in the short term, are essential for estimating the cost of not retiring technical debt. Because both MPrin and MICs can include contributions from almost any enterprise component, merely determining where to look for contributions to MPrin or MICs can be a complex task. So developing a checklist of potential contributions can help proposal writers develop a more complete and consistent picture of the MICs or MPrin associated with a technical debt. Below are three suggestions of broad areas worthy of close examination.

Revenue stream disruption

Technical debt can disrupt revenue streams either in the course of being retired, or when defects in production systems need attention. When those systems are taken out of production for repairs or testing, revenue capture might undergo short disruptions. Burdens of technical debt can extend those disruptions, or increase their frequency.

For example, a technical debt consisting of the absence of an automated test can lengthen a disruption while the system undergoes manual tests. A technical debt consisting of a misalignment between a testing environment and the production environment can allow defects in a repair or enhancement to slip through, creating new disruptions as those new defects get attended to. Even a short disruption of a high-volume revenue stream can be expensive.

Some associations between classes of technical debt and certain revenue streams can be discovered and defined in advance of any debt retirement effort. This knowledge is helpful in estimating the contributions to MICs or MPrin from revenue stream disruption.

Extended time-to-market

Although technologists are keenly aware of productivity effects of technical debt, these effects can be small compared to the costs of extended time-to-market. In the presence of outstanding technical debt, time-to-market expands not only as a result of productivity reduction, but also from resource shortages and resource contention. Extended time-to-market can lead to delays in realizing revenue potential, and persistent and  irreparable reductions in market share. To facilitate comparisons between different technical debt retirement proposals, estimates of these effects should follow standard patterns.

Data flow disruption

All data flow disruptions are not created equal. Some data flow processes can detect their own disruptions and backfill when necessary. For these flows, the main consequence that might contribute to MICs or MPrin is delay. And the most expensive of these are delays in receipt of orders, delays in processing orders, and delays in responding to anomalous conditions. Data flows that cannot detect disruptions are usually less critical, but they nevertheless have costs too. All of these consequences can be modeled and estimated, and we can develop standard packages for doing so that we can apply repeatedly to MICs or MPrin estimates for different kinds of technical debt.

Last words

Estimates of MICs or MPrin are helpful in estimating the costs of retiring technical debt. They’re also helpful in estimating the costs of not retiring technical debt. In either case, they’re only estimates, and as such, they have error bars and confidence limits. The accounting systems we now use have no error bars. That, too, is a shortcoming that must be addressed.

References

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

The Principal Principle: Focus on MICs

Last updated on December 11th, 2018 at 06:56 am

When some organizations first realize that their performance is being limited by technical debt, they begin by chartering a “technical debt inventory.” Their goals are to determine just how much technical debt they’re carrying, where it is, how much retiring it all will cost, and how fast they can retire it. That’s understandable. It’s not too different from how one would approach an out-of-control financial debt situation. Understandable, but in most cases, ineffective. With technical debt we need a different approach, because technical debt is different from financial debt. With technical debt, we must be guided by what I call the Principal Principle, which is:

The Metaphorical Principal (MPrin) of a technical debt, which is the cost of retiring it, isn’t what matters most. What usually matters most is the Metaphorical Interest Charges—the MICs.

The door to a bank vault
The door to a bank vault. One way to know that technical debt differs from financial debt is that banks aren’t involved in any way. Treating technical debt as if it had anything in common with financial debt—beyond our own sense of obligation—is a shortcut to real trouble. Remember the Principal Principle.

With technical debt, MICs can vary dramatically. For assets that aren’t being maintained or enhanced, the MICs can be Zero for extended periods. For retiring assets, their technical debt can vanish when the asset is retired. For other assets, MICs can be dramatically higher—beyond the total cost of replacing the asset.

Most people regard MICs as being restricted to productivity problems among engineers. I take a different approach. I include in MICs anything that depresses net income—lost or delayed revenue, increased expenses, anything. For instance, if technical debt causes a two-month delay in reaching a market, its effect on revenues can be substantial for years to come. I regard all of that total effect as contributing to MICs.

So the Principal Principle is that a focus on Principal can be your undoing. Focus on MICs. Drive them to Zero and keep them there.

Other posts in this thread

Technical debt use disorder

The American Psychiatric Association has identified a disorder they call Substance Use Disorder (SUD), which includes alcoholism, drug addiction, and other patterns related to substance use [APA 2013]. The research they’ve done can serve well as a model for understanding organizational behavior related to technical debt. In this post I’ll show how to use that model to describe a disorder of organizations that we could call Technical Debt Use Disorder (TDUD). It’s a pattern in which the organization can’t seem to retire much of its technical debt, or can’t stop incurring increasing burdens of technical debt, even though everyone in the organization realizes—or almost everyone realizes—that technical debt is harming the organization.

A ball and chain, with shackle
A ball and chain, with shackle.  Attaching this device to the legs of prisoners or slaves limits their ability to run, just as technical debt limits the ability of organizations to exploit new opportunities, or even to maintain their current market positions.

A brief description of the American Psychiatric Association’s publication, DSM-5®, might help explain the connection between SUD and TDUD. DSM-5 is the fifth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM). It’s used by health care professionals in the United States and much of the rest of the world as a guide to diagnosing mental disorders, and as a framework for further research. First published in 1952, the current revision, DSM-5, was released in 2013.

So what does DSM-5 have to do with technical debt?

What distinguishes responsible use of technical debt from irresponsible use is a topic that has generated many papers, conference presentations, and hallway debates over the years. Although there is consensus about the distinction in many cases, the debate continues. Sometimes, though, research in one field suggests paths forward in seemingly unrelated fields. So much thought and study has been invested in DSM-5 that it’s worth a look to see if the technical debt community can harvest something useful from the research in psychiatry.

I looked at DSM because I noticed that organizations that carry significant volumes of technical debt seem to have difficulty retiring it. In some cases, they also have difficulty halting accumulation of technical debt, or even slowing the rate of accumulation. This struck me as similar to the substance use problems some people encounter. I began to wonder whether there might be parallels between the substance use disorders that afflict people—alcoholism, drug addiction, and so on—and the technical debt problems that afflict organizations.

In the table below is one set of parallels I’ve found. The left column of the table is the list of diagnostic criteria for Substance Use Disorder provided in the DSM. The right column is my rewording of those criteria in an attempt to make them apply to how organizations deal with technical debt. I had thought initially that the rewording exercise might be difficult—that it might be a stretch. And here and there, it was a bit of a stretch. But overall, the SUD framework is a very good fit.

Diagnosing technical debt use disorder

Have a look at the table, and then check out the comments below it about how health care professions use the criteria.

DSM-5 Criteria for Substance Use Disorder (SUD)Criteria for Enterprise Technical Debt Use Disorder (TDUD)
1. Taking the substance in larger amounts or for longer than you’re meant to.
1. Incurring technical debt in larger amounts than you intended and carrying it for longer than you intended.
2. Wanting to cut down or stop using the substance but not managing to.2. Wanting to retire your technical debt or reduce the rate of incurring it but not managing to.
3. Spending a lot of time getting, using, or recovering from use of the substance.3. Spending a lot of time dealing with the consequences of the technical debt you’ve already incurred.
4. Cravings and urges to use the substance.4. Insistent demands on precious resources, causing the enterprise to incur “just a little more” technical debt.
5. Not managing to do what you should at work, home, or school because of substance use.5. Not managing to attend to the needs of existing products, services, or technological infra­structure because of the demands resulting from metaphorical interest charges on technical debt.
6. Continuing to use, even when it causes problems in relationships.6. Continuing to carry technical debt, or continuing to incur yet more technical debt, even though it causes toxic conflict among employees, and problems in customer relationships and strategic partnerships.
7. Giving up important social, occupational, or recreational activities because of substance use.7. Giving up developing important new products or services, or upgrading critical infrastructure, or pursuing new initiatives because of resource deficits traceable to technical debt service.
8. Using substances again and again, even when it puts you in danger.8. Incurring technical debt again and again, even when it puts the enterprise in fiscal danger or danger of losing market position.
9. Continuing to use, even when you know you have a physical or psychological problem that could have been caused or made worse by the substance.9. Continuing to incur technical debt, even when you know you have a fiscal or market leadership problem that could have been caused or made worse by technical debt.
10. Needing more of the substance to get the effect you want (tolerance).10. Needing to incur more technical debt to get the fiscal effect you need—a product delivered or a contract completed.
11. Development of withdrawal symptoms, which can be relieved by taking more of the substance.11. Upon attempting to retire existing technical debt, or halting incurring yet more technical debt, fiscal or market position problems develop in short order, which can be relieved only by incurring yet more debt.

In health care, two or three symptoms indicate a mild substance use disorder; four or five symptoms indicate a moderate substance use disorder, and six or more symptoms indicate a severe substance use disorder. Have a look at the right-hand column. How would you score your organization? Can we categorize the severity of an organization’s problem with technical debt using a scale similar to the one health care professionals use for SUD?

Conclusion

Technical debt isn’t inherently evil. Its existence among technological assets isn’t proof of engineering malpractice. For example, we can decide—responsibly—to deliver a system that carries technical debt, provided that we’ve carefully weighed the consequences of incurring that debt against the consequences of delayed delivery, and provided that we have a workable plan for retiring that debt, or for carrying the burden of its MICs.

But organizations can nevertheless become trapped in a cycle of technical debt, unable to make much progress in reducing it. In some cases, business as usual won’t work. Drastic action is required.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

Using SMART goals for technical debt reduction

Last updated on December 24th, 2018 at 01:42 pm

Attempting to reduce the enterprise burden of technical debt by setting so-called “SMART goals” in the obvious way can often produce disappointing results. SMART, due to George T. Doran [Doran 1981], is a widely used approach for expressing management goals. “SMART” is an acronym for “Specific, Measurable, Attainable, Realistic, and Time-boxed,” though the last three words (the “ART”) are chosen in various alternative ways. Doran himself used “assignable, realistic, and time-related.”

Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives
Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives. Watch a 2010 interview of Prof. Doran at YouTube.
SMART is so embedded in management culture that many assume without investigation that expressing technical debt reduction goals directly using the SMART pattern will produce the desired results. Also embedded in management culture is the aphorism, “You get what you measure.” [Ariely 2010]  [Bouwers 2010] Combining these two ideas in a straightforward way, one might express the technical debt reduction goal as, “Reduce the burden of technical debt by 20% per year for each of the next five years.”

There is ample support for a claim that this “direct” approach to applying the SMART technique will be ineffective. The fundamental issue is that so much of employee behavior affects technical debt indirectly that it overwhelms the effects of employee behaviors that affect technical debt directly. The result is that although the direct approach does cause some employees to adopt desirable behaviors, their impact is not significant enough compared to the effects of the behaviors of employees who see little connection between their own activities and the burden of technical debt, or who are subject to competing constraints on their behaviors that then cause them to act in ways that increase technical debt.

That’s why it’s necessary for management to develop a series of SMART goals that affect behaviors that have indirect effects on technical debt. In the first part of this post, “Setting a direct SMART goal for technical debt reduction is problematic,” I explore the problems inherent in the direct approach. In the second part, “How to set SMART goals for technical debt,” I provide examples of SMART goals that touch on behaviors that have indirect effects on technical debt.

Setting a direct SMART goal for technical debt reduction is problematic

Let’s begin by exploring some of the problems with the direct approach. In this section, I assume that management has set a SMART goal for the enterprise in the form, “Reduce the burden of technical debt by 20% per year for each of the next five years.” But there’s nothing special about the numbers. My comments below apply to the form of the goal, rather than the specific numbers.

The direct approach assumes measurability

To attain a goal of a 20% reduction in technical debt in a given year, we must be able to measure the level of technical debt at the beginning of the year and the level at the end of the year, presumably with confidence in the 90% range or better. Such a measurement with the precision required might not be possible. Moreover, in most cases the probability that such a measurement is possible is low. For these reasons, periodic goals for total technical debt is not a useful management tool.

Consider a simple example. One form of technical debt—and it’s a common form—is missing or incompletely implemented capability. In some instances, the metaphorical principal (MPrin) of a given instance of this debt in the current year can change spontaneously to a dramatically larger value in the following year (or even the following week), due to changes in the underlying asset unrelated to the technical debt, or due to debt contagion, or due to any number of other reasons. When this happens, the technical debt retirement effort for that year can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well.

The direct approach assumes a static principal

With most financial debts, the principal amount is specified at the time of loan origination. Moreover, we can compute the principal at any time given the mathematical formulas specified in the loan agreement.

By contrast, in many cases, the metaphorical principal amount of a technical debt might be neither fixed nor readily computable. We can estimate the MPrin of a given kind of technical debt at a given time, and we can even make forward projections of those estimates. But they are merely estimates, and their error bars can be enormous. See “Policy implications of the properties of MPrin” and “Useful projections of MPrin might not be attainable.”

The direct approach focuses on MPrin, not MICs

Objectives expressed in terms of the volume of technical debt—the total MPrin—are at risk of missing the point. Total MPrin is not what matters most. What matters is MICs—the total cost of carrying the debt. Even more important is the timing of arrival of the MICs.

And like MPrin, MICs can vary in wild and unpredictable ways. For example, the MICs for a piece of technical debt borne by an asset that isn’t undergoing maintenance or enhancement can be zero; in a later time period, when that asset is undergoing enhancement, the MICs can be very high indeed. See  “MICs on technical debt can be unpredictable” for a detailed discussion.

Priority setting for technical debt retirement is most effective when it takes into account the timing of MICs. For example, if we know that we must enhance a particular asset by FY21 Q3, and if we know that it bears technical debt that adds to the cost of the enhancement, retiring that debt in FY20 would be advisable. On the other hand, if that form of technical debt has no effect on MICs for the foreseeable future, retiring that technical debt might not be worth the effort.

The direct approach fails to distinguish legacy technical debt from incremental technical debt

Unless policies are already in place governing the formation of new technical debt—what I call incremental technical debt—technical debt retirement programs might encounter severe difficulty meeting their goals. The technical debt retirement program might simply be unable to keep up with the formation of new technical debt resulting from new development or from ongoing maintenance and enhancement of existing assets.

The direct approach fails to anticipate the formation of enterprise-exogenous technical debt

Technical debt can sometimes form as a result of innovations, changes in standards, or changes in regulations that occur entirely external to the enterprise. I call such technical debt enterprise-exogenous. When this happens, the technical debt retirement effort can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well. Before initiating a technical debt reduction program, it’s wise to first deploy a program that’s capable of retiring technical debt at a pace that at least equals the pace of formation of enterprise-exogenous technical debt.

Incurring technical debt is sometimes the responsible thing to do

At times, incurring technical debt is prudent. In some situations, accepting the debt you’ve incurred—even for the long term—might be called for. Because strict goals about total technical debt can lead to reluctance to incur debt that has a legitimate  business purpose, whatever goals are set for total technical debt must be nuanced enough to deal with these situations. Goals for total technical debt that adhere strictly to the SMART goal pattern sometimes lack the necessary level of nuance.

How to set SMART goals for technical debt

SMART goals can work for technical debt management, but we must express them in ways that are more closely related to behavioral choices. Here are some examples of SMART goals that can be effective elements of the technical debt management program. Some of these examples are admittedly incomplete. For example,  I offer no proof of assignability, attainability, or realism, because they can vary from organization to organization, or because the goal in question must be distributed across multiple organizational elements in ways peculiar to the organization.

At least 30% of incremental technical debt will be secured technical debt at the end of Year 1; 60% by the end of Year 2

Incremental technical debt is technical debt that’s incurred in the course of work currently underway or just recently completed. Because it’s so well understood, its MPrin can be estimated with higher precision than is usually possible with legacy technical debt. That precision is needed for defining the collateral and resources used to secure the debt.

A secured technical debt, like a secured financial debt, is one for which the enterprise reserves the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. Beyond financial resources, they might include particular staff, equipment, test beds, and downtime. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes.

Within one year, at least 50% of all incremental technical debt will be retired within one year of its origination; 70% within 18 months

This goal also exploits the fact that incremental technical debt can be estimated with relatively high precision. As a goal, it builds on the goal above by requiring that the resources pledged to retire incremental debts actually be expended as intended.

Within one year, all engineers and their direct supervisors will be educated in basic technical debt concepts

The educational materials will be developed in the next five months and piloted with 10% of the technical staff within seven months. The material will include an online proficiency test that 90% of engineers will have successfully passed within a year.

Within one year, 90% of project plans will include projections of the MPrin of the incremental technical debt they expect to generate for each delivery cycle

This information is useful for making forward projections of resources needed to secure incremental technical debt. Tracking the accuracy of these projections helps project planners improve their estimates.

Within one year, initiate a practice of identifying the top five forms of legacy technical debt, ranked by the volume of the contagion

Debt contagion is the propagation of a given form of technical debt by creating new system elements or assets in forms compatible with elements already identified as technical debt. By examining the body of incremental technical debt created enterprise-wide in a given time period (say, by fiscal quarter), we can determine the portion of that incremental debt that results from contagion, for each type of contagious legacy technical debt. This data is needed to identify the most contagious forms of legacy technical debt. They are prime candidates for debt retirement.

Within one year, initiate an industrial intelligence practice that is responsible for projecting the formation of enterprise-exogenous technical debt

This group must have a sophisticated grasp of the technologies in use within the enterprise that already bear enterprise-exogenous technical debt, or which could be subject to the formation of enterprise-exogenous technical debt. Its responsibilities cover enterprise products and services, as well as enterprise infrastructure. It issues advisories as needed, and an annual forecast. The group is also responsible for recommending and monitoring participation in industrial standards organizations. The group reports to the CIO or CTO.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

Malfeasance can be a source of technical debt

Last updated on April 29th, 2018 at 06:36 am

Although creating and deploying policies to manage technical debt is a necessary step, it isn’t sufficient for achieving control in every case. Even if training and communication programs are effective, the possibility of intentional circumvention of technical debt management policy remains. Malfeasance can lead to incurring new technical debt by circumventing any policy. And malfeasance can be an obstacle to retiring—or even identifying—existing technical debt. Moreover, indirect effects of forms of malfeasance seemingly unrelated to technical debt can nevertheless incur technical debt or extend the lifetime of existing technical debt.

Examples of malfeasance as a source of technical debt

Elizabeth Holmes Backstage at TechCrunch Disrupt San Francisco 2014
Elizabeth Holmes Backstage at TechCrunch Disrupt San Francisco 2014. She is the founder, chairman, and CEO of Theranos, a startup that grew to a total valuation of $9 billion in 2015, and has since dramatically declined in value, now on the edge of its second bankruptcy. Theranos, through Holmes, claimed to have developed a technology that enabled blood testing with small amounts of blood—0.1% to 1% of the amount of blood required by conventional blood testing technologies. These claims proved false. After a series of collisions with U.S. government agencies, the U.S. Securities and Exchange Commission sued Holmes and Theranos. In March 2018, a settlement was reached in which Holmes accepted severe financial penalties, loss of voting control of Theranos, and a ban from serving as an officer or director of any public company for ten years. Photo (cc) Max Morse for TechCrunch.

Consider an example from software engineering. To save time, an engineer might intentionally choose to use a deprecated approach. When the malfeasance is discovered, one question naturally arises: in what other places and on what other projects has this individual (or other individuals) been making such choices? In a conventional approach to controlling this form of technical debt, we might be concerned only with the engineer’s current assignment. But a more comprehensive investigation might uncover a trail of technical debt artifacts in the engineer’s previous assignments.

Allman relates a hardware-oriented example [Allman 2012]. He describes an incident involving the University of California at Berkeley’s CalMail system, which failed catastrophically in November 2011, when one disk in a RAID (Redundant Array of Inexpensive Disks) failed due to deferred maintenance. Allman regards this incident as traceable to the technical debt consisting of the deferred RAID maintenance. While this particular case isn’t an example of malfeasance, it’s reasonable to suppose that decisions to defer technical maintenance on complex systems frequently are arguably negligent.

History provides us with many clear examples of malfeasance leading to technical debt indirectly. Consider the Brooklyn Bridge. Many of the suspension cables of the bridge contain substandard steel wire, which was provided to the bridge constructors by an unscrupulous manufacturer. When the bridge engineer discovered the malfeasance, he recognized that the faulty wire that had already been installed could not be removed, or even inspected. So he compensated for the faulty wire by adding additional strands to the affected cables. For more, see “Non-technical precursors of non-strategic technical debt.”

What kinds of malfeasance deserve special attention and why

Malfeasance that leads to incurring technical debt or which extends the life of existing technical debt has the potential to expose the enterprise to uncontrolled increases in operating expenses and unknown obstacles to revenue generation. The upward pressures on operating expenses derive from the MICs associated with technical debt. Although MICs can include obstacles to revenue generation, considering these obstacles separately helps to clarify of the effects of malfeasance.

Malfeasance deserves special attention because the financial harm to the enterprise can dramatically exceed the financial benefit the malfeasance confers on its perpetrators. This property of technical-debt-related malfeasance is what makes its correction, detection, and prevention so important.

For example, when hiring engineers, some candidates claim to have capabilities and experience that they actually don’t have. Once they’re on board, they expose the enterprise to the risk of technical debt creation through substandard work that can escape notice for indefinite periods. The malfeasance here consists of the candidate’s misrepresentation of his or her capabilities. Although the candidate, once hired, does receive some benefit arising from the malfeasance, the harm to the enterprise can exceed that benefit by orders of magnitude.

As a second example, consider the behavior of organizational psychopaths [Babiak 2007] [Morse 2004]. Organizational psychopathy can be a dominant contributing factor to technical debt formation when the primary beneficiary of a proposed strategy is the decision-maker or the advocate who takes credit for the short term effects of the decision, and when he or she intends knowingly to move on to a new position or to employment elsewhere before the true long term cost of the technical debt becomes evident. This behavior is malfeasance of the highest order. And although it’s rare, its impact can be severe. For more, see “Organizational psychopathy: career advancement by surfing the debt tsunami.”

What’s required to control malfeasance

When a particular kind of malfeasance can incur technical debt or extend the life of existing technical debt, it merits special attention. Examples like those above suggest three attributes that technical debt management programs must have if they are to deal effectively with malfeasance.

Corrective measures

Corrective measures can be undertaken in a straightforward manner when inadvertent policy violations occur. For example, unexpected difficulties in setting priorities for technical debt retirement efforts might be the result of individual performance metrics that conflict with the technical debt control program. Such conflicts can be inadvertent and can be resolved collaboratively.

But with regard to malfeasance, difficulties arise when policy violations are discovered or reported. When the violations are intentional, corrective action usually entails investigation of the means by which the infraction was achieved, and the means by which it was concealed. When these activities involve many individuals attached to multiple business units, some means of allocating the cost of corrective action might be needed. Allocating the cost of corrections can also be difficult when one party has reaped extraordinary benefits by taking steps that led to incurring significant technical debt. In some cases, corrective measures might include punitive actions directed at individuals.

Detection measures

When intentional violations are covert, or those who committed the violations claim that they’re unintentional, investigation is necessary to determine whether or not a pattern of violations exists. Technical debt forensic activities require resources, including rigorous audits and robust record-keeping regarding the decisions that led to the formation or persistence of technical debt. Automated detection techniques might be necessary to control the cost of detection efforts, and to ensure reliable detection.

Preventative measures

Successful prevention of policy violations requires education, communication, and effective enforcement. The basis of effective policy violation prevention programs includes widespread understanding of the technical debt concept and the technical debt management policies, and the certainty of discovery of intentional infractions. These factors require commitment and continuing investment.

Policy frameworks are at risk of depressed effectiveness if they pay too little attention to malfeasance and other forms of misconduct. Such misbehavior deserves special attention because it’s often accompanied both by attempts to conceal any resulting technical debt, and attempts to mislead investigators and managers about its existence. These situations do arise, though rarely, and when they do, they must be addressed in policy terms.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

Order from Amazon

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

How budget depletion leads to technical debt

Some projects undergo budget depletion exercises when their budgets are reduced, or when there’s evidence that the funds remaining will be insufficient to complete the work originally planned. Formats vary, but the typical goal of these exercises is downscoping — removing, relaxing, deferring, or suspending some requirements. Since funds are limited, downscoping is often executed in a manner that leads to technical debt.

The Old River Control Complex on the Mississippi River
The Old River Control Complex on the Mississippi River. Built and operated by the US Army Corps of Engineers (USACE) the Old River Control Complex is used for controlling the flow from the Mississippi into a distributary known as the Atchafalaya River. Were it not for this facility, the Mississippi would long ago have rerouted itself into the Atchafalaya, which has a much steeper gradient to the ocean. Since that change would have deprived New Orleans and all the industrial facilities along the lower Mississippi of access to the water and navigational channels they now enjoy, USACE maintains a complex of flow control facilities to prevent Nature taking its course, and to prevent flooding along the lower Mississippi.
The industrial facilities of the lower Mississippi constitute a technical debt, in the sense that their existence is no longer compatible with the “update” Nature is trying to deploy, in the form of rerouting the flow of water from the Mississippi to the Atchafalaya. Because our national budget doesn’t allow for repositioning the city of New Orleans and all the industrial facilities from the lower Mississippi to somewhere along the Atchafalaya, we’ve elected to redirect the flow of water from the course Nature prefers to a course more compatible with the existing industrial base. Operating and maintaining the Old River Control Complex, together with a multitude of levees, dredging projects, and gates throughout lower Louisiana, are the MICs we pay for the technical debt that is the outdated position of the city of New Orleans and its associated industrial base.
For more about Atchafalaya, see the famous New Yorker article by John McPhee [MacFee 1987]. Photo courtesy USACE
Here’s an illustrative scenario. At the time downscoping begins, the work product might contain incomplete implementations of items that are to be removed from the list of objectives. A most insidious type of debt, and most difficult to detect, consists of accommodations contained in surviving artifacts that are no longer needed because the items they were intended to support have been cancelled. This class of technical debt is difficult to detect because the affected system components appear to be merely overly complicated. Recognizing it as a residual of a cancelled capability requires knowledge of its history. Unless these artifacts are documented at the time of the downscoping, that knowledge may be lost.

Other items of technical debt that arise from budget depletion include tests that no longer need to be executed, or documentation that’s no longer consistent with the rest of the work product, or user interface artifacts no longer needed. When budgets become sufficiently tight, funds aren’t available for documenting these items of technical debt as debt, and the enterprise can lose track of them when team members move on or are reassigned.

In some instances, budget depletion takes effect even before the work begins. This happens, for example, when project champions unwittingly underestimate costs in order to obtain approval for the work they have in mind. The unreasonableness of the budget becomes clear soon after the budget is approved, and the effects described above take hold soon thereafter.

Budget depletion can also have some of the same effects as schedule pressure. When the team devises the downscoping plan, it must make choices about what to include in the revised project objectives. In some cases, the desire to include some work can bias estimates of the effort required to execute it. If the team underestimates the work involved, the result is increased pressure to perform that work. With increased pressure comes technical debt. See “With all deliberate urgency” for more.

A policy that could limit technical debt formation in response to budget depletion would require identifying the technical debt such action creates, and later retiring that debt. Because these actions do require resources, they consume some of the savings that were supposed to accrue from downscoping. In some cases, they could consume that amount in its entirety, or more. Most decision-makers probably over-estimate the effectiveness of the downscoping strategy. Often, it simply reduces current expenses by trading them for increased technical debt, which raises future expenses and decreases opportunities in future periods.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

Order from Amazon

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

Other posts in this thread

Feature bias: unbalanced concern for capability vs. sustainability

Enterprise decision-makers affected by feature bias tend to harbor distorted views of the importance of new capability development compared to technical debt management. This tendency is likely due to the customer’s relative sensitivity to features, and relative lack of awareness of sustainability. Whatever the cause, customers tend to be more attracted to features than they are to indicators of sound technical debt management and other product sustainability practices. This tendency puts decision-makers at risk of feature bias: unbalanced concern for capability vs. sustainability.

Alaska crude oil production 1990-2015
Alaska crude oil production 1990-2015. This chart [Yen 2015] displays Alaska crude oil produced and shipped through the Trans Alaska Pipeline System (TAPS) from 1990 to 2015. Production had dropped by 75% in that period, and the decline is projected to continue. In January 2018, in response to pressure from Alaskan government officials and the energy industry, the U.S. Congress passed legislation that opened the Arctic National Wildlife Refuge to oil exploration, despite the threat to ecological sustainability that exploration poses. If we regard TAPS as a feature of the U.S. energy production system, we can view its excess capacity as a source of feature bias bias, creating pressure on decision-makers to add features to the U.S. energy system instead of acting to enhance the sustainability of Alaskan and global environmental systems [Wight 2017].
Changes in cost accounting could mitigate some of this feature bias by projecting more accurately total MICs based on historical data and sound estimation. I’ll explore possible accounting changes later in this post, and in future posts; meanwhile, let’s explore the causes and consequences of the distorted perspective I call feature bias.

For products or services offered for sale outside the enterprise, the sales and marketing functions of the enterprise represent the voice of the customer [Gaskin 1991]. But customers are generally unaware of product or service attributes that determine maintainability, extensibility, or cybersecurity — all factors that affect the MICs for technical debt. On the other hand, customers are acutely aware of capabilities — or missing or defective capabilities — in products or services. Customer comments and requests, therefore, are unbalanced in favor of capabilities as compared to maintainability, extensibility, cybersecurity, and other attributes related to sustainability. The sales and marketing functions tend to accurately transmit this unbalanced perspective to decision-makers and technologists.

An analogous mechanism prevails with respect to infrastructure and the internal customers of that infrastructure. Internal customers tend to be more concerned with capabilities — and missing capabilities — than they are with sustainability of the processes and systems that deliver those capabilities. Thus, pressure from internal customers on the developers and maintainers of infrastructure elements tends to emphasize capability at the expense of sustainability. The result of this imbalance is pressure to allocate excessive resources to capability enhancement, compared to activities that improve maintainability, extensibility, or cybersecurity, and which therefore would aid in controlling or reducing technical debt and its MICs.

Nor is this the only consequence of feature bias. It provides unrelenting pressure for increasing numbers of features, despite the threats to architectural coherence and overall usability that such “featuritis” or “featurism” present. Featurism leads, ultimately, to feature bloat, and to difficulties for users, who can’t find what they need among the clutter of features that are often too numerous to document. For example, in Microsoft Word, many users are unaware that Shift+F5 moves the insertion point and cursor to the point in the active document that was last edited, even if the document has just been freshly loaded into Word. Useful, but obscure.

Feature bias, it must be noted, is subject to biases itself. The existing array of features appeals to a certain subset of all potential customers. Because it is that subset that’s most likely to request repair of existing features, or to suggest additional features, the pressure for features tends to be biased in favor of the needs of the most vociferous segments of the existing user base. That is, systems experience pressure to evolve to better meet the needs of existing users, in preference to meeting the needs of other stakeholders or potential stakeholders who might be even more important to the enterprise than are the existing users. This bias in feature bias presents another risk that can affect decision-makers.

Organizations can take steps to mitigate the risk of feature bias. An example of such a measure might be the use of focus groups to study how education in sustainability issues affects customers’ perspectives relative to feature bias. Educating decision makers about feature bias can also reduce this risk.

At the enterprise scale, awareness of feature bias would be helpful, but awareness alone is unlikely to counter its detrimental effects, which include underfunding of technical debt management efforts. Eliminating the source of feature bias is extraordinarily difficult, because customers and potential customers aren’t subject to enterprise policy. Feature bias and feature bias bias are therefore givens. To mitigate the effects of feature bias, we must adopt policies that compel decision-makers to consider the need to deal with technical debt. One possible corrective action might be improvement of accounting practices for MICs, based, in part, on historical data. For example, since there’s a high probability that any project might produce new technical debt, it might be prudent to fund the retirement of that debt, in the form of reserves, when we fund the project. And if we know that a project has encountered some newly recognized form of technical debt, it might be prudent to reserve resources to retire that debt as soon as possible. Ideas such as these can rationalize resource allocations with respect to technical debt.

These two examples illustrate what’s necessary if we want to mitigate the effects of feature bias. They also illustrate just how difficult such a task will be.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

Order from Amazon

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Where the misunderstandings about MICs come from

Last updated on December 21st, 2017 at 04:27 pm

The differences between technical debt and financial debt are numerous and significant, and often overlooked, in part, because of the metaphor itself. Attempting to manage technical debt as one would manage financial debt is risky for two reasons. First, such an approach would most likely fail to exploit properties of technical debt that can reduce the costs of both carrying and retiring technical debt. More important are the opportunities lost or unrecognized because of reticence about addressing the technical debt to the extent necessary if we were to exploit those opportunities.

The right tool for the wrong job
Managing technical debt by using approaches that work well for financial debt is analogous to using the right tool for the wrong job.

The debt metaphor itself is probably at the root of the misalignment between the conventional concept of fixed or slowly varying interest rates and the reality of loss of enterprise agility or lost revenue due to technical debt. For the more familiar kinds of financial debts, the interest rate and any rules for adjusting it are set at the time of loan origination. Moreover, financial debts are unitary in the sense that each loan is a single point transaction with a single interest rate formula. For example, the interest rate formula for the most common kind of credit card balance is the same for every purchase. Technical debt isn’t unitary — each kind of technical debt and each manifestation of that kind of technical debt is, in effect, a separate loan that can carry its own independently variable MICs.

The cost of carrying technical debt can vary with time. It can vary for a given class of technical debt, or it can vary instance-by-instance. Costs depend on the nature of the work undertaken on the assets that carry the debt. This fact is a source of flexibility useful for planning technical debt management programs, which can exploit it to set priorities for debt retirement and debt prevention efforts. That flexibility implies, for instance, that planning technical debt retirement programs to satisfy the urge to retire all instances of a given class of technical debt might not be sensible.

When making technical debt management decisions, respect the constraints that technical debt imposes. Exploit the flexibility that technical debt provides.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

Order from Amazon

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Related posts

MICs can change when other debts are retired

Last updated on December 22nd, 2017 at 12:17 pm

The metaphorical interest charges (MICs) and metaphorical principal (MPrin) of a particular class of technical debt can change as a result of retiring other seemingly unrelated classes of technical debt. In most cases, engineering expertise is required to determine technical debt retirement strategies that can exploit this property of some kinds of technical debt.

Financial debts usually have associated interest rates that are used to compute the periodic interest charges. Typically, the interest charge on a financial debt for a given period is the periodic interest rate multiplied by the principal, and then scaled for the length of the time period.

But there are no “rates” for technical debt. Their existence would imply that MICs were proportional to the analog of “principal,” which, in the case of technical debt, is the cost of retiring the debt — the MPrin. MICs depend only weakly on the cost of retiring the debt. Instead, they depend more strongly on the impact of the debt on ongoing operations.

Decision-makers who understand the world of financial instruments at a very sophisticated level might tend to overvalue arguments favoring technical debt management in ways analogous to the ways we manage financial debts. Financial sophisticates might find appealing any argument for a technical debt management program that parallels financial approaches. Such programs are unlikely to work, for two reasons. First, as we’ve already noted, the uncertainties associated with estimating MPrin and MICs make technical debt management decisions more dependent on engineering and project management judgment than they are on the results of calculations and projections (see MPrin uncertainties and MICs uncertainties).

Second, as noted above, the familiar concept of interest rate is inapplicable to technical debt, because the MICs depend on the degree of interaction between ongoing activities and the debt itself, rather than the cost of retiring the debt. And that means that MICs (and MPrin) of one class of debt can change when another class is retired.

Implications of this effect

The possibility that retiring one class of technical debt can alter the financial burdens presented by another class of technical debt has both favorable and unfavorable implications.

MICs can change when other debts are retired
An example illustrating one way in which MICs on one kind of technical debt  can change as a result of retiring a different kind of technical debt. The structure at the left represents the situation before any debt retirement occurs. The balloons labeled “A” represent instances of asset A. The balloon labeled “B” represents asset B. The orange circles represent instances of technical debt D1 and D2, respectively. The arrows connecting the As to B indicate that asset A depends on Asset B. The structure at the right represents the situation after debt retirement.

As an example of a favorable implication, consider software remodularization. Suppose we have a software asset A that depends on another software asset B. As shown in the left image of the figure, asset A, of which there are many copies, bears two classes of technical debt, D1 and D2. As shown, there is only one copy of asset B. Suppose further that an asset that bears debt D2 also bears debt D1, but an asset that bears D1 might or might not bear debt D2.

To retire D2, engineers have decided to modify B by having it assume responsibility for the tasks that formerly bore debt type D2. They do this even though, as a consequence of this change, B will now bear debt of type D1. Next, debt type D2 is retired. The right half of the figure shows the resulting implementation. The system still bears debt D1, but now it’s located in B instead of A. All instances of type A assets change, and those modifications relieve them of both types of debt. This is a sensible approach, because there are several assets of type A and only one of type B. The end result is that D2 vanishes, and only a single instance of D1 remains. In this way, retiring debt D2 has reduced the MICs and MPrin for D1.

Policymakers can help

Exploiting the salutary opportunities of this property of technical debt provides an example of the risks of adhering too closely to the financial model of debt.

Many different scenarios have the property that retiring one kind of technical debt can reduce the MICs associated with other kinds of debt. Because technologists understandably tend to be more concerned with technical debt retirement strategies that emphasize short-term improvement of their own productivity, policymakers can provide guidance that steers the organization in the direction of enterprise benefits.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Allman 2012] Eric Allman. “Managing Technical Debt: Shortcuts that save money and time today can cost you down the road,” ACM Queue, 10:3, March 23, 2012.

Available: here; Retrieved: March 16, 2017

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Babiak 2007] Paul Babiak and Robert D. Hare. Snakes in Suits: When Psychopaths Go to Work. New York: HarperCollins, 2007. ISBN:978-0-06-114789-0

Order from Amazon

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Iowa DOT 2016] “Construction drawing for the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation, February 2, 2016.

Available: here; Retrieved: October 31, 2018

Cited in:

[Iowa DOT 2018] “Work Continues on the Northbound I-35 Flyover Ramp at U.S. 30 Near Ames,” Iowa Department of Transportation News Release, June 27, 2018.

Available: here; Retrieved: October 31, 2018

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[McIntosh 2003] Shane McIntosh, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. “The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects,” in Proceedings of the 11th Working Conference on Mining Software Repositories, Premkumar Devanbu, ed., New York: ACM, 192-201

Cited in:

[Morse 2004] Gardiner Morse. “Executive psychopaths,” Harvard Business Review, 82:10, 20-22, 2004.

Available: here; Retrieved: April 25, 2018

Cited in:

[Rittel 1973] Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”, Policy Sciences 4, 1973, 155-169.

Available: here; Retrieved: October 16, 2018

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Related posts