MICs on technical debt can be difficult to measure

Last updated on July 3rd, 2018 at 08:02 am

For a financial debt, creditors regularly inform debtors of periodic interest charges, principal remaining, and other parameters of the loan. In many cases, laws require regular reports and explicit statements about interest charges when prospective creditors interact with prospective debtors. By contrast, for technical debt, metaphorical interest charges (MICs) can be difficult to compute with useful precision, even if we know that they’re accumulating. Many decision-makers are actually unaware that MICs are accumulating at all. For an organization to appreciate the full financial consequences of carrying technical debt, everyone in the enterprise must appreciate the concept of MICs.

A stack of floppy disks
A stack of floppy disks. You don’t see many of these around much anymore. Very little of the software or hardware we use is as obsolete as these floppies. But much of it is obsolete, and it therefore comprises technical debt. It still works, but it’s slow and probably no longer supported by its manufacturer. On the basis of speed alone, the MICs it incurs can easily justify replacement. And some of it is vulnerable to cyberattack. One significant breach can ruin a brand.

Unlike financial debt, for technical debt there are no legally required reports or disclosures. We can sometimes estimate MICs, but most organizations don’t track the data necessary to estimate MICs with useful precision. Indeed, developing useful estimates is often technically impossible.

The difficulty of measuring MICs arises from three sources. First, people whose productivity is most directly affected by technical debt — usually engineers — often have difficulty determining with precision the extent of the impact of technical debt on their efforts.

Second, many people are unaware of the impact technical debt has on their results. For example, if a product arrives late to market, the financial costs attributable to technical debt can be computed if we realize that technical debt is partially — or wholly — responsible for the delay. Too often, those who could perform such calculations aren’t sufficiently familiar with the concept of MICs, and in any case, the data they would need for calculating a useful estimate is rarely available.

Finally, a more insidious form of the consequences of technical debt is what we might call the terrifying opportunity. This situation arises when the organization rejects (or fails to recognize) a market opportunity because exploiting it would involve modifying an existing asset or product offering that harbors a heavy load of technical debt. The debt causes decision-makers to assess that the probability of success is so depressed by technical debt that the opportunity seems terrifying, and they therefore reject the opportunity. Typically, terrifying opportunities would be exploitable if the debt-bearing assets didn’t exist at all, because then we would be starting fresh. But given that terrifying opportunities require modifying existing assets that bear heavy loads of technical debt, commitment requires faith that the technical debt can be addressed successfully.

The sense of risk isn’t a reflection on the capabilities of the technical organization. Rather, it’s a result of the challenges involved in working with assets that bear high levels of technical debt. Given past performance of the technical organization relative to these debt-bearing assets, success can seem unlikely.

Computing the cost of a terrifying opportunity requires estimating the cost of not exploiting the opportunity, a difficult task in the best of circumstances. But whatever that cost is, it’s a form of MICs that we rarely recognize.

Building expertise in estimating MICs in all their forms is advantageous to any organization that seeks to make its technical debt more manageable. By making MICs visible, we can bring about better recognition of the cost of carrying technical debt, thereby providing an appropriate motivator for retiring technical debt.

Related posts

MICs can sometimes be deferred or advanced without penalty

Last updated on December 26th, 2017 at 07:57 pm

Although rescheduling interest payments on financial debts is possible only by prearrangement, by special arrangement, or in bankruptcy, MICs on technical debt can often be deferred or advanced by simply rescheduling any work that might incur them. This is possible because, for some kinds of technical debt, MICs accumulate only if we perform engineering work that’s affected by that debt. This property is especially useful when we plan to retire an asset that bears technical debt, because when it’s removed from service, the technical debt it carries vanishes.

A rehabilitated Green Line car of the Massachusetts Bay Transit Authority
A rehabilitated Green Line car of the Massachusetts Bay Transit Authority. Trolley cars still travel on surface streets in Boston, but the only active lines are in medians of divided roadways. Many streets in and around Boston still contain buried trolley tracks. They comprise a technical debt, and MICs continue to accrue in the form of broken pavement and a near-continuous need to patch roadways, due to surface decomposition from the freeze-thaw cycle and the constant small movements of the buried tracks due to traffic loads. A recent sewer upgrade project in Cambridge required removal of buried tracks to remove and replace the old sewer line. This presented an opportunity to defer street surface maintenance (MICs) to take advantage of the surface rebuilding that was included in the sewer project, though I don’t know whether that opportunity was actually exploited.

For most conventional financial debts, interest charges accumulate until the debt is retired. Interest charges might be zero for defined time periods, but they’re never negative. Failure to meet the contractual payment schedule can result in penalties and additional interest charges.

But at times, for technical debt, MICs can be deferred or advanced without penalty and without additional “interest charges.” In other words, the organization can arrange to temporarily nullify the MICs on a particular class of technical debt, or for particular instances of that class, by simply rescheduling a project or projects. This is possible when the nature of the debt is such that MICs accrue only if there is a need to perform work on assets that are affected by the debt in question. In a given fiscal period, if no work is performed on those assets, the MICs can be zero. By scheduling projects accordingly, organizations can arrange for MICs to be zero.

There is one caveat. As discussed in “How technical debt can create more technical debt,” as long as a particular class of technical debt remains in place, its associated MPrin might increase. Deferring retirement of a class of technical debt is wise only if its associated MPrin is controlled or if projected changes in its MPrin are acceptable.

Related posts

Balance technical debt and engineering resources

Last updated on December 11th, 2018 at 09:42 am

Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—should create significant savings, and many competitive advantages. These benefits should arise from the reductions in metaphorical interest charges that result from retiring technical debt. But these benefits become available to the organization only if engineering capacity increases relative to the burdens presented by the remaining reduced levels of technical debt. After the technical debt management program is in place, if the balance between engineering resources and the burdens imposed by the remaining technical debt becomes more favorable, then organizational effectiveness will improve. But if the balance becomes less favorable, as a result of reductions in engineering resources,  organizational effectiveness won’t improve, even at lower levels of technical debt.

Flooding from Hurricane Katrina in New Orleans, 2005.
Flooding from Hurricane Katrina in New Orleans, 2005. Any levee humans can  build can be overtopped or undermined by the forces of Nature. So it is with  technology. Any technology humans can devise to attain mastery over technical debt can be overcome or undermined by organizational policy and organizational politics. To master technical debt, technology is not enough—we must also deal with policy and politics.

Unfortunately, it’s possible to adopt advanced technical debt management practices while at the same time reducing engineering capacity to a level such that engineering effectiveness is no better than it was before the technical debt management program was initiated. The reason for this is that the engineering process is not the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves non-technical causes in place. That’s why technological solutions to the technical debt management problem might not be sufficient to produce benefits in organizational effectiveness and agility.

The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has been developed [Izurieta 2017]. It’s reasonable to assume that effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has not been confined to debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].

But it’s also reasonable to ask whether such developments will have much impact on the limiting effects of carrying technical debt, even in software. Given the necessary resources, much of the technical debt now extant could be retired. That is, debt retirement rates are determined only by the will and the capacity to invest in debt retirement. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.

The open question is this: will technological developments alone suffice to gain control of the problem of technical debt? Perhaps not. Organizations could exploit the advancements in technical debt management to execute reductions in engineering staffing—and therefore cost—while they divert savings to other parts of the enterprise, thereby allowing technical debt to remain at levels that, although much reduced, are nevertheless sufficient to compromise the effectiveness of that reduced engineering staff.

For example, schedule pressure is widely recognized as contributing to technical debt formation and persistence. If engineering groups become more adept at managing and preventing technical debt, but marketing and sales groups do not improve their own intelligence and planning processes and consequently demand new capabilities with even shorter timelines than they now do, the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.

Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of non-technical causes of technical debt to persist, and possibly even to grow.

This blog focuses on the non-technical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.

References

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Technical debt in the highway system

Last updated on December 11th, 2018 at 11:26 am

The ghost ramps of highway I-695 in Somerville, Massachusetts
The ghost ramps of highway I-695 in Somerville, Massachusetts. Photo (cc) Nick Allen.

Interstate 695 was planned in 1955 as an “inner belt” circumferential route in Boston and adjacent towns. When it was cancelled in 1971, construction had already begun. Rights of way that had been cleared have since been reused for roads and mass transit, but some never-used structures remain to this day, including a “ghost ramp” in Somerville that would have connected I-695 to I-93. This ramp, which is a mere stub that begins on an elevated stretch of I-93 and ends in mid-air, and which is blocked off to prevent use, constitutes technical debt in the form of incomplete implementation. Google satellite view

For safety reasons, the ghost ramp must be regularly inspected, maintained, and insured, but it provides no utility and it is not used for any civic purpose. Because the cost of retiring this technical debt—namely, demolition costs—would likely exceed the present value of the lifetime costs of inspection, maintenance, and insurance, the ghost ramp remains.

Sometimes, the best way to deal with technical debt is to leave it in place.

References

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Case 3: Help desk operations

Last updated on November 21st, 2017 at 08:34 am

This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.

Background

A tech support person at workDuring the troubles following release of UGI’s StrawIntoGold 1.0 product, the Help Desk operated by UGI’s Customer Service Department was inundated with calls for assistance. Customer Service alerted Engineering, which provided an explanation and an estimated repair date for the customer service representatives to pass on to callers, but in the crush and panic, neither Engineering management nor Customer Service management provided a script for the representatives to use when the calls came in. Consequently, calls took approximately 15% longer to handle than they would have if a carefully worded script had been available. Further, the message conveyed to customers was not always clear or consistent, which resulted in some customers calling again with the same issue.

Discussion

The decision not to provide customer service representatives with a script can be viewed as incurring a technical debt. The extra time handling calls, the extra calls that resulted from the absence of a script, and even a few lost customers, can be viewed as the metaphorical interest charges on that technical debt. Because of the singular nature of this incident, it’s doubtful that a script will ever be written, but if it were, the cost of doing so, and the cost of distributing it and training all customer service representatives would be the metaphorical principal of this technical debt.

The policymaker’s perspective

UGI doesn’t have a means of making those who incurred the debt accountable for the metaphorical interest charges on that debt. In this case, the Customer Service function incurs additional operating expenses because the Engineering and Customer Service, together, elected not to develop a script for the customer service representatives.

Another component of the metaphorical interest charges is the total of lost sales, damage to UGI’s reputation, and possible loss of market share. Marketing could have stepped in to assist with limiting that damage, but because they viewed the problem as technical, they did not participate. A whole-enterprise perspective on managing the technical debt might have led to a collaboration between Engineering, Marketing, and Customer Support to build better relationships with the customers who were affected by the incident.

Accounting properly for the metaphorical interest charges associated with technical debt can lead to a better understanding of the effects of technical debt.

References

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Case 2: New software Development

Last updated on November 21st, 2017 at 08:35 am

This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue is not actually technical debt.

Background

A typical smartphoneOne reason why growth has been so unbelievable at Unbelievable Growth, Inc., is the recent release of their new app for Android and iPhone, StrawIntoGold 1.0, which has an uncanny ability to predict the price movements of specific common stocks over the next 60 seconds. Some users had complained about the unresponsiveness of the user interface in release 1.0, which caused them to miss the 60-second window. Engineering did some quick work and shipped StrawIntoGold 1.1 to fix the problem. In their haste, they were unable to automate all testing, and 17 interns that were hired just for that release performed some critical tests manually to ensure that the problem had been fixed. Because those manual tests have not yet been automated, and because the interns have been let go, the engineers working on StrawIntoGold 2.0 are doing their own tests manually, which is reducing their productivity considerably. Release 2.0 is now three months late, so far, and the projected ship date is at least three months from now.

Discussion

We can regard the decision to ship release 1.1 without automating all tests as incurring a technical debt consisting of the tests that were not automated.  Until that debt is retired, something analogous to interest charges are being paid, in the form of reduced engineering productivity, which raises the costs of producing the next release, and which also delays the revenue stream projected for StrawIntoGold 2.0. That causes a loss of revenue, which is another contribution to metaphorical interest charges on the outstanding technical debt. The metaphorical principal of the technical debt is the cost of implementing the automated testing facilities, including documentation and training for the engineers who will be using them.

The policymaker’s perspective

Some conventional views of technical debt do not regard the missing test automation facilities as technical debt, because they aren’t part of the product. For example, Kruchten, et. al. [Kruchten 2013], take the definition of technical debt to be restricted to items characterized as “direct system characteristics.”

But even among those who regard the missing tests as technical debt, and the depressed engineering productivity as a metaphorical interest charge on that debt, some would not regard the delayed revenue as a metaphorical interest charge.

From the policymaker’s perspective, any loss of organizational effectiveness attributable to the condition or absence of a technological artifact is potentially a metaphorical interest charge arising from the technical debt associated with that artifact.

It’s difficult for organizations to allocate resources to technical debt management unless they know the full costs associated with carrying technical debt.

References

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Kruchten 2013] Philippe Kruchten, Robert L. Nord, Ipek Ozkaya, and D. Falessi, “Technical debt: towards a crisper definition report on the 4th international workshop on managing technical debt.” ACM SIGSOFT Software Engineering Notes, 38:5, 51-54, 2013.

Includes a comment that testing debt is not technical debt. Includes a comment that technical debt is result of quick and dirty work.

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Case 1: A platform upgrade

Last updated on November 21st, 2017 at 08:35 am

This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker’s view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.

Background
File servers in a rack
File servers in a rack. Photo (cc) Abigor courtesy Wikimedia Commons.

Growth at the fictional company Unbelievable Growth, Inc. (UGI) has been so unbelievable that there is now a shortage of financial resources for migrating the last groups of SharePoint users from SharePoint 2010 to SharePoint 2013. Consequently, the CFO instructed IT to continue to support SharePoint 2010 for at least two more quarters. Meanwhile, the affected SharePoint users must continue to use SharePoint 2010. Someday, currently set for two quarters from now, IT and the users of SharePoint 2010 will be required to migrate to SharePoint 2013. Both IT and the users might need to expend resources to keep the SharePoint 2010 site operational. Users who make enhancements to their SharePoint 2010 sites will need to migrate that work to the SharePoint 2013 site, and that might require some rework that would have been unnecessary if the migration had not been deferred.

Discussion

We can regard as a debt UGI’s decision to defer the SharePoint migration. Because it isn’t a financial obligation, we call it a technical debt. UGI must retire that technical debt two quarters from now, when they finally execute the migration from SharePoint 2010 to SharePoint 2013. We can regard the cost of the final migration as the (metaphorical) principal of the technical debt. In the meantime, IT and the users must do some work that might have been unnecessary if they could have performed the migration now. We can regard that extra work as the (metaphorical) interest charges on that technical debt.

The policymaker’s perspective

Some — indeed most — conventional views of technical debt do not regard the deferred upgrade as technical debt, for various reasons: it isn’t software, or it isn’t in a product, or it isn’t a shortcut taken for expedience, and so on. Moreover, the person who made the decision to take on the debt was the CFO, who is not an engineer, and who might not even realize that the implications of the decision result in taking on technical debt.

But from the viewpoint of the policy maker, the commitment to execute the upgrade in the future is equivalent to accepting a technical obligation. For the enterprise, it is a technical debt. Following UGI’s current account procedures, the metaphorical interest on that technical debt will be paid by the SharePoint users and by IT, and it will appear as an operating expense for those groups. That’s unfortunate, because the purpose of deferring the upgrade was unrelated to their operations. It was an enterprise cost-leveling maneuver whose costs should probably be accounted for at the enterprise level to ensure that operational costs for the SharePoint users and for IT are accurately represented, and to accurately represent the CFO’s operations.

Non-technical decisions, occurring anywhere in the enterprise, can sometimes lead to incurring technical debt. Enterprise policy intended to support effective technical debt management must take these phenomena into account.

References

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Kruchten 2013] Philippe Kruchten, Robert L. Nord, Ipek Ozkaya, and D. Falessi, “Technical debt: towards a crisper definition report on the 4th international workshop on managing technical debt.” ACM SIGSOFT Software Engineering Notes, 38:5, 51-54, 2013.

Includes a comment that testing debt is not technical debt. Includes a comment that technical debt is result of quick and dirty work.

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Show Buttons
Hide Buttons