Technical debt smell

Last updated on July 7th, 2021 at 05:16 pm

“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872)
“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872). Photo courtesy Wikimedia Commons.
Uncas then deduces amazing details about the man who left the track he examined—150 years ahead of Sherlock Holmes. Natty Bumppo, Cooper’s Hawkeye character, calls the signs Uncas uses tell-tales.

Technical debt smell would be a useful indicator of the presence of a severe problem of technical debt in an organization. Unfortunately, technical debt doesn’t usually have a smell, as such. But metaphorically, it might have indirect indicators, just as we might say, “I smell a rat” when probing a mystery. Actually, the idea of indirect indicators has a long and storied tradition. In one scene midway through James Fenimore Cooper’s The Last of the Mohicans, Uncas (who is the actual last of the Mohicans) demonstrates his tracking skills [Cooper 1857]:

The young Mohican bent over the track, and removing the scattered leaves from around the place, he examined it with much of that sort of scrutiny that a money-dealer, in these days of pecuniary doubts, would bestow on a suspected due-bill. At length he arose from his knees, satisfied with the result of the examination.

Tell-tales in Nature

Nature abounds with examples of such skill at noticing tell-tales. Lions, tigers, bears, and all sorts of fauna use their olfactory senses to detect food, predators, mates, offspring, weather, and even the change of seasons. Smell gives them access to information they need, often before sight or hearing can.

That’s probably a part of why smell has become a useful metaphor in software engineering. The technical literature about code smells is vast and growing [Haque 2018]. In a blog post titled “CodeSmell,” Martin Fowler defines code smell as, “…a surface indication that usually corresponds to a deeper problem in the system.” [Fowler 2006] Code smells are traits that are easy to recognize, and often—but not always—indicators of problems.

Enter the “red flag”

The concept is also useful in the business domain, though there we use a different metaphor and a different term. In the business context, we call smells red flags. Investopedia defines a red flag as, “…an indicator of potential problems with a security, such as any undesirable characteristic that stands out to an analyst as it pertains to a company’s stock, financial statements or negative news reports.” But I’ve heard the term red flag used in the context of evaluating proposals, assessments, status reports, personnel, and intelligence of all kinds.

Whether called tell-tales, smells, red flags, or just indicators, their value is that they suggest the outlines of something we haven’t yet seen clearly enough to identify with certainty. Their principal attributes are that they’re available at the surface of the domain we’re surveying, they’re relatively cheap to obtain, and, if found, they suggest trouble, and deeper investigation might be worthwhile.

Cultural smells

In the software engineering community, technical debt is regarded as a smell that indicates trouble in the system’s software. So we might ask, “Among policymakers, what are the smells that indicate trouble in the organization?” If technical debt is the trouble we’re looking for, what are the cultural smells that indicate that technical debt might be a problem?

Said differently, can we find, or can we develop, a set of attributes of enterprise culture that indicate the degree of severity of an organization’s problems with technical debt?

Here are some possible “technical debt smells”—aspects of enterprise culture that could indicate problems with technical debt:

  • There is a general belief that technical debt afflicts some organizations, but not ours
  • We’re a new startup—just a year old—so we have no technical debt.
  • We don’t build software, therefore no technical debt
  • Nontechnical members of our executive team have little if any knowledge of the concept of technical debt
  • No enterprise resources are allocated to educating nontechnical employees about technical debt
  • The VP of Marketing doesn’t believe that anything she does could possibly contribute to technical debt
  • There is a general belief that if we have technical debt, it’s due solely to malpractice on the part of engineers
  • We’ve tried to assess the total cost of eliminating all of our technical debt. But we found the estimates so unreliable that we decided to leave well enough alone.
  • We do believe that technical debt does have costs. But because it only affects the productivity of engineers, we just hired more engineers and decided to live with it.

Last words

Clearly we could assemble a list of technical debt smells—beliefs about technical debt and behaviors that affect it—and check for their presence in a given organization. But fortunately, some of that work has already been done, albeit in a very different context; That context is a malady psychiatrists call “Substance Use Disorder.” More about that next time.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Contract restrictions can lead to technical debt

Last updated on July 8th, 2021 at 01:42 pm

When the owner of an asset, especially a software asset, contracts to provide a capability to a customer incorporating a use of that asset, contract restrictions can lead to technical debt. The work involved might require modification or enhancement of that asset. When the contract permits such work without transferring ownership of the asset itself, performing it is relatively straightforward. But complications can arise unless the contractor can perform the work in a manner compatible with any pre-existing or anticipated future other uses of the asset. Even so, some contract restrictions can cause the owner of the asset to incur technical debt.

How technical debt can enter the picture

A power adaptor/converter for international travelers with U.S. standard equipment.
A power adaptor/converter for international travelers with U.S. standard equipment. This device provides conversion for both hardware connection and voltage supplied.
We can regard the wide variation in electric power standards worldwide as a technical debt. Someday, in the probably distant future, a world standard will emerge and we will retire that debt. Until then, adaptors like these are travel necessities.
Some contracts restrict such work. For example, a government customer might require ownership of the work performed. Potentially, all of the work might be classified as a national secret. In either of these cases, to retain control of the asset, the owner/contractor arranges to perform all of the work outside the periphery of the asset. To accomplish this, the owner/contractor might interface to the asset through an adaptor that the government customer can then own, or which can be classified as secret if necessary. These moves insulate the original asset from these ownership restrictions.

The result is tolerable after completion of one such contract. But over time, as their number increases, the adaptors become a form of technical debt. The asset owner must maintain each adaptor against any changes in the original asset. Moreover, making changes to the original asset can become a project of such scale that the temptation to create a static “clone” of the asset for each customer is irresistible. When that happens, cloning replicates both the asset and any technical debt it carries. And correcting defects in the asset requires correcting that same defect in any clones that carry it.

The general forms of the problem

The problem is more general than suggested above. It also appears in the case of software that supports multiple platforms, or which is available in multiple versions supporting a single platform.

But it gets worse. Suppose the maintainers decide to update the asset to make it more extensible, or to make it more maintainable. They must then perform that update, including all testing and documentation, on each clone. If the asset owner elects not to update all clones, then the clones will begin to diverge from each other. Engineers performing tasks on one of the clones must then have knowledge of how that clone differs from other clones. If they discover a new defect, it might or might not be present in every clone. Implementing a new extension or other modification might not be possible in all clones. Or implementing it in some clones might require a unique approach. Life can get very complicated.

Organizations entering into contracts of this kind would be wise to include language limiting their obligations to maintain the original asset against any changes. Or they might include an explicit statement of the parties’ intentions relative to financial support for any continuing obligations to maintain that asset.

Last words

Organizations offering products for multiple platforms would be wise to consider as strategic the management of technical debt that arises from platform multiplicity. Sound management of this form of technical debt can extend their ability to support multiple platforms. And that can dramatically increase returns on investment in the core asset.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Technical debt and engineering resources

Last updated on July 7th, 2021 at 10:34 am

Flooding from Hurricane Katrina in New Orleans, 2005.
Flooding from Hurricane Katrina in New Orleans, 2005. The forces of Nature can overtop or undermine any levee humans can build. So it is with technology. Organizational policy and politics can overcome or undermine any technology humans can devise to attain mastery over technical debt. To master technical debt, technology isn’t enough—we must also deal with policy and politics.

Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—should create significant savings and competitive advantages. These benefits arise from reductions in metaphorical interest charges (MICs) that result from retiring technical debt. But these benefits become available only if engineering capacity increases relative to the total debt-related workload. After the technical debt management program is in place, if the balance between engineering resources and debt-related workload becomes more favorable, then organizational effectiveness can improve. But if the balance becomes less favorable, as a result of reductions in engineering resources, organizational effectiveness won’t improve, even at lower levels of technical debt.

Unfortunately, some organizations adopt advanced technical debt management practices while reducing engineering capacity. If reductions are dramatic enough, engineering effectiveness is no better than it was before initiating the technical debt management program. The reason for this is that the engineering process isn’t the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves nontechnical causes in place. That’s why technological solutions to the technical debt management problem might not produce benefits in organizational effectiveness and agility.

The focus of technical debt research has been technology

The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has appeared [Izurieta 2017]. Effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has transcended debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].

Such developments might or might not have much impact on the limiting the effects of carrying technical debt. Given the necessary resources, engineering organizations could retire much of the technical debt now extant. That is, the will and the capacity to invest in debt retirement determines debt retirement rates. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.

Technological development isn’t enough

The open question is this: will technological developments alone give us control of the problem of technical debt? Perhaps not. Advancements in technical debt management do benefit organizations. But they could use that benefit to execute reductions in engineering staffing. If they do, they could divert savings to other parts of the enterprise. That would allow technical debt to remain at reduced levels that could still compromise the effectiveness of that reduced engineering staff.

For example, research has shown that schedule pressure contributes to technical debt formation and persistence. Suppose that the engineering groups of an organization become more adept at managing and preventing technical debt. Suppose further that the organization’s marketing and sales groups don’t improve their own intelligence and planning processes. Then Marketing might demand new capabilities with ever shorter timelines. That could lead to increased schedule pressure for the engineering groups. Then the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.

Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of nontechnical causes of technical debt to persist, and possibly even to grow.

This blog focuses on the nontechnical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Case 1: A platform upgrade

Last updated on July 18th, 2021 at 07:13 pm

This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker’s view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.

Background
File servers in a rack
File servers in a rack. Photo (cc) Abigor courtesy Wikimedia Commons.

Growth at the fictional company Unbelievable Growth, Inc., (UGI) has been so unbelievable that there is now a shortage of financial resources. Resources are especially tight for migrating the last groups of SharePoint users from SharePoint 2010 to SharePoint 2013. Consequently, the CFO instructed IT to continue to support SharePoint 2010 for at least two more quarters. Meanwhile, the affected SharePoint users must continue to use SharePoint 2010. Someday, currently set for two quarters from now, IT and the users of SharePoint 2010 will migrate to SharePoint 2013. Both IT and the users might need to expend resources to keep the SharePoint 2010 site operational. Users who make enhancements to their SharePoint 2010 sites will migrate that work to the SharePoint 2013 site. Unfortunately, that might require some rework that would have been unnecessary if the migration had not been deferred.

Discussion

We can regard as a debt UGI’s decision to defer the SharePoint migration. Because it isn’t a financial obligation, we call it a technical debt. UGI must retire that technical debt two quarters from now, when they finally execute the migration from SharePoint 2010 to SharePoint 2013. We can regard the cost of the final migration as the (metaphorical) principal of the technical debt. In the meantime, IT and the users must do some work that might have been unnecessary if they could have performed the migration now. We can regard that extra work as the (metaphorical) interest charges on that technical debt.

The policymaker’s perspective

Some—indeed most—conventional views of technical debt do not regard the deferred upgrade as technical debt, for various reasons—it isn’t software, or it isn’t in a product, or it isn’t a shortcut taken for expedience, and so on. Moreover, the person who made the decision to take on the debt was the CFO, who isn’t an engineer, and who might not even realize that the implications of the decision result in taking on technical debt.

But from the viewpoint of the policymaker, the commitment to execute the upgrade in the future is equivalent to accepting a technical obligation. For the enterprise, it’s a technical debt. Following UGI’s current accounting procedures, the SharePoint users and IT will pay the metaphorical interest on that technical debt. It will appear as an operating expense for those groups. That’s unfortunate, because the purpose of deferring the upgrade was unrelated to their operations. It was an enterprise cost-leveling maneuver. As such, the enterprise should probably account for the costs at the enterprise level to ensure accurate representation of the operational costs for the SharePoint users and for IT, and to accurately represent the CFO’s operations.

Non-technical decisions, occurring anywhere in the enterprise, can sometimes lead to incurring technical debt. Enterprise policy intended to support effective technical debt management must take these phenomena into account.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

Show Buttons
Hide Buttons