Posted on

Technical debt smell

Technical debt smell would be a useful indicator of the presence of a severe problem of technical debt in an organization. Unfortunately, technical debt doesn’t usually have a smell, as such. But speaking metaphorically, maybe it does have indirect indicators, in the same way that we might say, “I smell a rat” when probing a mystery. Actually, the idea of indirect indicators has a long and storied tradition.  In one scene midway through James Fenimore Cooper’s The Last of the Mohicans, Uncas (who is the actual last of the Mohicans) demonstrates his tracking skills [Cooper 1857]:

The young Mohican bent over the track, and removing the scattered leaves from around the place, he examined it with much of that sort of scrutiny that a money-dealer, in these days of pecuniary doubts, would bestow on a suspected due-bill. At length he arose from his knees, satisfied with the result of the examination.

“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872)
“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872). Photo courtesy Wikimedia Commons.
Uncas then deduces amazing details about the man who left the track he examined—150 years ahead of Sherlock Holmes. Natty Bumppo, Cooper’s Hawkeye character, calls the signs Uncas uses tell-tales.

Nature abounds with examples of such skill at noticing tell-tales. Lions, tigers, bears, and all sorts of fauna use their olfactory senses to detect food, predators, mates, offspring, weather, and even the change of seasons. Smell gives them access to information they need, often before sight or hearing can.

That’s probably a part of why smell has become a useful metaphor in software engineering. The technical literature about code smells is vast and growing [Haque 2018]. In a blog post titled “CodeSmell,” Martin Fowler defines code smell as, “…a surface indication that usually corresponds to a deeper problem in the system.” [Fowler 2006] Code smells are traits that are easy to recognize, and often—but not always—indicators of problems.

The concept is also useful in the business domain, though there we use a different metaphor and a different term. In the business context, we call smells red flags. Investopedia defines a red flag as, “…an indicator of potential problems with a security, such as any undesirable characteristic that stands out to an analyst as it pertains to a company’s stock, financial statements or negative news reports.” But I’ve heard the term red flag used in the context of evaluating proposals, assessments, status reports, personnel, and intelligence of all kinds.

Whether called tell-tales, smells, red flags, or just indicators, their value is that they suggest the outlines of something we haven’t yet seen clearly enough to identify with certainty. Their principal attributes are that they’re available at the surface of the domain we’re surveying, they’re relatively cheap to obtain, and, if found, they suggest trouble, and deeper investigation might be worthwhile.

In the software engineering community, technical debt is regarded as a smell that indicates trouble in the system’s software. So we might ask, “Among policymakers, what are the smells that indicate trouble in the organization?” If technical debt is the trouble we’re looking for, what are the cultural smells that indicate that technical debt might be a problem?

Said differently, can we find, or can we develop, a set of attributes of enterprise culture that indicate the degree of severity of an organization’s problems with technical debt?

Here are some possible “technical debt smells”—aspects of enterprise culture that could indicate problems with technical debt:
  • There is a general belief that technical debt afflicts some organizations, but not ours
  • We’re a new startup—just a year old—so we have no technical debt.
  • We don’t build software, therefore no technical debt
  • Non-technical members of our executive team have little if any knowledge of the concept of technical debt
  • No enterprise resources are allocated to educating non-technical employees about technical debt
  • The VP of Marketing doesn’t believe that anything she does could possibly contribute to technical debt
  • There is a general belief that if we have technical debt, it’s due solely to malpractice on the part of engineers
  • We’ve tried to assess the total cost of eliminating all of our technical debt, and found the estimates so unreliable that we decided to leave well enough alone
  • We do believe that technical debt does have costs, but because it only affects the productivity of engineers, we just hired more engineers and decided to live with it

Clearly we could assemble a list of technical debt smells—beliefs about technical debt and behaviors that affect it—and check for their presence in a given organization. But fortunately, some of that work has already been done, albeit in a very different context; That context is a malady psychiatrists call “Substance Use Disorder.” More about that next time.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Posted on

Contract restrictions can lead to technical debt

Last updated on March 22nd, 2018 at 11:18 pm

When the owner of an asset, especially a software asset, contracts to provide a capability to a customer incorporating a use of that asset, the work involved might require modification or enhancement of that asset. When the contract permits such work, without transferring ownership of the asset itself, performing it is relatively straightforward, provided the work can be done in a manner compatible with any pre-existing or anticipated future other uses of the asset. But some contract restrictions can cause the owner of the asset to incur technical debt.

A power adaptor/converter for international travelers with U.S. standard equipment.
A power adaptor/converter for international travelers with U.S. standard equipment. This device provides conversion for both hardware connection and voltage supplied.
The wide variation in electric power standards worldwide can be viewed as a technical debt. Someday, in the probably distant future, a world standard will emerge and that debt will be retired. Until then, adaptors like these are travel necessities.

But some contracts restrict such work. For example, for a government customer, ownership of the work performed might be required to transfer to the government customer. Potentially, all of the work might be classified as a national secret. In either of these cases, to retain control of the asset, the owner/contractor arranges to perform all of the work outside the periphery of the asset. To accomplish this, the owner/contractor might interface to the asset through an adaptor that can be transferred to the government customer, or which can be classified as secret as the case may be, thereby insulating the original asset from these ownership restrictions.

The result is tolerable after one such contract is completed. But over time, as the number of adaptors increases, they become a form of technical debt. Each must be maintained against any changes in the original asset. Moreover, making changes to the original asset can become a project of such scale that the temptation to create a static “clone” of the asset for each customer is irresistible. When that happens, any technical debt already present in the asset is also cloned. And any work performed to correct defects in the asset must be performed on each affected clone.

The problem is more general than suggested above. It also appears in the case of software offered for multiple platforms, or multiple versions of a single platform.

But it gets worse. Suppose the maintainers decide to update the asset to make it more extensible, or to make it more maintainable. That update, including all testing and documentation, must then be performed on each clone. If the asset owner elects not to update all clones, then the clones will begin to diverge from each other. Engineers performing tasks on one of the clones must then have knowledge of how it differs from other clones. If a new defect is discovered, it might or might not be present in every clone. Implementing a new extension or other modification might not be possible in all clones, or it might have to be implemented differently in some clones. Life can get very complicated.

Organizations entering into contracts of this kind would be wise either to include language limiting their obligations to maintain the original asset against any changes, internal or external, in its ability to perform its functions. Or they might include an explicit statement of the parties’ intentions relative to financial support for any continuing obligations to maintain that asset.

Organizations offering products for multiple platforms would be wise to consider as strategic the management of technical debt that arises from platform multiplicity. Sound management of this form of technical debt can extend their ability to support multiple platforms, which can dramatically increase returns on investment in the core asset.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Posted on

Balance technical debt and engineering resources

Last updated on December 11th, 2018 at 09:42 am

Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—should create significant savings, and many competitive advantages. These benefits should arise from the reductions in metaphorical interest charges that result from retiring technical debt. But these benefits become available to the organization only if engineering capacity increases relative to the burdens presented by the remaining reduced levels of technical debt. After the technical debt management program is in place, if the balance between engineering resources and the burdens imposed by the remaining technical debt becomes more favorable, then organizational effectiveness will improve. But if the balance becomes less favorable, as a result of reductions in engineering resources,  organizational effectiveness won’t improve, even at lower levels of technical debt.

Flooding from Hurricane Katrina in New Orleans, 2005.
Flooding from Hurricane Katrina in New Orleans, 2005. Any levee humans can  build can be overtopped or undermined by the forces of Nature. So it is with  technology. Any technology humans can devise to attain mastery over technical debt can be overcome or undermined by organizational policy and organizational politics. To master technical debt, technology is not enough—we must also deal with policy and politics.

Unfortunately, it’s possible to adopt advanced technical debt management practices while at the same time reducing engineering capacity to a level such that engineering effectiveness is no better than it was before the technical debt management program was initiated. The reason for this is that the engineering process is not the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves non-technical causes in place. That’s why technological solutions to the technical debt management problem might not be sufficient to produce benefits in organizational effectiveness and agility.

The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has been developed [Izurieta 2017]. It’s reasonable to assume that effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has not been confined to debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].

But it’s also reasonable to ask whether such developments will have much impact on the limiting effects of carrying technical debt, even in software. Given the necessary resources, much of the technical debt now extant could be retired. That is, debt retirement rates are determined only by the will and the capacity to invest in debt retirement. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.

The open question is this: will technological developments alone suffice to gain control of the problem of technical debt? Perhaps not. Organizations could exploit the advancements in technical debt management to execute reductions in engineering staffing—and therefore cost—while they divert savings to other parts of the enterprise, thereby allowing technical debt to remain at levels that, although much reduced, are nevertheless sufficient to compromise the effectiveness of that reduced engineering staff.

For example, schedule pressure is widely recognized as contributing to technical debt formation and persistence. If engineering groups become more adept at managing and preventing technical debt, but marketing and sales groups do not improve their own intelligence and planning processes and consequently demand new capabilities with even shorter timelines than they now do, the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.

Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of non-technical causes of technical debt to persist, and possibly even to grow.

This blog focuses on the non-technical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Posted on

Case 1: A platform upgrade

Last updated on November 21st, 2017 at 08:35 am

This case involves deferring a platform upgrade for SharePoint sites. It illustrates the importance of the policymaker’s view of technical debt, as compared to the view that restricts technical debt to the engineering or IT functions.

Background
File servers in a rack
File servers in a rack. Photo (cc) Abigor courtesy Wikimedia Commons.

Growth at the fictional company Unbelievable Growth, Inc. (UGI) has been so unbelievable that there is now a shortage of financial resources for migrating the last groups of SharePoint users from SharePoint 2010 to SharePoint 2013. Consequently, the CFO instructed IT to continue to support SharePoint 2010 for at least two more quarters. Meanwhile, the affected SharePoint users must continue to use SharePoint 2010. Someday, currently set for two quarters from now, IT and the users of SharePoint 2010 will be required to migrate to SharePoint 2013. Both IT and the users might need to expend resources to keep the SharePoint 2010 site operational. Users who make enhancements to their SharePoint 2010 sites will need to migrate that work to the SharePoint 2013 site, and that might require some rework that would have been unnecessary if the migration had not been deferred.

Discussion

We can regard as a debt UGI’s decision to defer the SharePoint migration. Because it isn’t a financial obligation, we call it a technical debt. UGI must retire that technical debt two quarters from now, when they finally execute the migration from SharePoint 2010 to SharePoint 2013. We can regard the cost of the final migration as the (metaphorical) principal of the technical debt. In the meantime, IT and the users must do some work that might have been unnecessary if they could have performed the migration now. We can regard that extra work as the (metaphorical) interest charges on that technical debt.

The policymaker’s perspective

Some — indeed most — conventional views of technical debt do not regard the deferred upgrade as technical debt, for various reasons: it isn’t software, or it isn’t in a product, or it isn’t a shortcut taken for expedience, and so on. Moreover, the person who made the decision to take on the debt was the CFO, who is not an engineer, and who might not even realize that the implications of the decision result in taking on technical debt.

But from the viewpoint of the policy maker, the commitment to execute the upgrade in the future is equivalent to accepting a technical obligation. For the enterprise, it is a technical debt. Following UGI’s current account procedures, the metaphorical interest on that technical debt will be paid by the SharePoint users and by IT, and it will appear as an operating expense for those groups. That’s unfortunate, because the purpose of deferring the upgrade was unrelated to their operations. It was an enterprise cost-leveling maneuver whose costs should probably be accounted for at the enterprise level to ensure that operational costs for the SharePoint users and for IT are accurately represented, and to accurately represent the CFO’s operations.

Non-technical decisions, occurring anywhere in the enterprise, can sometimes lead to incurring technical debt. Enterprise policy intended to support effective technical debt management must take these phenomena into account.

References

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

Related posts

  • Case 3: Help desk operations
    This case illustrates how a decision to incur technical debt in one part of an organization can burden other parts of the organization with the metaphorical interest charges on that debt. To gain effective control of technical debt, it’s probably necessary to hold accountable those who make the decisions that lead to incurring new debt.
  • Case 2: New software Development
    This case illustrates the importance of recognizing as technical debt any artifact whose condition or absence contributes to a loss of organizational effectiveness or agility. It describes a situation related to software development, and which some would argue is not actually technical debt.
  • Technical debt in the highway system
    Here’s an example of using the technical debt metaphor in civil engineering. In this case, it helps us understand why, in some cases, leaving some technical debt in place can be a better choice than retiring it.
  • Technical debt in a rail system
    Here’s an example of using the technical debt metaphor to think about how elements of a hardware asset can limit the ability of the enterprise to attain and maintain market leadership. The asset in this example is the trackbed of Amtrak’s Acela Express.