Contract restrictions can lead to technical debt

Last updated on July 8th, 2021 at 01:42 pm

When the owner of an asset, especially a software asset, contracts to provide a capability to a customer incorporating a use of that asset, contract restrictions can lead to technical debt. The work involved might require modification or enhancement of that asset. When the contract permits such work without transferring ownership of the asset itself, performing it is relatively straightforward. But complications can arise unless the contractor can perform the work in a manner compatible with any pre-existing or anticipated future other uses of the asset. Even so, some contract restrictions can cause the owner of the asset to incur technical debt.

How technical debt can enter the picture

A power adaptor/converter for international travelers with U.S. standard equipment.
A power adaptor/converter for international travelers with U.S. standard equipment. This device provides conversion for both hardware connection and voltage supplied.
We can regard the wide variation in electric power standards worldwide as a technical debt. Someday, in the probably distant future, a world standard will emerge and we will retire that debt. Until then, adaptors like these are travel necessities.
Some contracts restrict such work. For example, a government customer might require ownership of the work performed. Potentially, all of the work might be classified as a national secret. In either of these cases, to retain control of the asset, the owner/contractor arranges to perform all of the work outside the periphery of the asset. To accomplish this, the owner/contractor might interface to the asset through an adaptor that the government customer can then own, or which can be classified as secret if necessary. These moves insulate the original asset from these ownership restrictions.

The result is tolerable after completion of one such contract. But over time, as their number increases, the adaptors become a form of technical debt. The asset owner must maintain each adaptor against any changes in the original asset. Moreover, making changes to the original asset can become a project of such scale that the temptation to create a static “clone” of the asset for each customer is irresistible. When that happens, cloning replicates both the asset and any technical debt it carries. And correcting defects in the asset requires correcting that same defect in any clones that carry it.

The general forms of the problem

The problem is more general than suggested above. It also appears in the case of software that supports multiple platforms, or which is available in multiple versions supporting a single platform.

But it gets worse. Suppose the maintainers decide to update the asset to make it more extensible, or to make it more maintainable. They must then perform that update, including all testing and documentation, on each clone. If the asset owner elects not to update all clones, then the clones will begin to diverge from each other. Engineers performing tasks on one of the clones must then have knowledge of how that clone differs from other clones. If they discover a new defect, it might or might not be present in every clone. Implementing a new extension or other modification might not be possible in all clones. Or implementing it in some clones might require a unique approach. Life can get very complicated.

Organizations entering into contracts of this kind would be wise to include language limiting their obligations to maintain the original asset against any changes. Or they might include an explicit statement of the parties’ intentions relative to financial support for any continuing obligations to maintain that asset.

Last words

Organizations offering products for multiple platforms would be wise to consider as strategic the management of technical debt that arises from platform multiplicity. Sound management of this form of technical debt can extend their ability to support multiple platforms. And that can dramatically increase returns on investment in the core asset.

Other posts in this thread

How budget depletion leads to technical debt

Last updated on July 8th, 2021 at 01:38 pm

Some projects undergo budget depletion exercises after budget cuts. Or the exercises might occur when there’s evidence that the funds remaining won’t cover the work remaining. Formats vary, but the typical goal of these exercises is downscoping. We remove, relax, defer, or suspend some requirements. With limited funds, we execute downscoping in a manner that leads to technical debt.

A physical example

The Old River Control Complex on the Mississippi River
Photo courtesy USACE
The accompanying photo shows the Old River Control Complex on the Mississippi River. The US Army Corps of Engineers (USACE) built it and operates it. It controls the flow from the Mississippi into the Atchafalaya River, a distributary. The Mississippi would otherwise have rerouted itself into the Atchafalaya, which has a steeper gradient to the ocean. Since that would have deprived New Orleans and its industrial facilities of water and navigational channels, USACE maintains flow control facilities.

The industrial facilities of the lower Mississippi constitute a technical debt. Their existence is no longer compatible with the “update” Nature is trying to deploy. But our national budget won’t support repositioning New Orleans and its industrial facilities. So we redirect the flow of water from Nature’s course to one more compatible with the industrial base. The Old River Control Complex, with levees, dredging projects, and gates throughout lower Louisiana, are the MICs we pay for the technical debt that is the outdated position of New Orleans and its industrial base. For more about Atchafalaya, see the famous New Yorker article by John McPhee [MacFee 1987].

A broad array of effects

Here’s an illustrative scenario. At the time downscoping begins, the work product might contain incomplete implementations of items that are due for removal from the list of objectives. This removal renders unnecessary a set of accommodations contained in surviving artifacts. They comprise a most insidious type of debt that’s difficult to detect. It’s difficult to detect because the affected system components appear to be merely overly complicated. Recognizing them as a residual of a cancelled capability requires knowledge of their history. Unless we document these artifacts at the time of the downscoping, that knowledge may be lost.

Other items of technical debt that arise from budget depletion include tests that no longer serve a purpose, or documentation that’s no longer consistent with the rest of the work product, or user interface artifacts no longer needed. When budgets become sufficiently tight, funds aren’t available for documenting these items of technical debt as debt. The enterprise might then lose track of them when team members move on to other work.

Sometimes, budget depletion takes effect even before the work begins. This happens, for example, when project champions unwittingly underestimate costs to gain approval for the work they have in mind. The unreasonableness of the budget becomes clear soon after the budget approval, and its effects take hold soon thereafter.

Budget depletion can also have some of the same effects as schedule pressure. When the team devises the downscoping plan, it must make choices about what to include in the revised project objectives. In some cases, the desire to include some work can bias estimates of the effort required to execute it. If the team underestimates the work involved, the result is increased pressure to perform that work. With increased pressure comes technical debt. See “With all deliberate urgency” for more.

Last words

A policy that could limit technical debt formation in response to budget depletion would require identifying the technical debt such action creates, and later retiring that debt. Because these actions do require resources, they consume some of the savings that were supposed to accrue from downscoping. In some cases, they could consume that amount in its entirety, or more. Most decision makers probably over-estimate the effectiveness of the downscoping strategy. Often, it simply reduces current expenses by trading them for increased technical debt, which raises future expenses and decreases opportunities in future periods.

References

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

Other posts in this thread

How outsourcing leads to increasing technical debt

Last updated on July 8th, 2021 at 01:37 pm

Most of the nontechnical precursors of technical debt that afflict in-house work also afflict outsourced work. For example, the planning fallacy affects internal planners. But it also afflicts the bidders for contracts that cover outsourced work. As described in “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma,” Boehm, et al., call this the “Conspiracy of Optimism.” [Boehm 2016] But outsourcing engineering work can introduce pathways for incurring technical debt that are specific to outsourcing.

The risks of incurring technical debt from outsourcing

Green fields
Green fields. Greenfield outsourcing, also known as startup outsourcing, is the outsourcing of activity that the enterprise has never performed in-house. Greenfield outsourcing is especially tricky with respect to technical debt formation. The risk arises because much of the expertise necessary to perform the work in question is probably not resident within the enterprise. That void in enterprise expertise makes for difficulties in managing technical debt in the outsourced artifacts.
Outsourcing is inherently more likely to incur technical debt than is equivalent work performed in-house. When most enterprises contract for development of systems or software, the criteria for acceptance rarely include specifications for maintainability or extensibility. This happens, in part, because such qualitative specifications are difficult to define quantitatively. That’s why the condition of deliverables relative to maintainability and extensibility is so variable. Outsourced deliverables can best be described as bearing an unknown level of technical debt.

The root cause of the problem is likely a lack of a universally accepted metrics for quantifying technical debt [Li 2015]. That’s why it’s difficult to specify in the vendor contract an acceptability threshold for technical debt. And since the consequences of technical debt in deliverables potentially remain hidden during the lifetime of the outsource contract, years might pass before the issue becomes evident. That inevitably complicates the task of understanding the root cause of the problem.

Six ways outsourcing can contribute to technical debt

In what follows, I use the term vendor to denote the organization performing the outsourced work. I use the term enterprise to denote the organization that has outsourced a portion of its engineering work. The retained organization is the portion of the enterprise directly relevant to the outsourced work, and which has remained in-house.

Among the mechanisms that lead to incurring technical debt in the outsourcing context are the six mechanisms sketched below. Gauging the size of these effects by auditing deliverables or by auditing the internal processes of the vendor could be helpful in managing levels of technical debt arising from outsourcing.

This list isn’t exhaustive. Quite possibly other phenomena also contribute to technical debt formation in the context of outsourcing.

Progressive erosion of retained organization capabilities

Over time, after outsourcing work on a particular asset, the enterprise can expect erosion of some engineering expertise. The expertise most vulnerable is that needed to manage, evaluate, understand, or, if need be, to re-insource (or backsource) the outsourced work [Willcocks 2004][Beardsell 2010]. Typically, staff who formerly performed the outsourced work do move on to other work, voluntarily or not, either within the enterprise or elsewhere. Indeed, cost savings from terminations and employee buyouts often accompany outsourcing decisions. That reallocation is part of the economic justification of outsourcing decisions.

But even if the enterprise continues to employ the people who formerly performed the outsourced work, the deleterious effects remain. Those employees, filling new roles, likely become less familiar with the current work and therefore less able to perform it. And since they’re now carrying out other assignments, their availability is limited. In the public sector, the organizations that perform the outsourced work exacerbate this phenomenon by recruiting staff from their former agencies [Kusnet 2007]. In manufacturing, Kinkel, et al., suggest that, paraphrasing, outsourcing disturbs the formation of internal competence [Kinkel 2016].

Thus, outsourcing engineering efforts can erode the ability of the enterprise to perform the outsourced work internally. Likewise, it erodes enterprise capability to monitor or evaluate the work when vendors perform it. Consequently, the enterprise is less able to monitor, evaluate, or retire any technical debt that accumulates in the outsourced work products.

Stovepiping among vendors

Most multi-vendor efforts use a separation-of-concerns approach [Laplante 2007] to dividing the work. That is, each vendor is empowered to use any approach it can, consistent with its own contract and statement of work. In some cases, two or more vendors might have overlapping needs that cause them each to produce similar capabilities as elements of their respective deliverables. Sharing their results is of course possible. But unless both of their contracts anticipate the need for sharing, sharing is unlikely. Failure to share those results that could have been shared can lead to incurring unrecognized technical debt.

Stovepiping within vendors

Technical debt formation is possible even if there is only one vendor. Different teams or individuals working for that vendor might unwittingly create elements with overlapping capabilities. That duplication could lead to technical debt, or it could constitute technical debt in itself. For example, two teams working for the same vendor might have similar needs, and might develop duplicative tools independently. As a second example, consider a version of stovepiping combined with temporal displacement. Suppose that one team is unaware that a previous effort for the same customer had already developed a capability that it now needs. That team then might re-create that already-existing capability.

Stovepiping within vendors is less likely when the vendor operates under a single vendor technical lead, and the enterprise interacts with that single lead through a single in-house technical lead. When either side of the relationship manages communications through multiple contacts, stovepiping is more likely. New technical debt is more likely to form.

Loss of continuity in the outsourced engineering staff

Unless the agreement between the vendor and the enterprise specifically addresses staff volatility, an additional risk arises. Staff turnover or reassignment within the vendor organization can lead to technical debt. This can happen in just the same way that turnover in-house can lead to technical debt. But the risk is most significant at interfaces between one version of the product or service and the next. With outsourcing, however, the vendor has little internalized motivation to control this phenomenon. Moreover, the enterprise likely has less control and less awareness of staff assignments within the vendor organization than it does within the enterprise. Thus, loss of continuity in the outsourced engineering staff is both more likely and more likely to lead to technical debt.

Reduced coordination of engineering approaches and business objectives

Lack of coordination between engineering approaches and business planning can cause technical debt formation. This happens because engineers create and deploy artifacts that they must revisit later. The need for rework arises after engineers learn of business plans they didn’t know about at the time they produced those artifacts. See “Failure to communicate long-term business strategy.” This scenario is more likely, and possibly irresolvable, when the enterprise withholds its long-term plans from the outsourcing vendor to protect its strategy.

Hiring and retention problems

In some instances, the enterprise has never before performed work like the outsourced work [Delen 2007]. This kind of outsourcing is startup outsourcing or greenfield outsourcing. In these cases, typically, the enterprise must reassign existing employees or hire new employees to interface with the outsource vendor. These roles are analogous to remote supervisors, except that the teams they supervise aren’t enterprise employees. Hiring and retaining people for these roles can be difficult. Startup challenges arise both within the enterprise and within the vendor organization. Recruitment and especially retention problems in these roles can decrease the likelihood of controlling technical debt, and increase the likelihood of incurring technical debt.

Last words

A policy that would address these risks is one that would facilitate accomplishing three objectives:

  • Retain organizational capability sufficient to assess the effect on technical debt of any outsourced engineering work
  • Represent the effect on technical debt faithfully in any financial models used in making the outsourcing decision
  • Include in financial models the effects of technical debt, the cost of carrying technical debt, and the effects on controlling technical debt when deciding whether to extend outsourcing contracts with vendors.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Confirmation bias and technical debt

Last updated on July 8th, 2021 at 01:35 pm

Confirmation bias is a cognitive bias [Kahneman 2011]. It’s the human tendency to favor and seek only information that confirms our preconceptions. It also causes us to avoid information that disconfirms our preconceptions. For example, the homogeneity of cable news channel audiences is a result of confirmation bias. Another result is the alignment between preconceptions of the audience and the slant of the newscast for that channel.

How confirmation bias can lead to technical debt

Third stage ignition, sending the Mars Climate Orbiter to Mars in December, 1998
Computer-generated image of the third stage ignition, sending the Mars Climate Orbiter (MCO) to Mars in December, 1998. The spacecraft eventually broke up in the Martian atmosphere due to what is now called the “metric mix-up.” The Lockheed Martin team that constructed and programmed the MCO used Imperial units. But the team at JPL that was responsible for flying the MCO used metric units. After the loss of the MCO, an investigation led by NASA uncovered the mix-up.
One of the many changes resulting from this loss was increased use of reviews and inspections. We don’t know why reviews and inspections weren’t as thorough before the loss of the MCO as they are now. But one possibility is the effects of confirmation bias in assessing the need for reviews and inspections. Image courtesy Engineering Multimedia, Inc., and U.S. National Aeronautics and Space Administration
Confirmation bias causes technical debt by biasing the information on which decision makers base decisions involving technical debt. Most people in these roles have objectives they regard as having priority over eliminating technical debt. This causes them to bias their searches for information about technical debt. The bias favors information that would support directly the achievement of those primary objectives. Decisions tend, for example, to discount warnings of technical debt issues. They also tend to underfund technical debt assessments, and set aside advice regarding avoiding debt formation in current projects.

An example

For example, anyone determined to find reasons to be skeptical of the need to manage technical debt need only execute a few Google searches. Searching for there is no such thing as technical debt yields about 300,000 results at this writing; technical debt is a fraud about 1.6 million; and technical debt is a bad metaphor about 3.7 million. Compare this to technical debt which yields only 1.6 million. A skeptic wouldn’t even have to read any of these pages to come away convinced that technical debt is at best a controversial concept. This is, of course, specious reasoning, if it’s reasoning at all. But it does serve to illustrate the potential for confirmation bias to contribute to preventing or limiting rational management of technical debt.

Last words

Detecting confirmation bias in oneself is extraordinarily difficult because confirmation bias causes us to (a) decide not to search for data that would reveal confirmation bias; and (b) if data somehow becomes available, to disregard or to seek alternative explanations for it if that data tends to confirm the existence of confirmation bias. Moreover, another cognitive bias known as the bias blind spot causes individuals to see the existence and effects of cognitive biases much more in others than in themselves [Pronin 2002].

Still, the enterprise would benefit from monitoring the possible existence and effects of confirmation bias in decisions with respect to allocating resources to managing technical debt. Whenever decisions are made, we must manage confirmation bias risk.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma

Last updated on July 8th, 2021 at 01:31 pm

In a 1977 report, Daniel Kahneman and Amos Tversky identify one particular cognitive bias [Kahneman 2011], the planning fallacy, which afflicts planners [Kahneman 1977] [Kahneman 1979]. They discuss two types of evidence planners use. Singular evidence is specific to the case at hand. Distributional evidence is specific to similar past efforts. The planning fallacy is planners’ tendency to pay too little attention to distributional evidence and too much to singular evidence. They do this even when the singular evidence is scanty or questionable. Failing to harvest lessons from the distributional evidence, which is inherently more diverse than singular evidence, the planners tend to underestimate cost and schedule. So there’s a tendency to promise lower costs, faster delivery, and greater benefits than anyone can reasonably expect.

Enter the n-person prisoner’s dilemma

Boehm et al. [Boehm 2016] describe a dynamic that exacerbates the problem. They observe that because organizational resources are finite, project champions compete with each other for resources. This competition compels them to be unrealistically optimistic about their objectives, costs, and schedules. Although Boehm et al. call this mechanism the “Conspiracy of Optimism,” possibly facetiously, it isn’t actually a conspiracy. Rather, it’s a variant of the N-Person Prisoner’s Dilemma [Hamburger 1973].

A special property of pressure-induced debt

Hoover Dam, aerial view, September 2017
Hoover Dam, aerial view, September 2017. Under construction from 1931 to 1936, the cost of the dam was $48.8M ($639M in 2016 dollars) under a fixed-price contract. It was complete two years early. Apparently the planning fallacy doesn’t act inevitably. 112 men died in incidents associated with its construction. 42 more died of a condition diagnosed as pneumonia, but which experts now believe to have been carbon monoxide poisoning due to poor ventilation in the dam’s diversion tunnels during construction. There’s little doubt that unrealistic optimism affects more than budget and schedule projections. It also affects risk projections, including deaths. Photo (cc) Mariordo (Mario Roberto Durán Ortiz), courtesy Wikimedia Commons.
Unrealistic optimism creates budget shortfalls and schedule pressures. In turn, they both contribute to conditions favorable for creating nonstrategic technical debt. And this mechanism, or any mechanism associated with schedule or budget pressure, tends to produce technical debt that’s subtle—it’s the type least likely to become evident in the short term. For example, technical debt that might make a particular enhancement more difficult in the next project is more likely to appear than technical debt that creates trouble in the current effort. Debt that creates trouble in the current effort is more likely to be retired in the short term, if not in the current effort. Awkward architecture might be more difficult to identify. It’s therefore more likely to survive in the intermediate or long term.

The bad news of schedule pressure

In other words, the technical debt most likely to be generated is that which is the most benign in the short term, and which is therefore more likely to escape notice. If noticed, it’s more likely to be forgotten unless carefully documented. And that action is unlikely under schedule and budget pressure. In this way, the nonstrategic technical debt created as a result of unrealistic optimism is more likely than most technical debt to eventually become legacy technical debt.

Last words

Policymakers can assist in addressing the consequences of unrealistic optimism by advocating for education about it. They can also advocate for changes in incentive structures and performance management systems. It’s good business to establish organizational standards with respect to realism in promised benefits, costs, and schedules.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Feature bias: unbalanced concern for capability vs. sustainability

Last updated on July 7th, 2021 at 09:56 pm

Alaska crude oil production 1990-2015
Alaska crude oil production 1990-2015. This chart [Yen 2015] displays Alaska crude oil produced and shipped through the Trans Alaska Pipeline System (TAPS) from 1990 to 2015. Production had dropped by 75% in that period, and the decline is projected to continue. In January 2018, in response to pressure from Alaskan government officials and the energy industry, the U.S. Congress passed legislation that opened the Arctic National Wildlife Refuge to oil exploration, despite the threat to ecological sustainability that exploration poses. If we regard TAPS as a feature of the U.S. energy production system, we can view its excess capacity as a source of feature bias. It creates pressure on decision makers to add features to the U.S. energy system. Alternatively, they could act to enhance the sustainability of Alaskan and global environmental systems [Wight 2017].

<

Enterprise decision makers affected by feature bias tend to harbor distorted views of the importance of new capability development compared to technical debt management. This tendency is likely due to the customer’s relative sensitivity to features, and relative lack of awareness of sustainability. Whatever the cause, customers tend to be more attracted to features than they are to indicators of sound technical debt management and other product sustainability practices. This tendency puts decision makers at risk of feature bias: unbalanced concern for capability vs. sustainability.

h4>Accounting changes can help

Changes in cost accounting could mitigate feature bias effects by projecting more accurately total MICs based on historical data and sound estimation. I explore possible accounting changes later in this post, and in future posts; meanwhile, let’s explore the causes and consequences of the distorted perspective I’m calling feature bias.

Causes and consequences of feature bias

For products or services offered outside the enterprise, the sales and marketing functions of the enterprise represent the voice of the customer [Gaskin 1991]. But customers are generally unaware of product or service attributes that determine maintainability, extensibility, or cybersecurity. These factors, the sustainability factors, affect the MICs for technical debt. But customers are acutely aware of capabilities—or missing or defective capabilities. Customer comments and requests are therefore unbalanced in favor of capability over sustainability. The sales and marketing functions tend to accurately transmit this unbalanced perspective to decision makers and technologists.

An analogous mechanism prevails with respect to infrastructure and its internal customers. Internal customers tend to be more concerned with capabilities than they are with sustainability of the processes and systems that deliver those capabilities. Thus, pressure from internal customers tends to emphasize capability at the expense of sustainability. The result of this imbalance is pressure to allocate excessive resources to capability enhancement, compared to activities that improve sustainability. And therefore controlling or reducing technical debt and its MICs gets less attention.

Nor is this the only consequence of feature bias. It provides unrelenting pressure for increasing numbers of features, despite the threats to architectural coherence and overall usability that such “featuritis” or “featurism” present. Featurism leads, ultimately, to feature bloat, and to difficulties for users, who can’t find what they need among the clutter of features that are often too numerous to document. For example, in Microsoft Word, many users are unaware that Shift+F5 moves the insertion point and cursor to the point in the active document that was last edited, even if the document has just been freshly loaded into Word. Useful, but obscure.

Feature bias bias

Feature bias, it must be noted, is subject to biases itself. The existing array of features appeals to a certain subset of all potential customers. Because it is that subset that’s most likely to request repair of existing features. And they’re also the most likely to suggest additional features. The pressure for features tends to be biased in favor of the needs of the most vociferous users. That is, there is pressure to evolve to better meet the needs of existing users. That pressure can force to lower priority any efforts toward meeting the needs of other stakeholders or potential stakeholders. These other stakeholders might be even more important to the enterprise than are the existing users. This bias in feature bias presents another risk that can affect decision makers.

Organizations can take steps to mitigate the risks of feature bias. An example of such a measure might be using focus groups to study how educating customers in sustainability issues affects their perspectives relative to feature bias. Educating decision makers about feature bias can also reduce this risk.

At the enterprise scale, awareness of feature bias would be helpful. But awareness alone is unlikely to counter its detrimental effects. These effects include underfunding technical debt management efforts. Eliminating the source of feature bias is extraordinarily difficult, because customers and potential customers aren’t subject to enterprise policy. Feature bias and feature bias bias are therefore givens. To mitigate the effects of feature bias, we must adopt policies that compel decision makers to consider the need to deal with technical debt.

A possible corrective action

One possible corrective action might be improving accounting practices for MICs, based on historical data. For example, there’s a high probability that any project might produce new technical debt. It might be prudent to fund the retirement of that debt in the form of reserves when we fund projects. And if we know that a project has encountered some newly recognized form of technical debt, it might be prudent to reserve resources to retire that debt as soon as possible. Ideas such as these can rationalize resource allocations with respect to technical debt.

These two examples illustrate what’s necessary if we want to mitigate the effects of feature bias. They also illustrate just how difficult such a task will be.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

The fundamental attribution error

Last updated on July 8th, 2021 at 01:29 pm

When we try to understand the behavior of others, we often make a particularly human mistake. We tend to attribute too much to character and disposition and too little to situation and context. This mistake is so common that it has a name: The Fundamental Attribution Error (FAE) (See “The Fundamental Attribution Error” at my other blog). And although little experimental data is available regarding its effects on technical debt, we can plausibly argue that its effects are significant—and unwelcome.

Arapaho moccasins ca. 1880-1910.
Arapaho moccasins ca. 1880-1910. An American Indian proverb advises, “Don’t judge any man until you have walked two moons in his moccasins.” From the perspective of the FAE, the proverb is a way of mitigating FAE risks. Photo of Arapaho moccasins, ca. 1880-1910 on exhibit at the Bata Shoe Museum, in Toronto, Canada. Photo by Daderot, courtesy Wikimedia
The FAE contributes to technical debt in at least two ways. First, it distorts assessments by non-engineers of the motivations of engineers as they warn of future difficulties from technical debt. Second, it distorts assessments by engineers of the motivations of non-engineers as they oppose allocation of resources to technical debt retirement. They oppose these allocations in order to conserve resources for their own efforts or to accelerate efforts in which they have more immediate interest. The two effects are symmetrical in the large, though not in detail.

Below is a description of the effects of the FAE on engineers and non-engineers. Some of the non-engineers are the internal customers of the engineers. I examine the effects of the FAE that arise from three different claims of the parties to the exchange.

Claim: Technical debt depresses engineering productivity

Many engineers or their managers hold this position.

Engineers notice incidents in which some of the work they must perform on an asset would be much easier or even unnecessary were it not for the technical debt that the asset carries. They sense the burden of the extra effort because they know how much easier and faster the work would be if they could retire the debt.

The internal customers of engineers don’t see these circumstances as clearly as engineers and their managers do. Consequently, they tend to discount engineers’ claims of depressed productivity. Some experience engineers’ complaints, requests, and warnings as whining, self-serving nest feathering, or worse. They tend to attribute engineers’ complaints to faults in the character or “work ethic” of engineers.

Claim: Instead of retiring the technical debt now, just document it for later

This is a suggestion senior managers or the engineers’ internal customers often put forth.

The internal customers of engineers have pressing needs for immediate engineering results. They see new products or repairs to existing products as a means of achieving the objectives the enterprise sets. Focusing limited engineering resources on technical debt retirement conflicts with producing results that would help internal customers of engineers meet these more immediate objectives. As a compromise, non-engineers propose that engineers document instances of technical debt as they find them, so that they can be addressed more efficiently after engineers meet the immediate needs of internal customers.

Engineers discount the validity of this approach for three reasons. First, they don’t experience the pressures their internal customers do. To engineers, their customers’ reports of more pressing needs seem to be merely excuses to get what they want when they want it. Second, the proposed documentation work doesn’t advance the engineers’ customer’s current project toward its objectives. Instead, it actually delays the current project, in ways invisible to non-engineers. These delays induce increases in schedule pressure, and therefore technical debt. The technical debt occurs because the customer of the current project rarely cares enough about the technical debt documentation effort to allow for the extra time it takes. Finally, because many assets evolve continuously, such documentation has a short shelf life. And that limits its value in ways non-engineers might not appreciate.

In these ways, the FAE both creates the documentation suggestion, and limits the ability of engineers to appreciate its motivation. But it also limits the ability of non-engineers to appreciate how limited is the value of the documentation.

Claim: Addressing technical debt is important, but not urgent

Senior managers or the engineers’ internal customers are most common among adherents of this belief.

When the engineering organization presents a business case for investing in addressing technical debt, they aren’t alone. Other functions in the enterprise also make business cases of their own. Too often, these cases are evaluated against each other. Investment in one entails reduced investment in another. But the benefits of technical debt retirement tend to become most visible to non-engineers much later than do the benefits of some other proposals. Sometimes the benefits of technical debt retirement are wholly invisible to non-engineers. For these reasons, technical debt retirement projects tend perhaps more often than most to be deferred at best, or, worse, rejected.

The FAE is in part responsible for the perception of non-engineers that the benefits of technical debt remediation arrive in the distant future. Engineers notice the benefits relatively immediately, because they interact with the rehabilitated assets on a daily basis. Since non-engineers don’t have these experiences, they notice the benefits only upon delivery of the results of engineering work. This mismatch of the timescales of perceptions of engineers and non-engineers prevents non-engineers from perceiving what is in daily evidence to engineers.

Last words

Both engineers and non-engineers are subject to deadlines and resource limitations beyond their control. Their ability to appreciate the challenges their counterparts face is the key to effective collaboration. Too often, neither part feels that it has the time or resources to accommodate the needs of the other.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[MacFee 1987] John MacFee. “Atchafalaya,” The New Yorker, February 23, 1987.

Available: here; Retrieved: February 5, 2018.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Show Buttons
Hide Buttons