Blog

Posted on

How outsourcing leads to increasing technical debt

Last updated on July 8th, 2021 at 01:37 pm

Most of the nontechnical precursors of technical debt that afflict in-house work also afflict outsourced work. For example, the planning fallacy affects internal planners. But it also afflicts the bidders for contracts that cover outsourced work. As described in “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma,” Boehm, et al., call this the “Conspiracy of Optimism.” [Boehm 2016] But outsourcing engineering work can introduce pathways for incurring technical debt that are specific to outsourcing.

The risks of incurring technical debt from outsourcing

Green fields
Green fields. Greenfield outsourcing, also known as startup outsourcing, is the outsourcing of activity that the enterprise has never performed in-house. Greenfield outsourcing is especially tricky with respect to technical debt formation. The risk arises because much of the expertise necessary to perform the work in question is probably not resident within the enterprise. That void in enterprise expertise makes for difficulties in managing technical debt in the outsourced artifacts.
Outsourcing is inherently more likely to incur technical debt than is equivalent work performed in-house. When most enterprises contract for development of systems or software, the criteria for acceptance rarely include specifications for maintainability or extensibility. This happens, in part, because such qualitative specifications are difficult to define quantitatively. That’s why the condition of deliverables relative to maintainability and extensibility is so variable. Outsourced deliverables can best be described as bearing an unknown level of technical debt.

The root cause of the problem is likely a lack of a universally accepted metrics for quantifying technical debt [Li 2015]. That’s why it’s difficult to specify in the vendor contract an acceptability threshold for technical debt. And since the consequences of technical debt in deliverables potentially remain hidden during the lifetime of the outsource contract, years might pass before the issue becomes evident. That inevitably complicates the task of understanding the root cause of the problem.

Six ways outsourcing can contribute to technical debt

In what follows, I use the term vendor to denote the organization performing the outsourced work. I use the term enterprise to denote the organization that has outsourced a portion of its engineering work. The retained organization is the portion of the enterprise directly relevant to the outsourced work, and which has remained in-house.

Among the mechanisms that lead to incurring technical debt in the outsourcing context are the six mechanisms sketched below. Gauging the size of these effects by auditing deliverables or by auditing the internal processes of the vendor could be helpful in managing levels of technical debt arising from outsourcing.

This list isn’t exhaustive. Quite possibly other phenomena also contribute to technical debt formation in the context of outsourcing.

Progressive erosion of retained organization capabilities

Over time, after outsourcing work on a particular asset, the enterprise can expect erosion of some engineering expertise. The expertise most vulnerable is that needed to manage, evaluate, understand, or, if need be, to re-insource (or backsource) the outsourced work [Willcocks 2004][Beardsell 2010]. Typically, staff who formerly performed the outsourced work do move on to other work, voluntarily or not, either within the enterprise or elsewhere. Indeed, cost savings from terminations and employee buyouts often accompany outsourcing decisions. That reallocation is part of the economic justification of outsourcing decisions.

But even if the enterprise continues to employ the people who formerly performed the outsourced work, the deleterious effects remain. Those employees, filling new roles, likely become less familiar with the current work and therefore less able to perform it. And since they’re now carrying out other assignments, their availability is limited. In the public sector, the organizations that perform the outsourced work exacerbate this phenomenon by recruiting staff from their former agencies [Kusnet 2007]. In manufacturing, Kinkel, et al., suggest that, paraphrasing, outsourcing disturbs the formation of internal competence [Kinkel 2016].

Thus, outsourcing engineering efforts can erode the ability of the enterprise to perform the outsourced work internally. Likewise, it erodes enterprise capability to monitor or evaluate the work when vendors perform it. Consequently, the enterprise is less able to monitor, evaluate, or retire any technical debt that accumulates in the outsourced work products.

Stovepiping among vendors

Most multi-vendor efforts use a separation-of-concerns approach [Laplante 2007] to dividing the work. That is, each vendor is empowered to use any approach it can, consistent with its own contract and statement of work. In some cases, two or more vendors might have overlapping needs that cause them each to produce similar capabilities as elements of their respective deliverables. Sharing their results is of course possible. But unless both of their contracts anticipate the need for sharing, sharing is unlikely. Failure to share those results that could have been shared can lead to incurring unrecognized technical debt.

Stovepiping within vendors

Technical debt formation is possible even if there is only one vendor. Different teams or individuals working for that vendor might unwittingly create elements with overlapping capabilities. That duplication could lead to technical debt, or it could constitute technical debt in itself. For example, two teams working for the same vendor might have similar needs, and might develop duplicative tools independently. As a second example, consider a version of stovepiping combined with temporal displacement. Suppose that one team is unaware that a previous effort for the same customer had already developed a capability that it now needs. That team then might re-create that already-existing capability.

Stovepiping within vendors is less likely when the vendor operates under a single vendor technical lead, and the enterprise interacts with that single lead through a single in-house technical lead. When either side of the relationship manages communications through multiple contacts, stovepiping is more likely. New technical debt is more likely to form.

Loss of continuity in the outsourced engineering staff

Unless the agreement between the vendor and the enterprise specifically addresses staff volatility, an additional risk arises. Staff turnover or reassignment within the vendor organization can lead to technical debt. This can happen in just the same way that turnover in-house can lead to technical debt. But the risk is most significant at interfaces between one version of the product or service and the next. With outsourcing, however, the vendor has little internalized motivation to control this phenomenon. Moreover, the enterprise likely has less control and less awareness of staff assignments within the vendor organization than it does within the enterprise. Thus, loss of continuity in the outsourced engineering staff is both more likely and more likely to lead to technical debt.

Reduced coordination of engineering approaches and business objectives

Lack of coordination between engineering approaches and business planning can cause technical debt formation. This happens because engineers create and deploy artifacts that they must revisit later. The need for rework arises after engineers learn of business plans they didn’t know about at the time they produced those artifacts. See “Failure to communicate long-term business strategy.” This scenario is more likely, and possibly irresolvable, when the enterprise withholds its long-term plans from the outsourcing vendor to protect its strategy.

Hiring and retention problems

In some instances, the enterprise has never before performed work like the outsourced work [Delen 2007]. This kind of outsourcing is startup outsourcing or greenfield outsourcing. In these cases, typically, the enterprise must reassign existing employees or hire new employees to interface with the outsource vendor. These roles are analogous to remote supervisors, except that the teams they supervise aren’t enterprise employees. Hiring and retaining people for these roles can be difficult. Startup challenges arise both within the enterprise and within the vendor organization. Recruitment and especially retention problems in these roles can decrease the likelihood of controlling technical debt, and increase the likelihood of incurring technical debt.

Last words

A policy that would address these risks is one that would facilitate accomplishing three objectives:

  • Retain organizational capability sufficient to assess the effect on technical debt of any outsourced engineering work
  • Represent the effect on technical debt faithfully in any financial models used in making the outsourcing decision
  • Include in financial models the effects of technical debt, the cost of carrying technical debt, and the effects on controlling technical debt when deciding whether to extend outsourcing contracts with vendors.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Posted on

Confirmation bias and technical debt

Last updated on July 8th, 2021 at 01:35 pm

Confirmation bias is a cognitive bias [Kahneman 2011]. It’s the human tendency to favor and seek only information that confirms our preconceptions. It also causes us to avoid information that disconfirms our preconceptions. For example, the homogeneity of cable news channel audiences is a result of confirmation bias. Another result is the alignment between preconceptions of the audience and the slant of the newscast for that channel.

How confirmation bias can lead to technical debt

Third stage ignition, sending the Mars Climate Orbiter to Mars in December, 1998
Computer-generated image of the third stage ignition, sending the Mars Climate Orbiter (MCO) to Mars in December, 1998. The spacecraft eventually broke up in the Martian atmosphere due to what is now called the “metric mix-up.” The Lockheed Martin team that constructed and programmed the MCO used Imperial units. But the team at JPL that was responsible for flying the MCO used metric units. After the loss of the MCO, an investigation led by NASA uncovered the mix-up.
One of the many changes resulting from this loss was increased use of reviews and inspections. We don’t know why reviews and inspections weren’t as thorough before the loss of the MCO as they are now. But one possibility is the effects of confirmation bias in assessing the need for reviews and inspections. Image courtesy Engineering Multimedia, Inc., and U.S. National Aeronautics and Space Administration
Confirmation bias causes technical debt by biasing the information on which decision makers base decisions involving technical debt. Most people in these roles have objectives they regard as having priority over eliminating technical debt. This causes them to bias their searches for information about technical debt. The bias favors information that would support directly the achievement of those primary objectives. Decisions tend, for example, to discount warnings of technical debt issues. They also tend to underfund technical debt assessments, and set aside advice regarding avoiding debt formation in current projects.

An example

For example, anyone determined to find reasons to be skeptical of the need to manage technical debt need only execute a few Google searches. Searching for there is no such thing as technical debt yields about 300,000 results at this writing; technical debt is a fraud about 1.6 million; and technical debt is a bad metaphor about 3.7 million. Compare this to technical debt which yields only 1.6 million. A skeptic wouldn’t even have to read any of these pages to come away convinced that technical debt is at best a controversial concept. This is, of course, specious reasoning, if it’s reasoning at all. But it does serve to illustrate the potential for confirmation bias to contribute to preventing or limiting rational management of technical debt.

Last words

Detecting confirmation bias in oneself is extraordinarily difficult because confirmation bias causes us to (a) decide not to search for data that would reveal confirmation bias; and (b) if data somehow becomes available, to disregard or to seek alternative explanations for it if that data tends to confirm the existence of confirmation bias. Moreover, another cognitive bias known as the bias blind spot causes individuals to see the existence and effects of cognitive biases much more in others than in themselves [Pronin 2002].

Still, the enterprise would benefit from monitoring the possible existence and effects of confirmation bias in decisions with respect to allocating resources to managing technical debt. Whenever decisions are made, we must manage confirmation bias risk.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Posted on

Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma

Last updated on July 8th, 2021 at 01:31 pm

In a 1977 report, Daniel Kahneman and Amos Tversky identify one particular cognitive bias [Kahneman 2011], the planning fallacy, which afflicts planners [Kahneman 1977] [Kahneman 1979]. They discuss two types of evidence planners use. Singular evidence is specific to the case at hand. Distributional evidence is specific to similar past efforts. The planning fallacy is planners’ tendency to pay too little attention to distributional evidence and too much to singular evidence. They do this even when the singular evidence is scanty or questionable. Failing to harvest lessons from the distributional evidence, which is inherently more diverse than singular evidence, the planners tend to underestimate cost and schedule. So there’s a tendency to promise lower costs, faster delivery, and greater benefits than anyone can reasonably expect.

Enter the n-person prisoner’s dilemma

Boehm et al. [Boehm 2016] describe a dynamic that exacerbates the problem. They observe that because organizational resources are finite, project champions compete with each other for resources. This competition compels them to be unrealistically optimistic about their objectives, costs, and schedules. Although Boehm et al. call this mechanism the “Conspiracy of Optimism,” possibly facetiously, it isn’t actually a conspiracy. Rather, it’s a variant of the N-Person Prisoner’s Dilemma [Hamburger 1973].

A special property of pressure-induced debt

Hoover Dam, aerial view, September 2017
Hoover Dam, aerial view, September 2017. Under construction from 1931 to 1936, the cost of the dam was $48.8M ($639M in 2016 dollars) under a fixed-price contract. It was complete two years early. Apparently the planning fallacy doesn’t act inevitably. 112 men died in incidents associated with its construction. 42 more died of a condition diagnosed as pneumonia, but which experts now believe to have been carbon monoxide poisoning due to poor ventilation in the dam’s diversion tunnels during construction. There’s little doubt that unrealistic optimism affects more than budget and schedule projections. It also affects risk projections, including deaths. Photo (cc) Mariordo (Mario Roberto Durán Ortiz), courtesy Wikimedia Commons.
Unrealistic optimism creates budget shortfalls and schedule pressures. In turn, they both contribute to conditions favorable for creating nonstrategic technical debt. And this mechanism, or any mechanism associated with schedule or budget pressure, tends to produce technical debt that’s subtle—it’s the type least likely to become evident in the short term. For example, technical debt that might make a particular enhancement more difficult in the next project is more likely to appear than technical debt that creates trouble in the current effort. Debt that creates trouble in the current effort is more likely to be retired in the short term, if not in the current effort. Awkward architecture might be more difficult to identify. It’s therefore more likely to survive in the intermediate or long term.

The bad news of schedule pressure

In other words, the technical debt most likely to be generated is that which is the most benign in the short term, and which is therefore more likely to escape notice. If noticed, it’s more likely to be forgotten unless carefully documented. And that action is unlikely under schedule and budget pressure. In this way, the nonstrategic technical debt created as a result of unrealistic optimism is more likely than most technical debt to eventually become legacy technical debt.

Last words

Policymakers can assist in addressing the consequences of unrealistic optimism by advocating for education about it. They can also advocate for changes in incentive structures and performance management systems. It’s good business to establish organizational standards with respect to realism in promised benefits, costs, and schedules.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Posted on

Feature bias: unbalanced concern for capability vs. sustainability

Last updated on July 7th, 2021 at 09:56 pm

Alaska crude oil production 1990-2015
Alaska crude oil production 1990-2015. This chart [Yen 2015] displays Alaska crude oil produced and shipped through the Trans Alaska Pipeline System (TAPS) from 1990 to 2015. Production had dropped by 75% in that period, and the decline is projected to continue. In January 2018, in response to pressure from Alaskan government officials and the energy industry, the U.S. Congress passed legislation that opened the Arctic National Wildlife Refuge to oil exploration, despite the threat to ecological sustainability that exploration poses. If we regard TAPS as a feature of the U.S. energy production system, we can view its excess capacity as a source of feature bias. It creates pressure on decision makers to add features to the U.S. energy system. Alternatively, they could act to enhance the sustainability of Alaskan and global environmental systems [Wight 2017].

<

Enterprise decision makers affected by feature bias tend to harbor distorted views of the importance of new capability development compared to technical debt management. This tendency is likely due to the customer’s relative sensitivity to features, and relative lack of awareness of sustainability. Whatever the cause, customers tend to be more attracted to features than they are to indicators of sound technical debt management and other product sustainability practices. This tendency puts decision makers at risk of feature bias: unbalanced concern for capability vs. sustainability.

h4>Accounting changes can help

Changes in cost accounting could mitigate feature bias effects by projecting more accurately total MICs based on historical data and sound estimation. I explore possible accounting changes later in this post, and in future posts; meanwhile, let’s explore the causes and consequences of the distorted perspective I’m calling feature bias.

Causes and consequences of feature bias

For products or services offered outside the enterprise, the sales and marketing functions of the enterprise represent the voice of the customer [Gaskin 1991]. But customers are generally unaware of product or service attributes that determine maintainability, extensibility, or cybersecurity. These factors, the sustainability factors, affect the MICs for technical debt. But customers are acutely aware of capabilities—or missing or defective capabilities. Customer comments and requests are therefore unbalanced in favor of capability over sustainability. The sales and marketing functions tend to accurately transmit this unbalanced perspective to decision makers and technologists.

An analogous mechanism prevails with respect to infrastructure and its internal customers. Internal customers tend to be more concerned with capabilities than they are with sustainability of the processes and systems that deliver those capabilities. Thus, pressure from internal customers tends to emphasize capability at the expense of sustainability. The result of this imbalance is pressure to allocate excessive resources to capability enhancement, compared to activities that improve sustainability. And therefore controlling or reducing technical debt and its MICs gets less attention.

Nor is this the only consequence of feature bias. It provides unrelenting pressure for increasing numbers of features, despite the threats to architectural coherence and overall usability that such “featuritis” or “featurism” present. Featurism leads, ultimately, to feature bloat, and to difficulties for users, who can’t find what they need among the clutter of features that are often too numerous to document. For example, in Microsoft Word, many users are unaware that Shift+F5 moves the insertion point and cursor to the point in the active document that was last edited, even if the document has just been freshly loaded into Word. Useful, but obscure.

Feature bias bias

Feature bias, it must be noted, is subject to biases itself. The existing array of features appeals to a certain subset of all potential customers. Because it is that subset that’s most likely to request repair of existing features. And they’re also the most likely to suggest additional features. The pressure for features tends to be biased in favor of the needs of the most vociferous users. That is, there is pressure to evolve to better meet the needs of existing users. That pressure can force to lower priority any efforts toward meeting the needs of other stakeholders or potential stakeholders. These other stakeholders might be even more important to the enterprise than are the existing users. This bias in feature bias presents another risk that can affect decision makers.

Organizations can take steps to mitigate the risks of feature bias. An example of such a measure might be using focus groups to study how educating customers in sustainability issues affects their perspectives relative to feature bias. Educating decision makers about feature bias can also reduce this risk.

At the enterprise scale, awareness of feature bias would be helpful. But awareness alone is unlikely to counter its detrimental effects. These effects include underfunding technical debt management efforts. Eliminating the source of feature bias is extraordinarily difficult, because customers and potential customers aren’t subject to enterprise policy. Feature bias and feature bias bias are therefore givens. To mitigate the effects of feature bias, we must adopt policies that compel decision makers to consider the need to deal with technical debt.

A possible corrective action

One possible corrective action might be improving accounting practices for MICs, based on historical data. For example, there’s a high probability that any project might produce new technical debt. It might be prudent to fund the retirement of that debt in the form of reserves when we fund projects. And if we know that a project has encountered some newly recognized form of technical debt, it might be prudent to reserve resources to retire that debt as soon as possible. Ideas such as these can rationalize resource allocations with respect to technical debt.

These two examples illustrate what’s necessary if we want to mitigate the effects of feature bias. They also illustrate just how difficult such a task will be.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

The fundamental attribution error

Last updated on July 8th, 2021 at 01:29 pm

When we try to understand the behavior of others, we often make a particularly human mistake. We tend to attribute too much to character and disposition and too little to situation and context. This mistake is so common that it has a name: The Fundamental Attribution Error (FAE) (See “The Fundamental Attribution Error” at my other blog). And although little experimental data is available regarding its effects on technical debt, we can plausibly argue that its effects are significant—and unwelcome.

Arapaho moccasins ca. 1880-1910.
Arapaho moccasins ca. 1880-1910. An American Indian proverb advises, “Don’t judge any man until you have walked two moons in his moccasins.” From the perspective of the FAE, the proverb is a way of mitigating FAE risks. Photo of Arapaho moccasins, ca. 1880-1910 on exhibit at the Bata Shoe Museum, in Toronto, Canada. Photo by Daderot, courtesy Wikimedia
The FAE contributes to technical debt in at least two ways. First, it distorts assessments by non-engineers of the motivations of engineers as they warn of future difficulties from technical debt. Second, it distorts assessments by engineers of the motivations of non-engineers as they oppose allocation of resources to technical debt retirement. They oppose these allocations in order to conserve resources for their own efforts or to accelerate efforts in which they have more immediate interest. The two effects are symmetrical in the large, though not in detail.

Below is a description of the effects of the FAE on engineers and non-engineers. Some of the non-engineers are the internal customers of the engineers. I examine the effects of the FAE that arise from three different claims of the parties to the exchange.

Claim: Technical debt depresses engineering productivity

Many engineers or their managers hold this position.

Engineers notice incidents in which some of the work they must perform on an asset would be much easier or even unnecessary were it not for the technical debt that the asset carries. They sense the burden of the extra effort because they know how much easier and faster the work would be if they could retire the debt.

The internal customers of engineers don’t see these circumstances as clearly as engineers and their managers do. Consequently, they tend to discount engineers’ claims of depressed productivity. Some experience engineers’ complaints, requests, and warnings as whining, self-serving nest feathering, or worse. They tend to attribute engineers’ complaints to faults in the character or “work ethic” of engineers.

Claim: Instead of retiring the technical debt now, just document it for later

This is a suggestion senior managers or the engineers’ internal customers often put forth.

The internal customers of engineers have pressing needs for immediate engineering results. They see new products or repairs to existing products as a means of achieving the objectives the enterprise sets. Focusing limited engineering resources on technical debt retirement conflicts with producing results that would help internal customers of engineers meet these more immediate objectives. As a compromise, non-engineers propose that engineers document instances of technical debt as they find them, so that they can be addressed more efficiently after engineers meet the immediate needs of internal customers.

Engineers discount the validity of this approach for three reasons. First, they don’t experience the pressures their internal customers do. To engineers, their customers’ reports of more pressing needs seem to be merely excuses to get what they want when they want it. Second, the proposed documentation work doesn’t advance the engineers’ customer’s current project toward its objectives. Instead, it actually delays the current project, in ways invisible to non-engineers. These delays induce increases in schedule pressure, and therefore technical debt. The technical debt occurs because the customer of the current project rarely cares enough about the technical debt documentation effort to allow for the extra time it takes. Finally, because many assets evolve continuously, such documentation has a short shelf life. And that limits its value in ways non-engineers might not appreciate.

In these ways, the FAE both creates the documentation suggestion, and limits the ability of engineers to appreciate its motivation. But it also limits the ability of non-engineers to appreciate how limited is the value of the documentation.

Claim: Addressing technical debt is important, but not urgent

Senior managers or the engineers’ internal customers are most common among adherents of this belief.

When the engineering organization presents a business case for investing in addressing technical debt, they aren’t alone. Other functions in the enterprise also make business cases of their own. Too often, these cases are evaluated against each other. Investment in one entails reduced investment in another. But the benefits of technical debt retirement tend to become most visible to non-engineers much later than do the benefits of some other proposals. Sometimes the benefits of technical debt retirement are wholly invisible to non-engineers. For these reasons, technical debt retirement projects tend perhaps more often than most to be deferred at best, or, worse, rejected.

The FAE is in part responsible for the perception of non-engineers that the benefits of technical debt remediation arrive in the distant future. Engineers notice the benefits relatively immediately, because they interact with the rehabilitated assets on a daily basis. Since non-engineers don’t have these experiences, they notice the benefits only upon delivery of the results of engineering work. This mismatch of the timescales of perceptions of engineers and non-engineers prevents non-engineers from perceiving what is in daily evidence to engineers.

Last words

Both engineers and non-engineers are subject to deadlines and resource limitations beyond their control. Their ability to appreciate the challenges their counterparts face is the key to effective collaboration. Too often, neither part feels that it has the time or resources to accommodate the needs of the other.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Separating responsibility for maintenance and acquisition

Last updated on July 8th, 2021 at 01:28 pm

Separating responsibility for maintenance and acquisition or development of technical assets can lead to uncontrolled growth of technical debt. The problem arises when we measure without regard for technical debt the performance of the business acquisition function or the performance of the development organization. In that circumstance, technical debt is likely to expand unchecked. To limit such expansion, policymakers must devise performance measures that hold these organizations accountable for technical debt resulting from their actions.

Software systems

Road damage in Warwick, Rhode Island, resulting from historic storms in March 2010
Road damage in Warwick, Rhode Island, resulting from historic storms in March 2010 [NOAA 2013]. The storms were so severe that the floodwaters overtopped the gauge’s measurable range. Moreover, the National Weather Service (NWS) lacked a database of measurable ranges for flood gauges. Quoting the NWS report: “A lesson learned here was that maximum recordable river levels should be known by NWS staff, not only so staff aren’t surprised when this type of issue arises, but also to notify USGS personnel so that they can install a temporary gage and remove or elevate threatened equipment.” Technical debt, if ever I’ve seen it.
For systems consisting solely of software, separating responsibility for maintenance and acquisition or system development is risky. It enables the acquiring organization to act with little regard for the consequences of its decisions vis-à-vis maintenance matters [Boehm 2016]. This is unfortunate—it increases the rate of accumulation of new technical debt. And it increases the lifetime of legacy technical debt. This happens more frequently when the acquiring organization doesn’t suffer the MICs associated with the technical debt.

For example, a focus on performance of the organization that’s responsible for acquisition biases them in favor of attending to the direct and immediate costs of the acquisition. They are likely to have little or no regard for ongoing maintenance issues. The maintenance organization must then deal with whatever the acquired system contains (or lacks).

An analogous mechanism operates for organizations that develop, market, and maintain products or services. When there are software elements in their respective infrastructures, separation of the development function from the maintenance function enables the development function to act independently of the maintenance consequences of its decisions.

Systems that include hardware

But the separation-of-responsibilities mechanism that leads to uncontrolled technical debt isn’t restricted to software. Any technological asset that has ongoing maintenance needs (and most of them do) can potentially present this problem.

For example, in the United States, and many other countries, two streams of resources support publicly-owned infrastructure [Blair 2017]. The funding stream covers construction, operations and maintenance, and repairs. Its usual sources are taxes, tolls, licenses, other user fees, sale of ad space, and so on. The financing stream covers up-front construction costs, to bridge the period from conception through construction, until the funding stream begins delivering resources. The financing stream usually comes from bond sales.

Although legislatures, or agencies they establish, control both streams of resources, the effects of the streams differ fundamentally. The financing stream is dominant during construction and the early stages of the asset’s lifecycle. The funding stream is dominant after that—when maintenance and operations are most important. Legislators and agencies are generally reluctant to supply funding because of the impact on taxpayers and users. Legislators and agencies find financing much more palatable. For this reason, among others, funds for U.S. infrastructure maintenance are generally insufficient, and technical debt gradually accumulates.

So it is with technological assets in organizations. For accounting purposes, capital expenses are treated differently from operational expenses. The result is that operational expenses can have a more significant impact on current financial results than capital expenses do. This leads organizations to underfund operations and maintenance, which contributes to technical debt accumulation.

Last words

Control of new technical debt accumulation and enhancement of technical debt retirement rates is possible only if we can somehow hold accountable the acquisition or development organizations for the MICs that result from their actions. Securitization of the debt incurred, as I’ll address in a forthcoming post, is one possible means of imposing this accountability. But reserves are also required, because some of the debt incurred might not be known at the time the asset is acquired or created.

Separating responsibility for maintenance and acquisition or system development is actually a form of stovepiping. See “Stovepiping can lead to technical debt” for more on stovepiping.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.

Available: here; Retrieved: January 29, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[NOAA 2013] NOAA/National Weather Service. “The March, 2010 Floods in Southern New England,” WFO Taunton Storm Series Report #2013-01, January 2013.

Available: here; Retrieved: January 30, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Unrealistic definition of done

Last updated on July 8th, 2021 at 01:20 pm

Many an enterprise culture includes, perhaps tacitly, an unrealistic definition of done for projects. Some enterprise cultures assume definitions of done that fail to adequately acknowledge attributes related to sustainability. For such cultures, technical debt expands inexorably. In most organizations, the definition of done includes meeting the attributes that most internal customers understand and care about. These attributes might not include sustainability [Guo 2011]. Indeed, even among technologists, the definition of done might not enjoy precise consensus [Wake 2002].

Why retiring technical debt isn’t included in “done”

The 2009 Ford Focus SES coupe (North America) engine bay. Its design is “done” in the sense that it’s available to consumers.
The 2009 Ford Focus SES coupe (North America) engine bay. Typical owners can no longer learn how to maintain their own vehicles. Engines have become so complex that even experienced mechanics must train to maintain the engines they work on. Since these vehicles are available for sale to consumers, clearly their manufacturers regard their designs as “done.” But is technical debt a factor in the growing complexity of modern engines? It’s probably present in their software, and it would be most surprising if we found no technical debt in the mechanical design. Photo (cc) Porsche997SBS courtesy Wikimedia.
Internal customers understand less well the attributes of deliverables related to sustainability. It’s therefore perhaps unsurprising that sustainability might not receive the attention it needs. Applying scarce resources to enhance attributes the customer doesn’t understand, and cares about less, will always be difficult.

To gain control of technical debt, we must redefine done to include addressing sustainability of deliverables. Although there may be many ways to accomplish this, none will be easy. Resolution will necessarily involve educating internal customers to understand enough about sustainability to enable them to justify paying for it.

Redefining “done”

The typical definition of done for most projects ensures only that the deliverables meet the requirements. Because requirements usually omit reference to retiring newly incurred nonstrategic technical debt, we often declare projects complete with incremental technical debt still in place. A similar problem prevails with respect to legacy technical debt.

A more insidious form of this problem is intentional shifting of the definition of done. This can happen when the organization has adopted a reasonable definition of done that allows for addressing sustainability. But under severe time pressure, the definition is “temporarily” amended to allow the team to declare the effort complete, even though sustainability issues remain unaddressed.

For most projects, three conditions conspire to create steadily increasing levels of nonstrategic technical debt. First, for most tasks, the definition of done is that the deliverables meet the project objectives, or at least, they meet them well enough. Second, typical project objectives don’t restrict levels of newly incurred nonstrategic technical debt, nor do they demand retirement of incidentally discovered legacy technical debt. Third, budget authority usually terminates upon acceptance of delivery. These three conditions, taken together, restrain engineering teams from immediately retiring any debt they incur. Nor can they retire—or document or report—any legacy technical debt they encounter while fulfilling other requirements.

For example, for one kind of incremental technical debt—what Fowler calls [Fowler 2009] Inadvertent/Prudent (“Now we know how we should have done it”)—the realization that we’ve incurred new debt often occurs after the task is “done.” If budget authority has terminated, there are no resources available—financial or human—to retire that form of technical debt.

Last words

Unless team members document the technical debt they create or encounter, there is risk of lost knowledge. After team members move on to their next assignments the enterprise is likely to lose track of the location and nature of that debt. A more realistic definition of done would enable the team to continue working post-delivery to retire or document any newly incurred nonstrategic technical debt. They could also note any incidentally encountered legacy technical debt. Moreover, teams most likely leave in place any strategic technical debt—technical debt incurred intentionally for strategic reasons. Although the enterprise must eventually address such debt as well, the widespread definition of done doesn’t address it.

Policymakers are well positioned to advocate for the culture transformation needed to redefine done.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.

Available: here; Retrieved: January 29, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[NOAA 2013] NOAA/National Weather Service. “The March, 2010 Floods in Southern New England,” WFO Taunton Storm Series Report #2013-01, January 2013.

Available: here; Retrieved: January 30, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wake 2002] Bill Wake. “Coaching Drills and Exercises,” XP123 Blog, June 15, 2002.

Available: here

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Stovepiping can lead to technical debt

Last updated on July 8th, 2021 at 01:18 pm

Stovepiping can lead to technical debt. Actual stovepipes are the tubes that vent exhaust from stoves. These tubes serve as a metaphor for the flow of information in “stovepiped” organizations. In stovepiped organizations, information can flow predominantly (or only) up or down along the parallel chains of command. But information can flow only rarely (or never) from a point in one chain of command across to some other chain of command [Waters 2010]. The stovepipe metaphor is imperfect, in the sense that in actual stovepipes, smoke and fumes rarely flow downwards. By contrast, in organizations, some information does flow down the chains of command. But the metaphor does clarify the problem of limited flow of information. Transferring whatever the organization learns in one metaphorical stovepipe into other metaphorical stovepipes is difficult.

Two forms of stovepiping

The stovepipes in a wood-burning stove in a farm museum
A wood-burning stove in a farm museum in Lower Bavaria (German: Niederbayern). Lower Bavaria is one of the seven administrative regions of Bavaria, Germany. The stovepipe, which is the black tube running upwards from the stove, channels smoke and fumes out of the kitchen into the chimney.
Stovepiping can occur in both organizational structures and in engineered systems. These two forms of stovepiping are intimately related, and both can lead to uncontrolled formation of new technical debt, or increased persistence of existing technical debt.

In organizational structures, stovepiping occurs when elements of different organizational units with similar capabilities act relatively independently. An example is the dispersal of some elements of the IT function out into IT’s customers. When independent organizations have similar technical needs, they’re at risk of generating new technical debt. The debt they generate results from independently implementing technological capabilities that duplicate each other.

Stovepiping occurs in engineering, for example, when the organization manages and maintains independently two distinct technological assets [McGovern 2003]. In that situation, distinct engineering efforts working on those assets might happen to solve the same problem, possibly in two different ways. Then each party might be either ignorant or possibly disparaging, of the other’s efforts.

How stovepiping relates to technical debt

In whichever way duplication of technological capability comes about, it can increase levels of technical debt. Alternatively, it can increase persistence of existing technical debt. These effects happen because the organization might need to execute future maintenance or enhancement efforts multiple times—once for each instance of the technical artifact. That exposes the organization to additional cost, additional load on its staff, and additional risk of creating defects and incurring liability. Compare this situation to one in which all units that need a particular asset share it. Duplication is expensive.

The problem is actually even more worrisome. First, suppose there exists a defect in one version of a technological artifact. The people who are aware of the defect might not realize that another version of the artifact exists. If that second version also has an analogous defect, its defect might go unrecognized for some time, with all the usual attendant negative consequences. Second, suppose there is a necessary extension of the artifact’s capabilities. The maintainers of one version might recognize the need for the extension and implement it. Meanwhile, the maintainers of other versions might not recognize the need for the extension. They might not take action until something bad happens or a possibly urgent need arises. It’s easy to conjure other unfavorable—and costly—scenarios.

Stovepiping in technological systems

In engineering more generally, stovepiping can occur internally in systems, even though only one business unit is involved, and even though the stovepiped artifacts serve purposes invisible to the world outside the system. This can occur whenever there is weak communication between the teams designing or maintaining the portions of the system that host the similar artifacts. For readers familiar with the Apollo XIII incident, the incompatibility of the two carbon dioxide scrubbers in the command module and the lunar excursion module serves as an example of the risks of technical stovepiping.

When distinct business units or functions operate their own engineering or IT organizations, there’s an elevated probability of duplicating technological assets. The same effect can occur when they depend on a shared engineering function but require similar work. This happens when the organizational structure or the timing of the work encourages separate engineering efforts. Engineering or IT functions operated separately under the control of distinct business units or functions can clearly produce duplicated capabilities. However, duplication can also occur when the engineering function is shared across distinct business units or functions. This happens when the actual people and teams performing the work differ for different efforts. And it can happen too when communication is weak between those teams, whether or not the efforts are conducted contemporaneously.

Last words

Because identifying these forms of technical debt after they appear is notoriously difficult, preventing their formation is preferable. Prevention is possible if the enterprise establishes mechanisms that facilitate consultation and sharing among elements of different, separately operated technology development or maintenance functions. In other words, the organization must “break” the stovepipes—no mean feat, politically speaking.

Another challenge, of course, is providing resources for such sharing mechanisms, because preventing technical debt is rarely recognized as a value generator. If it were so recognized, the resources would likely appear. Changes in cost accounting might make such recognition more likely.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.

Available: here; Retrieved: January 29, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.

Order from Amazon

Cited in:

[NOAA 2013] NOAA/National Weather Service. “The March, 2010 Floods in Southern New England,” WFO Taunton Storm Series Report #2013-01, January 2013.

Available: here; Retrieved: January 30, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wake 2002] Bill Wake. “Coaching Drills and Exercises,” XP123 Blog, June 15, 2002.

Available: here

Cited in:

[Waters 2010] Donald Waters. Global Logistics: New Directions In Supply Chain Management, 6th Edition, London: Kogan Page Limited, 2010.

Order from Amazon

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Zero tolerance and work-to-rule create adversarial cultures

Last updated on July 8th, 2021 at 01:16 pm

Defining technical debt explicitly enough for project objectives is difficult. Confronted with this difficulty, some internal customers of technologists adopt a zero-tolerance approach to technical debt. But rarely do they define technical debt explicitly. Post-delivery, when customers discover technical debt, they hold technologists responsible. They do so even in cases when no one could have predicted that a specific artifact would eventually become technical debt. This establishes an adversarial culture in which technologists contend with their internal customers. To defend themselves, technologists sometimes adopt a work-to-rule approach to their work.

How adversarial cultures develop

Delayed or cancelled flights can indicate that pilots or others are engaged in a work-to-rule action
Trouble at the airport. When airline pilots engage in work-to-rule actions, the immediate result can be large numbers of delayed or cancelled flights. The longer-term result might be beneficial to pilots, the airline, and the public, but only if labor peace can be restored, and the damage to the flying public can be overcome. So it is with work-to-rule deliveries as a way of dealing with zero-tolerance technical debt policies. The organization must overcome the adversarial culture that results from indiscriminate attempts to control technical debt. Technologists do gain some measure of protection by working to rule, but the longer-term benefit of the organization’s learning to manage technical debt arrives only if the adversarial culture can be overcome. Image (cc) Hotelstvedi courtesy Wikimedia.
And that’s when the trouble begins.

Within this adversarial dynamic, technologists try to protect themselves against future recriminations by “working to rule.” They perform only work that the internal customer specifies. If they find something else that needs work, they perform that work only if they successfully obtain the customer’s approval. Some customers continue to adhere to zero-tolerance policies with respect to technical debt. But engineers cannot meet such non-specific requirements. Because technologists are “working to rule,” they use the ambiguity of the zero-tolerance requirement to assert that they performed all explicitly specified work. This level of performance is analogous to the work-to-rule actions of some employees involved in labor disputes with their employers. In these actions, employees are literally in compliance with the requirements of the employer, but only literally [LIBCom 2006].

Preventing formation of an adversarial culture

Requiring that deliverables be free of technical debt contributes to formation of an adversarial culture. In such cultures the adversaries are the technologists and their internal customers. Shedding that adversarial culture can be difficult once it sets in. Compelling employees, vendors, or contractors to deliver work that’s free of all technical debt is therefore unlikely to succeed. Whether employees or contractors perform the work in-house, or the work is outsourced, the results can be problematic. In the context of an adversarial culture, deliverables that meet the minimum possible interpretation of the stated objectives are almost certain to carry unacceptable levels of technical debt. What can we do to prevent this?

To avoid creating an adversarial culture, we can specify in project objectives the total removal of some kinds of technical debt. To ensure steady progress, develop a statement of objectives that includes complete retirement of at least one well-defined class of technical debt. Emphasize debt classes that have the highest anticipated MICs in the near term. Address other well-defined classes of technical debt on a best-effort basis.

We must accept as the “cost of doing business,” any other forms of technical debt that remain at the end of a given project. We must also accept that some artifacts might later become technical debt, even though they aren’t at present. We’ll get to them, but unfortunately, not this time.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.

Available: here; Retrieved: January 29, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[LIBCom 2006] “Work-to-rule: a guide.” libcom.org.

Available: here; Retrieved: May 9, 2017.

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.

Order from Amazon

Cited in:

[NOAA 2013] NOAA/National Weather Service. “The March, 2010 Floods in Southern New England,” WFO Taunton Storm Series Report #2013-01, January 2013.

Available: here; Retrieved: January 30, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[Wake 2002] Bill Wake. “Coaching Drills and Exercises,” XP123 Blog, June 15, 2002.

Available: here

Cited in:

[Waters 2010] Donald Waters. Global Logistics: New Directions In Supply Chain Management, 6th Edition, London: Kogan Page Limited, 2010.

Order from Amazon

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Performance management systems and technical debt

Last updated on July 7th, 2021 at 07:57 pm

Treats are a performance management system for dogs
A dog receiving a reward. Many performance management systems implement a model that assumes that the correct configuration of incentives and disincentives will produce the desired result. This theory is questionable.

Few performance management systems provide guidance with respect to behaviors relating to technical debt. One reason, perhaps, is that technical debt isn’t widely understood. Or perhaps only engineers and their managers regard technical debt as a concern. Still, to gain control of technical debt organizations must ensure that performance standards are clear. They must clearly state expectations with respect to behaviors that could affect technical debt. If changes are necessary, policymakers can be effective advocates—provided that they understand what the appropriate role for performance management is in controlling technical debt. This post should be helpful.

A fundamental premise of many performance management systems is that incentives can encourage desirable behavior. Likewise, disincentives can discourage undesirable behavior. Unfortunately, serious questions have arisen about the effectiveness of these behavioral control mechanisms in general [Kohn 1999]. Employees find ways to harvest incentives without exhibiting the desired behavior. Similarly, they find ways to circumvent disincentives while continuing to engage in undesired behavior.

Behavioral control for technical debt is problematic

Moreover, specifically for technical debt management, behavioral control is especially problematic. Troubles arise because some of the desirable behaviors are inherently immeasurable. For example, the design of an incentive structure to encourage legacy technical debt retirement is debatable. The technical difficulties involved relate to the problems of defining legacy technical debt.

Managing performance vis-à-vis technical debt, therefore, presents a problem of the kind Austin calls partially supervised [Austin 1996]. Supervising engineers whose work can affect technical debt can only be partial. The issue is that measuring technical debt is only partially practical given the state of the art. Austin shows how partial supervision frequently leads to dysfunctional performance management. But the problem is especially vexing for managing technical debt. For example, engineers’ work can sometimes incur technical debt that remains unrecognized for months or years after the work is completed. To fully supervise such work would require inventing retroactive incentives and disincentives. This class of motivators doesn’t exist, but even if they did, they’re of questionable legality in most jurisdictions.

The doctrine of commander’s intent

Although incentives and disincentives cannot serve to manage performance relative to technical debt, a very effective model is available. Enterprise leaders could communicate their intentions relative to technical debt, and empower the people of the organization to take steps to reduce debt. In the United States military, and others as well, a doctrine that implements this approach is called commander’s intent [Mattis 2008][US Army 2010].

Gen. Mattis offers five principles that guide what the military calls “effect-based operations.” For technical debt management, the effect we seek is rational control of the technical debt portfolio. Here are his five principles, transformed to the field of technical debt.

  1. Technology development, maintenance, and cyberdefense in the future will require a balance of conventional and unconventional approaches.
  2. Technology evolves rapidly, and we must be willing to adapt our methods.
  3. Technologies are dynamic, with an infinite number of variables; therefore, it isn’t scientifically possible to accurately predict the level of technical debt that will result from any given effort. To suggest otherwise runs contrary to historical experience and the nature of modern technological assets.
  4. We are in error when we think that what works (or does not work) in efforts involving one technology in one enterprise will be universally applicable to all technologies in all enterprises.
  5. Finally, to paraphrase General Sherman, “Every attempt to make technical debt management easy and safe will result in humiliation and disaster.”

Limitations of the doctrine of commander’s intent

Most organizations rely on supervisors to communicate the analog of commander’s intent to their subordinates. Currently, it’s fair to say that few supervisors outside the technology-oriented elements of the enterprise communicate much about technical debt to their subordinates.

That situation might explain why most performance management systems encourage behaviors that unwittingly expand the body of technical debt, especially for non-technologist performers. There are situations in which the widely applauded actions of the outstanding performer actually incur technical debt strategically and responsibly. Technical debt so incurred is what McConnell calls Type II [McConnell 2008] and what Fowler calls Deliberate and Prudent [Fowler 2009]. But most performance management systems, especially for non-technologists, say nothing about technical debt. They risk encouraging behaviors that indirectly exacerbate the problems associated with technical debt.

An illustrative story

Distinguishing responsible and irresponsible behaviors is possible only if understanding of the nature of technical debt is widespread in the organization, even beyond the technologists. Here’s an example:

It was ambitious. It was what advocates called a “stretch goal.” But the VP of Marketing approved the plan to release the new app by the end of the fiscal quarter. After a month of meetings, and much jawboning, the CTO agreed. The VP of New Product had serious objections, but the executive team set them aside. Engineers and testers were able to meet the date, but they had to incur significant technical debt. When they asked for resources to retire that debt after the release, the VP of Marketing opposed the request. She needed additional resources for the promotional campaign due to our late entry into the market.

Stories like this illustrate scenarios in which technical debt considerations are subordinate to “business priorities.” These latter include market timing, market development, and revenue generation. Standards for setting priorities closely parallel the standards defined in the performance management system. Indeed, performance management should support enterprise goals. In the scenario above, the organization might meet the immediate goal of a successful release. But it does so by incurring technical debt, thereby imperiling the next release. This scenario illustrates why changing the performance management system might achieve a better balance between immediate goals responsible technical debt management.

Last words

Since anyone in the enterprise can take actions or make decisions that lead to incurring new technical debt, or cause existing technical debt to remain in place, organizations need performance standards that guide employees with respect to technical debt. To provide guidance for distinguishing responsible behavior from irresponsible behavior, performance management systems must acknowledge the potential of any employee to affect technical debt. Performance management systems must be reviewed with respect to alignment with technical debt policy. They might then support a mechanism analogous to Gen. Mattis’s vision of commander’s intent.

References

[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6

Contains an extensive discussion of the consequences of partial supervision of performance. Since technical debt can only be partially supervised, the concept is relevant to understanding the effects of performance management systems on technical debt. Order from Amazon

Cited in:

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.

Available: here; Retrieved: January 29, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.

Cited in:

[Hamburger 1973] Henry Hamburger. “N-person Prisoner’s Dilemma,” Journal of Mathematical Sociology, 3, 27–48, 1973. doi:10.1080/0022250X.1973.9989822

Cited in:

[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.

Available: here; Retrieved: September 19, 2017

Cited in:

[Kahneman 1979] Daniel Kahneman and Amos Tversky, “Intuitive Prediction: Biases and Corrective Procedures,” Management Science 12, 313-327, 1979.

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1

Order from Amazon

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[LIBCom 2006] “Work-to-rule: a guide.” libcom.org.

Available: here; Retrieved: May 9, 2017.

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Li 2015] Zengyang Li, Paris Avgeriou, and Peng Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here; Retrieved November 9, 2017.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.

Order from Amazon

Cited in:

[NOAA 2013] NOAA/National Weather Service. “The March, 2010 Floods in Southern New England,” WFO Taunton Storm Series Report #2013-01, January 2013.

Available: here; Retrieved: January 30, 2018

Cited in:

[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.

Available: here; Retrieved: July 10, 2017

Cited in:

[US Army 2010] U.S. Army (2010) Field Manual 5.0 – The Operations Process U.S. Department of the Army.

Describes the concept, value, and importance of the doctrine of commander’s intent. See the index for “commander’s intent,” and especially paragraphs 2-90 and 2-91. Available: here; Retrieved: Dec. 22, 2019.

Cited in:

[Wake 2002] Bill Wake. “Coaching Drills and Exercises,” XP123 Blog, June 15, 2002.

Available: here

Cited in:

[Waters 2010] Donald Waters. Global Logistics: New Directions In Supply Chain Management, 6th Edition, London: Kogan Page Limited, 2010.

Order from Amazon

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Show Buttons
Hide Buttons