Confirmation bias is a cognitive bias [Kahneman 2011]. It’s the human tendency to favor and seek only information that confirms our preconceptions, or to avoid information that disconfirms them. For example, the homogeneity of cable news channel audiences, and the alignment between preconceptions of the audience and the slant of the newscast for that channel, are results of confirmation bias.
Computer-generated image of the third stage ignition, sending the Mars Climate Orbiter (MCO) to Mars in December, 1998. The spacecraft eventually broke up in the Martian atmosphere as a result of what is now often called the “metric mix-up.” The team at Lockheed Martin that constructed the spacecraft and wrote its software used Imperial units for computing thrust data. But the team at JPL that was responsible for flying the spacecraft was using metric units. The mix-up was discovered after the loss of the spacecraft by the investigation panel established by NASA. One of the many operational changes deployed as a result of this loss was increased use of reviews and inspections. While we do not know why reviews and inspections weren’t as thorough before the loss of the MCO as they are now, one possibility is the effects of confirmation bias in assessing the need for reviews and inspections. Image courtesy Engineering Multimedia, Inc., and U.S. National Aeronautics and Space AdministrationConfirmation bias causes technical debt by biasing the information on which decision makers base decisions involving technical debt. Most people in these roles have objectives they regard as having priority over eliminating technical debt. This causes them to bias their searches for information about technical debt in favor of information that would support directly the achievement of those primary objectives. They tend, for example, to discount warnings of technical debt issues, to underfund technical debt assessments, and to set aside advice regarding avoiding debt formation in current projects.
For example, anyone determined to find reasons to be skeptical of the need to manage technical debt need only execute a few Google searches. Searching for there is no such thing as technical debt yields about 300,000 results at this writing; technical debt is a fraud about 1.6 million; and technical debt is a bad metaphor about 3.7 million. Compare this to technical debt which yields only 1.6 million. A skeptic wouldn’t even have to read any of these pages to come away convinced that technical debt is at best a controversial concept. This is, of course, specious reasoning, if it’s reasoning at all. But it does serve to illustrate the potential for confirmation bias to contribute to preventing or limiting rational management of technical debt.
Detecting confirmation bias in oneself is extraordinarily difficult because confirmation bias causes us to (a) decide not to search for data that would reveal confirmation bias; and (b) if data somehow becomes available, to disregard or to seek alternative explanations for it if that data tends to confirm the existence of confirmation bias. Moreover, another cognitive bias known as the bias blind spot causes individuals to see the existence and effects of cognitive biases much more in others than in themselves [Pronin 2002].
Still, the enterprise would benefit from monitoring the possible existence and effects of confirmation bias in decisions with respect to allocating resources to managing technical debt. Whenever decisions are made, we must manage confirmation bias risk.
References
[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
In a 1977 report, Daniel Kahneman and Amos Tversky identify one particular cognitive bias [Kahneman 2011], the planning fallacy, which afflicts planners [Kahneman 1977] [Kahneman 1979]. They discuss two types of information planners use. Singular information is specific to the case at hand; distributional information is drawn from similar past efforts. The planning fallacy is the tendency of planners to pay too little attention to distributional evidence and too much to singular evidence, even when the singular evidence is scanty or questionable. Failing to harvest lessons from the distributional evidence, which is inherently more diverse than singular evidence, the planners tend to underestimate cost and schedule. So for any given project, there’s an inherent tendency in human behavior to promise lower costs, faster delivery, and greater benefits than anyone can reasonably expect.
Aerial view of Hoover Dam, September 2017. Under construction from 1931 to 1936, the dam was built for $48.8M ($639M in 2016 dollars) under a fixed-price contract. It was completed two years ahead of schedule. Apparently the planning fallacy doesn’t act inevitably. 112 men died in incidents associated with its construction. 42 more died of a condition diagnosed as pneumonia, but which is now thought to have been carbon monoxide poisoning due to poor ventilation in the dam’s diversion tunnels during construction. There’s little doubt that unrealistic optimism affects not only projections of budget and schedule, but also projections of risks, including deaths. Photo (cc) Mariordo (Mario Roberto Durán Ortiz), courtesy Wikimedia Commons.But the problem is exacerbated by a dynamic described by Boehm et al. [Boehm 2016], who observe that because organizational resources are finite, project sponsors compete with each other for resources. They’re compelled by this competition to be unrealistically optimistic about their objectives, costs, and schedules. Although Boehm et al. call this mechanism the “Conspiracy of Optimism,” possibly facetiously, it isn’t actually a conspiracy. Rather, it’s a variant of the N-Person Prisoner’s Dilemma [Hamburger 1973].
Unrealistic optimism creates budget shortfalls and schedule pressures, both of which contribute to conditions favorable for creating non-strategic technical debt. And the kinds of technical debt produced by this mechanism, or any mechanism associated with schedule or budget pressure, tend to be subtle — they’re the types least likely to become evident in the short term. For example, technical debt that might make a particular kind of enhancement more difficult in the next project is more likely to appear than technical debt in the form of a copy of some code that should have been replaced by a utility routine. Copies of code are more easily discovered and more likely to be retired in the short term, if not in the current project. Awkward architecture might be more difficult to identify, and is therefore more likely to survive in the intermediate or long term.
In other words, the forms of technical debt most likely to be generated are those that are the most benign in the short term, and which are therefore more likely to escape notice. If noticed, they’re more likely to be forgotten unless carefully documented, an action that’s unlikely to be taken under conditions of schedule and budget pressure. In this way, the non-strategic technical debt created as a result of unrealistic optimism is more likely than most technical debt to eventually become legacy technical debt.
Policymakers can assist in addressing the consequences of unrealistic optimism by advocating for education about it. They can also advocate for changes in incentive structures and performance management systems to include organizational standards with respect to realism in promised benefits, costs, and schedules.
References
[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
Enterprise decision-makers affected by feature bias tend to harbor distorted views of the importance of new capability development compared to technical debt management. This tendency is likely due to the customer’s relative sensitivity to features, and relative lack of awareness of sustainability. Whatever the cause, customers tend to be more attracted to features than they are to indicators of sound technical debt management and other product sustainability practices. This tendency puts decision-makers at risk of feature bias: unbalanced concern for capability vs. sustainability.
Alaska crude oil production 1990-2015. This chart [Yen 2015] displays Alaska crude oil produced and shipped through the Trans Alaska Pipeline System (TAPS) from 1990 to 2015. Production had dropped by 75% in that period, and the decline is projected to continue. In January 2018, in response to pressure from Alaskan government officials and the energy industry, the U.S. Congress passed legislation that opened the Arctic National Wildlife Refuge to oil exploration, despite the threat to ecological sustainability that exploration poses. If we regard TAPS as a feature of the U.S. energy production system, we can view its excess capacity as a source of feature bias bias, creating pressure on decision-makers to add features to the U.S. energy system instead of acting to enhance the sustainability of Alaskan and global environmental systems [Wight 2017].Changes in cost accounting could mitigate some of this feature bias by projecting more accurately total MICs based on historical data and sound estimation. I’ll explore possible accounting changes later in this post, and in future posts; meanwhile, let’s explore the causes and consequences of the distorted perspective I call feature bias.
For products or services offered for sale outside the enterprise, the sales and marketing functions of the enterprise represent the voice of the customer [Gaskin 1991]. But customers are generally unaware of product or service attributes that determine maintainability, extensibility, or cybersecurity — all factors that affect the MICs for technical debt. On the other hand, customers are acutely aware of capabilities — or missing or defective capabilities — in products or services. Customer comments and requests, therefore, are unbalanced in favor of capabilities as compared to maintainability, extensibility, cybersecurity, and other attributes related to sustainability. The sales and marketing functions tend to accurately transmit this unbalanced perspective to decision-makers and technologists.
An analogous mechanism prevails with respect to infrastructure and the internal customers of that infrastructure. Internal customers tend to be more concerned with capabilities — and missing capabilities — than they are with sustainability of the processes and systems that deliver those capabilities. Thus, pressure from internal customers on the developers and maintainers of infrastructure elements tends to emphasize capability at the expense of sustainability. The result of this imbalance is pressure to allocate excessive resources to capability enhancement, compared to activities that improve maintainability, extensibility, or cybersecurity, and which therefore would aid in controlling or reducing technical debt and its MICs.
Nor is this the only consequence of feature bias. It provides unrelenting pressure for increasing numbers of features, despite the threats to architectural coherence and overall usability that such “featuritis” or “featurism” present. Featurism leads, ultimately, to feature bloat, and to difficulties for users, who can’t find what they need among the clutter of features that are often too numerous to document. For example, in Microsoft Word, many users are unaware that Shift+F5 moves the insertion point and cursor to the point in the active document that was last edited, even if the document has just been freshly loaded into Word. Useful, but obscure.
Feature bias, it must be noted, is subject to biases itself. The existing array of features appeals to a certain subset of all potential customers. Because it is that subset that’s most likely to request repair of existing features, or to suggest additional features, the pressure for features tends to be biased in favor of the needs of the most vociferous segments of the existing user base. That is, systems experience pressure to evolve to better meet the needs of existing users, in preference to meeting the needs of other stakeholders or potential stakeholders who might be even more important to the enterprise than are the existing users. This bias in feature bias presents another risk that can affect decision-makers.
Organizations can take steps to mitigate the risk of feature bias. An example of such a measure might be the use of focus groups to study how education in sustainability issues affects customers’ perspectives relative to feature bias. Educating decision makers about feature bias can also reduce this risk.
At the enterprise scale, awareness of feature bias would be helpful, but awareness alone is unlikely to counter its detrimental effects, which include underfunding of technical debt management efforts. Eliminating the source of feature bias is extraordinarily difficult, because customers and potential customers aren’t subject to enterprise policy. Feature bias and feature bias bias are therefore givens. To mitigate the effects of feature bias, we must adopt policies that compel decision-makers to consider the need to deal with technical debt. One possible corrective action might be improvement of accounting practices for MICs, based, in part, on historical data. For example, since there’s a high probability that any project might produce new technical debt, it might be prudent to fund the retirement of that debt, in the form of reserves, when we fund the project. And if we know that a project has encountered some newly recognized form of technical debt, it might be prudent to reserve resources to retire that debt as soon as possible. Ideas such as these can rationalize resource allocations with respect to technical debt.
These two examples illustrate what’s necessary if we want to mitigate the effects of feature bias. They also illustrate just how difficult such a task will be.
References
[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
When we try to understand the behavior of others, we often make a particularly human mistake. We tend to attribute too much to character and disposition and too little to situation and context. This mistake is so common that it has a name: The Fundamental Attribution Error (FAE) (See “The Fundamental Attribution Error” at my other blog). And although little experimental data is available regarding its effects on technical debt, we can plausibly argue that its effects are significant — and unwelcome.
Arapaho moccasins ca. 1880-1910. An American Indian proverb advises: “Don’t judge any man until you have walked two moons in his moccasins.” Viewed from the perspective of the Fundamental Attribution Error, the proverb is a way of mitigating its risks. Photo of Arapaho moccasins, ca. 1880-1910 on exhibit at the Bata Shoe Museum, in Toronto, Canada, taken by Daderot, courtesy WikimediaThe FAE contributes to technical debt in at least two ways. First, it distorts assessments by non-engineers of the motivations of engineers as they warn of future difficulties arising from accumulating technical debt. Second, it distorts assessments by engineers of the motivations of non-engineers as they oppose allocation of resources to technical debt retirement efforts in order to conserve resources for their own efforts or to accelerate efforts in which they have more immediate interest. The two effects are symmetrical in the large, though not in detail.
Below is a description of the effects of the FAE on engineers and non-engineers, some of whom are the internal customers of the engineers. Let’s examine the effects of the FAE that are precipitated by three different claims or positions of the parties to the exchange.
Technical debt depresses engineering productivity
This is a position often held by engineers or their managers.
Engineers notice incidents in which some of the work they must perform on an asset would be much easier or even unnecessary were it not for the technical debt that the asset carries. They sense the burden of the extra effort because they know how much easier and faster the work would be if they could retire the debt.
The internal customers of engineers don’t see these circumstances as clearly as engineers (and their managers) do. Consequently, they tend to discount engineers’ claims of depressed productivity. Some experience engineers’ complaints, requests, and warnings as whining, self-serving nest feathering, or worse. They tend to attribute engineers’ complaints to faults in the character or “work ethic” of engineers.
Instead of retiring the technical debt now, just document it for later
This is a suggestion often put forth by senior managers or the engineers’ internal customers.
The internal customers of engineers have pressing needs for immediate engineering results. They see new products or repairs to existing products as a means of achieving the objectives the enterprise sets. Focusing limited engineering resources on technical debt retirement conflicts with producing results that would help internal customers of engineers meet these more immediate objectives. As a compromise, non-engineers propose that engineers document instances of technical debt as they find them, so that they can be addressed more efficiently after engineers meet the immediate needs of internal customers.
Engineers discount the validity of this approach for three reasons. First, they don’t experience the pressures their internal customers do. To engineers, their customers’ reports of more pressing needs seem to be merely excuses to get what they want when they want it. Second, the proposed documentation work doesn’t advance the engineers’ customer’s current project toward its objectives. Instead, it actually delays the current project, even though non-engineers aren’t aware of the extent of this effect. These delays induce increases in schedule pressure — and therefore technical debt — because the customer of the current project rarely cares enough about the technical debt documentation effort to allow for the extra time it takes. Finally, because many assets evolve continuously, such documentation has a very short shelf life, which limits its value in ways non-engineers might not appreciate.
In these ways, the FAE both creates the documentation suggestion, and limits the ability of engineers to appreciate its motivation, while it also limits the ability of non-engineers to appreciate how limited the value of the documentation is.
Addressing technical debt is important, but not urgent
Senior managers or the engineers’ internal customers are most common among adherents of this belief.
When the engineering organization presents a business case for investing time and resources in addressing technical debt issues, other functions in the enterprise also make business cases of their own. Too often, these cases are evaluated against each other. Investment in one entails reduced investment in another. And since the benefits of technical debt retirement tend to become most visible to non-engineers much later than do the benefits of some other proposals, technical debt retirement projects tend perhaps more often than most to be deferred at best, or, often, rejected.
The FAE is in part responsible for the perception of non-engineers that the benefits of technical debt remediation arrive in the distant future. Engineers notice the benefits relatively immediately, because they interact with the rehabilitated assets on a daily basis. Since non-engineers don’t have these experiences, they notice the benefits only upon delivery of the results of engineering work. This mismatch of the timescales of perceptions of engineers and non-engineers prevents non-engineers from perceiving what is in daily evidence to engineers.
Both engineers and non-engineers are subject to deadlines and resource limitations beyond their control. Their ability to appreciate the challenges their counterparts face is the key to effective collaboration. Too often, neither part feels that it has the time or resources to accommodate the needs of the other.
References
[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Separating responsibility for maintenance and acquisition of technical assets can lead to uncontrolled growth of technical debt. When the performance of the business acquisition function or the performance of the development organization is measured without regard for the technical debt that arises as a consequence of their actions, technical debt is likely to expand unchecked. To limit such expansion, policymakers must devise performance measures that hold these organizations accountable for the technical debt arising from their actions.
Road damage in Warwick, Rhode Island, resulting from historic storms in March 2010 [NOAA 2013]. The storms were so severe that at least one river flood gauge “flat-lined” — the floodwaters overtopped the gauge’s measurable range. Moreover, the National Weather Service (NWS) lacked a database of measurable ranges for flood gauges. Quoting the NWS report: “A lesson learned here was that maximum recordable river levels should be known by NWS staff, not only so staff aren’t surprised when this type of issue arises, but also to notify USGS personnel so that they can install a temporary gage and remove or elevate threatened equipment.” Technical debt, if ever I’ve seen it.For systems consisting solely of software, separation of responsibility for system maintenance and system development or acquisition enables the acquiring organization to act with little regard for the consequences of its decisions vis-à-vis maintenance matters [Boehm 2016]. This is an unfortunate state of affairs that increases the rate of accumulation of new technical debt, and increases the lifetime of legacy technical debt, because the MICs associated with the technical debt aren’t borne by the acquiring organization.
For example, a focus on performance of the organization that’s responsible for acquisition biases them in favor of attending to the direct and immediate costs of the acquisition, with little or no regard for ongoing maintenance issues. The maintenance organization is then left to deal with whatever the acquired system contains (or lacks).
An analogous mechanism operates for organizations that develop, market, and maintain products or services with software elements in their respective infrastructures. In that case, separation of the development function from the maintenance function enables the development function to act independently of the consequences of its decisions for maintenance matters.
But the separation-of-responsibilities mechanism that leads to uncontrolled technical debt isn’t restricted to software. Any technological asset that has ongoing maintenance needs (and most of them do) can potentially present this problem.
For example, in the United States, and many other countries, two streams of resources support publicly-owned infrastructure [Blair 2017]. The funding stream covers construction, operations and maintenance, and repairs. Its usual sources are taxes, tolls, licenses, other user fees, sale of ad space, and so on. The financing stream covers up-front construction costs, to bridge the period from conception through construction, until the funding stream begins delivering resources. The financing stream usually comes from bond sales.
Although both streams are controlled by legislatures, or by agencies they establish, the effects of the two streams differ fundamentally. The financing stream is dominant in the early stages of the asset’s lifecycle — during construction. The funding stream is dominant after that — when maintenance and operations are most important. Legislators and agencies are generally reluctant to supply funding because of the impact on taxpayers and users. Legislators and agencies find financing much more palatable. For this reason, among others, U.S. infrastructure maintenance is generally under-resourced, and technical debt gradually accumulates.
So it is with technological assets in organizations. For accounting purposes, capital expenses are treated differently from operational expenses, and the result is that operational expenses can have a more significant impact on current financial results than capital expenses do. This leads organizations to underfund operations and maintenance, which contributes to the accumulation of technical debt.
Control of new technical debt accumulation and enhancement of technical debt retirement rates is possible only if the acquisition or development organizations can somehow be held accountable for the MICs that result from their actions. Securitization of the debt incurred, as I’ll address in a forthcoming post, is one possible means of imposing this accountability. But reserves are also required, because some of the debt incurred might not be known at the time the asset is acquired or created.
Separation of responsibility for system maintenance and system acquisition or system development is actually a form of stovepiping. See “Stovepiping can lead to technical debt” for more on stovepiping.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Many an enterprise culture includes, perhaps tacitly, an unrealistic definition of done. When an enterprise culture assumes a definition of done for projects that excludes — or fails to adequately acknowledge — attributes related to sustainability of deliverables, technical debt expands inexorably. In most organizations, the definition of done for projects includes meeting the attributes that most internal customers understand and care about. These attributes might not include sustainability [Guo 2011]. Indeed, even among technologists, the definition of done might not enjoy precise consensus [Wake 2002].
The 2009 Ford Focus SES coupe (North America) engine bay. Gone are the days when typical owners could learn how to maintain their own vehicles. Engines have become so complex that even experienced mechanics must be trained to maintain engines with which they’re unfamiliar. Since these vehicles are being offered for sale to consumers, clearly their manufacturers regard their designs as “done.” But is technical debt a factor in the growing complexity of modern motor vehicle engines? It’s probably present in their software, and it would be most surprising if we found no technical debt in the mechanical design. Photo (cc) Porsche997SBS courtesy Wikimedia.
Because attributes related to sustainability of deliverables are less well understood by internal customers — indeed, by nearly everyone — it is perhaps unsurprising that sustainability might not receive the attention it needs. Applying scarce resources to enhance attributes the customer doesn’t understand, and cares about less, will always be difficult.
To gain control of technical debt, we must redefine done to include addressing sustainability of deliverables. Although there may be many ways to accomplish this, none will be easy. Resolution will involve, inevitably, educating internal customers so that they understand enough about sustainability to enable them to justify paying for it.
The typical definition of done for most projects ensures only that the deliverables meet the requirements. Because requirements usually omit reference to retiring newly incurred non-strategic technical debt, projects are often declared complete with incremental technical debt still in place. A similar problem prevails with respect to legacy technical debt.
A more insidious form of this problem is intentional shifting of the definition of done. This happens when the organization has adopted a reasonable definition of done that allows for addressing sustainability, but under severe time pressure, the definition is “temporarily” amended to allow the team to declare the effort complete, even though sustainability issues remain unaddressed.
For most projects, three conditions conspire to create steadily increasing levels of non-strategic technical debt. First, for most tasks, the definition of done is that the deliverables meet the project objectives, or at least, they meet them well enough. Second, typical project objectives don’t restrict levels of newly incurred non-strategic technical debt, nor do they demand retirement of incidentally discovered legacy technical debt. Third, budget authority usually terminates upon acceptance of delivery. These three conditions, taken together, restrain engineering teams from immediately retiring any debt they incur and from retiring — or documenting or reporting — any legacy technical debt they encounter while fulfilling other requirements.
For example, for one kind of incremental technical debt — what Fowler calls [Fowler 2009] Inadvertent/Prudent (“Now we know how we should have done it”) — the realization that debt has been incurred often occurs after the task is “done.” If budget authority has terminated, there are no resources available — financial or human — to retire that form of technical debt.
Unless team members document the technical debt they create or encounter, after they move on to their next assignments, the enterprise is likely to lose track of the location and nature of that debt. A more realistic definition of done would enable the team to continue working post-delivery to retire, or at least document, any newly incurred non-strategic technical debt or incidentally encountered legacy technical debt. Moreover, strategic technical debt — technical debt incurred intentionally for strategic reasons — is also often left in place. Although it, too, must be addressed eventually, the widespread definition of done doesn’t address it.
Policymakers are well positioned to advocate for the culture transformation needed to redefine done.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Stovepiping can lead to technical debt. Actual stovepipes — the tubes that vent exhaust from stoves — serve as a metaphor for the flow of information in “stovepipe” organizations. In “stovepipe” organizations, information flows predominantly (or only) up or down along the parallel chains of command, and rarely (or never) from a point in one chain of command across to some other chain of command [Waters 2010]. The stovepipe metaphor is imperfect, in the sense that in actual stovepipes, smoke and fumes rarely flow downwards. By contrast, in organizations, some information does flow down the chains of command. But the metaphor does clarify the problem that whatever the organization learns in one metaphorical stovepipe isn’t easily transferred to other metaphorical stovepipes.
A wood-burning stove in a farm museum in Lower Bavaria (German: Niederbayern), which is one of the seven administrative regions of Bavaria, Germany. The stovepipe, which is the black tube running upwards from the stove, channels smoke and fumes out of the kitchen into the chimney, and then, presumably, out of the farmhouse.
Stovepiping can occur in both organizational structures and in engineered systems. These two forms of stovepiping are intimately related, and both can lead to uncontrolled formation of new technical debt, or increased persistence of existing technical debt.
In organizational structures, stovepiping occurs when similar capabilities are implemented in elements of two or more different organizational units that act relatively independently of each other. An example is the dispersal of some elements of the IT function out into IT’s customers. When independent organizations have similar technical needs, they’re at risk of independently implementing technological capabilities that duplicate each other, at least in some respects.
In engineering, stovepiping occurs, for example, when two or more technological assets are managed and maintained independently [McGovern 2003]. In that situation, distinct engineering efforts working on those assets might happen to solve the same problem, possibly in two different ways, with each party either ignorant, or possibly disparaging, of the other’s efforts.
In whichever way duplication of technological capability comes about, it can lead to increasing levels of technical debt, or to increased persistence of technical debt. This happens because the organization might be required to execute future maintenance or enhancement multiple times — once for each instance of the technical artifact. That exposes the organization to additional cost, additional load on its staff, and additional risk of creating defects and incurring liability, compared to a situation in which technical assets are shared among all who need them.
The problem is actually even more worrisome. First, in the case of a defect found in one version of a technological artifact, it’s possible that the people who are aware of the defect might not realize that another version of the artifact exists. If that other version also has an analogous defect, its defect might go unrecognized for some time, with all the usual attendant negative consequences. Second, in the case of a necessary extension of the artifact’s capabilities, the maintainers of one version might recognize the need for an extension, and implement it. Meanwhile the maintainers of other versions might not recognize the need for extension. They might not take action until something bad happens or a possibly urgent need arises. It’s easy to conjure other unfavorable — and costly — scenarios.
In engineering more generally, stovepiping can occur internally in systems, even though only one business unit is involved, and even though the stovepiped artifacts serve purposes invisible to the world outside the system. This can occur whenever communication is weak between the teams designing or maintaining the portions of the system that host the similar artifacts. For those familiar with the Apollo XIII incident, the incompatibility of the two carbon dioxide scrubbers in the command module and the lunar excursion module serves as an example of the risks of technical stovepiping.
When distinct business units or functions operate their own engineering or IT functions, or when they depend on a shared engineering function but require similar work, there is an elevated probability of duplication of technological assets or capabilities. This happens when the organizational structure, or the timing of the work, encourages separation of the engineering efforts. Engineering or IT functions operated separately under the control of distinct business units or functions can clearly produce duplicated capabilities. However, duplication can also occur when the engineering function is shared across distinct business units or functions, but the actual people and teams performing the work differ for different efforts, and when communication is weak between those teams. This can happen whether or not the efforts are conducted contemporaneously.
Because identifying these forms of technical debt after they’re created is notoriously difficult, preventing their formation is preferable to trying to detect them post-implementation. Prevention is possible if the enterprise establishes mechanisms that facilitate consultation and sharing among elements of different, separately operated technology development or maintenance functions. In other words, the organization must “break” the stovepipes — no mean feat, politically speaking.
Another challenge, of course, is providing resources for such sharing mechanisms, because preventing technical debt is rarely recognized as a value generator. If it were so recognized, the resources would likely appear. Changes in cost accounting might make such recognition more likely.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Defining technical debt at the level of specificity needed for project objectives is difficult. Confronted with this difficulty, some internal customers of technologists adopt a zero-tolerance approach to technical debt, without specifically defining technical debt. Post-delivery — sometimes much, much, post — when technical debt is discovered or recognized, technologists are held responsible, even in cases when no one could have predicted that a specific artifact would eventually come to be regarded as technical debt. This sets up an adversarial dynamic between technologists and their internal customers.
Trouble at the airport. When airline pilots engage in work-to-rule actions, the immediate result can be large numbers of delayed or cancelled flights. The longer-term result might be beneficial to pilots, the airline, and the public, but only if labor peace can be restored, and the damage to the flying public can be overcome. So it is with work-to-rule deliveries as a way of dealing with zero-tolerance technical debt policies. The organization must overcome the adversarial culture that results from indiscriminate attempts to control technical debt. Technologists do gain some measure of protection by working to rule, but the longer-term benefit of the organization’s learning to manage technical debt arrives only if the adversarial culture can be overcome. Image (cc) Hotelstvedi courtesy Wikimedia.
And that’s when the trouble begins.
Within this adversarial dynamic, technologists try to protect themselves against future recriminations by “working to rule.” They perform only work that is specified by the internal customer. If they find something additional that must be done, they perform that work only if they successfully obtain the customer’s approval. Some customers continue to adhere to a zero-tolerance policy with respect to technical debt, but such a non-specific requirement cannot be met. Because technologists are “working to rule,” they use the ambiguity of the zero-tolerance requirement to assert that they performed all work that was sufficiently specified. This level of performance is analogous to the work-to-rule actions of some employees involved in labor disputes with their employers, and who are literally in compliance with the requirements of the employer, but only literally [LIBCom 2006].
Requiring deliverables to be totally free of technical debt contributes to formation of an adversarial culture, wherein the adversaries are the technologists and their internal customers. Shedding that adversarial culture, once it sets in, can be difficult. Compelling employees, vendors, or contractors to deliver work that’s free of all technical debt is therefore unlikely to succeed. Whether work is performed in-house by employees, or is outsourced, or is performed in-house by contractors, deliverables that meet the minimum possible interpretation of the objectives of the effort are almost certainly burdened with unacceptable levels of technical debt. What can we do to prevent this?
To avoid creating an adversarial culture, we can specify in project objectives some kinds of technical debt that must be removed in toto. To ensure steady progress in technical debt retirement, develop a statement of objectives that includes complete retirement of at least one well-defined class of technical debt, emphasizing debt classes that have the highest anticipated MICs in the near term. Other well-defined classes of technical debt can be addressed on a best-effort basis.
We must accept that any other forms of technical debt that remain at the end of a given project, or any constructions that later come to be recognized as technical debt, are just the “cost of doing business.” We’ll get to them, but unfortunately, not this time.
References
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Few performance management systems provide guidance with respect to behaviors relating to technical debt, perhaps because technical debt is not widely understood, or perhaps because technical debt isn’t seen as a concern for the performance of anyone but engineers and their managers. Still, organizations that expect to gain control of technical debt must ensure that performance standards are clear about expectations with respect to behaviors that could affect technical debt. In organizations in which technical debt currently plays a minor role, if any, in the performance management system, policymakers can advocate for effective changes, if they understand what the appropriate role for performance management is in controlling technical debt. This post should be helpful.
A dog receiving a reward. Many performance management systems implement a model that assumes that the correct configuration of incentives and disincentives will produce the desired levels of performance. This theory is questionable.
A fundamental premise of many performance management systems is that incentives can encourage desirable behavior and disincentives can discourage undesirable behavior. Unfortunately, serious questions have arisen about the effectiveness of these behavioral control mechanisms in general [Kohn 1999]. The problem is that employees find ways to harvest incentives without exhibiting the desired behavior. Likewise, they find ways to circumvent disincentives while continuing to exhibit undesired behavior.
Moreover, specifically for technical debt management, behavioral control is especially problematic, because some of the behaviors that must be controlled are inherently immeasurable. For example, the design of an incentive structure to encourage legacy technical debt retirement is debatable, given the technical difficulties involved in even defining legacy technical debt, let alone measuring its size.
Managing performance vis-à-vis technical debt, therefore, presents a problem of the kind Austin calls partially supervised [Austin 1996]. Supervising engineers whose work touches on assets that bear technical debt can only be partial, because measuring technical debt is only partially practical given the state of the art. Austin shows how partial supervision frequently leads to dysfunctional performance management, but the problem is especially vexing for managing technical debt. For example, in some cases, engineers’ work can incur new technical debt that remains unrecognized for months or years after the work is completed. To fully supervise such work would require inventing retroactive incentives and disincentives, which not only do not exist, but which are of questionable legality in most jurisdictions.
Although incentives and disincentives cannot serve to manage performance relative to technical debt, a very effective model is available. Enterprise leaders could communicate their intentions relative to technical debt, and empower the people of the organization to take steps to reduce debt. In the United States military, and others as well, a doctrine that implements this approach is called commander’s intent [Mattis 2008].
Gen. Mattis offers five principles that guide what the military calls “effect-based operations.” For technical debt management, the effect we seek is rational control of the technical debt portfolio. Here are his five principles, transformed to the field of technical debt.
Technology development, maintenance, and cyberdefense in the future will require a balance of conventional and unconventional approaches.
Technology evolves rapidly, and we must be willing to adapt our methods.
Technologies are dynamic with an infinite number of variables; therefore, it is not scientifically possible to accurately predict the level of technical debt that will result from any given effort. To suggest otherwise runs contrary to historical experience and the nature of modern technological assets.
We are in error when we think that what works (or does not work) in efforts involving one technology in one enterprise will be universally applicable to all technologies in all enterprises.
Finally, to paraphrase General Sherman, “Every attempt to make technical debt management easy and safe will result in humiliation and disaster.”
Most organizations rely on supervisors to communicate the analog of commander’s intent to their subordinates. Currently, it’s fair to say that few supervisors outside the technology-oriented elements of the enterprise communicate much about technical debt to their subordinates.
That situation might explain why most performance management systems encourage behaviors that unwittingly expand the body of technical debt, especially for non-technologist performers. There are situations in which the widely applauded actions of the outstanding performer are such as to incur technical debt strategically and responsibly. Technical debt so incurred is what McConnell calls Type II [McConnell 2008] and what Fowler calls Deliberate and Prudent [Fowler 2009]. But most performance management systems, especially for non-technologists, say nothing about technical debt, and thus risk encouraging behaviors that indirectly exacerbate the problems associated with technical debt.
Distinguishing responsible and irresponsible behaviors is possible only if understanding of the nature of technical debt is widespread in the organization, even beyond the technologists. Here’s an example:
It was ambitious, what advocates called a “stretch goal,” but the VP of Marketing approved the plan to get the new app released by the end of the next fiscal quarter. After a month of meetings, and much jawboning, the CTO agreed to find a way to make it happen, despite serious objections from the VP of New Product Development. Engineers and testers were able to meet the date, but they had to incur significant technical debt, and when they asked for resources to retire that debt after the release, the VP of Marketing opposed the request, because she needed additional resources for the promotional campaign due to our late entry into the market.
Stories like this illustrate scenarios in which technical debt considerations are consistently assigned a lower priority than goals related to market timing, market development, and revenue generation. Standards for setting priorities closely parallel the standards defined in the performance management system. Indeed, the goal of performance management should be to support enterprise goals. In the scenario above, the organization might meet the immediate goal of a successful release, but it does so by incurring technical debt, thereby imperiling the next release. In this scenario, it’s evidently necessary to change the performance management system to achieve a better balance between immediate goals and the near-term future goals.
Since anyone in the enterprise can take actions or make decisions that lead to incurring new technical debt, or cause existing technical debt to remain in place, organizations need performance standards that guide employees with respect to technical debt. To provide guidance for distinguishing responsible behavior from irresponsible behavior, performance management systems must acknowledge the potential of any employee to affect technical debt, constructively or otherwise. Performance management systems must be reviewed with respect to alignment with technical debt policy, and adjusted to encompass a mechanism analogous to Mattis’s vision of commander’s intent.
References
[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6
Contains an extensive discussion of the consequences of partial supervision of performance. Since technical debt can only be partially supervised, the concept is relevant to understanding the effects of performance management systems on technical debt. Order from Amazon
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1
[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
Enterprises that grow by acquisition find themselves acquiring the technological assets of the organizations they acquire. And most enterprises acquire technological assets by other means as well. In either case, the contract negotiation teams need technical knowledge to evaluate and project the effects of these acquisitions on total enterprise technical debt. But as total enterprise technical debt grows, the capacity of enterprise technologists to support new contract negotiations declines, which leads to a self-sustaining cycle of technical knowledge deficits. Policymakers and strategic planners are likely the most effective possible advocates for breaking the cycle by hiring more technologists.
Contract negotiations can be complex
Negotiating contracts with vendors that provide outsourcing services or subcontracting work, or with organizations to be acquired, requires a sophisticated appreciation of the technical debt status of the assets acquired or to be acquired. The technical debt in question includes more than just the debt borne by the asset as it stands in its pre-acquisition context. It also includes the debt that the asset will carry after it’s inserted into the asset portfolio of the acquiring enterprise.
These two debts — pre-acquisition and post-acquisition — can differ, because the interfaces, standards, and approaches of the acquiring organization likely differ from those prevailing within the vendor organization or the acquired organization. Knowledge of the interfaces, standards, and approaches of both parties to the transaction is therefore required to make a valid assessment of the total post-acquisition levels of technical debt.
The enterprise negotiation team therefore requires the services of technologists who are familiar with the maintenance, extension, and cybersecurity work that will be performed on the acquired assets post-acquisition. When the technical debt situation in the enterprise reaches a level so serious that it requires the full attention of all available technologists, they cannot be spared for negotiating contracts. If this happens, then contract negotiation teams could experience a deficit of knowledge concerning the consequences of acquiring assets laden with technical debt. That leads to increasing levels of non-strategic technical debt, which then has the potential to exacerbate the technical knowledge deficit for the negotiating teams.
This situation is an example of what’s commonly called a vicious cycle. After technical debt has reached a critical level, there are really only two tactics that can break the cycle — get more engineers, or suspend some work.
References
[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6
Contains an extensive discussion of the consequences of partial supervision of performance. Since technical debt can only be partially supervised, the concept is relevant to understanding the effects of performance management systems on technical debt. Order from Amazon
[Blair 2017] Hunter Blair. “No free bridge: Why public–private partnerships or other ‘innovative’ financing of infrastructure will not save taxpayers money,” Economic Policy Institute blog, March 21, 2017.
[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.
[Guo 2011] Yuepu Guo, Carolyn Seaman, Rebeka Gomes, Antonio Cavalcanti, Graziela Tonin, Fabio Q. B. Da Silva, André L. M. Santos, and Clauirton Siebra. “Tracking Technical Debt: An Exploratory Case Study,” 27th IEEE International Conference on Software Maintenance (ICSM), 2011, 528-531.
[Kahneman 1977] Daniel Kahneman and Amos Tversky. “Intuitive Prediction: Biases and Corrective Procedures,” Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency, June 1977.
[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1
[McGovern 2003] James McGovern, Scott W. Ambler, Michael E. Stevens, James Linn, Vikas Sharan, and Elias K. Jo. A Practical Guide to Enterprise Architecture, Upper Saddle River, New Jersey: Prentice Hall PTR, 2003.
[Pronin 2002] Emily Pronin, Daniel Y. Lin, and Lee Ross. “The bias blind spot: Perceptions of bias in self versus others.” Personality and Social Psychology Bulletin 28:3, 369-381, 2002.
[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.
[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.
EngineBay2.jpg){kind=link}
