Managing technical debt

Last updated on November 24th, 2019 at 02:18 pm

Managing technical debt is something few organizations now do, and fewer do well. Several issues make managing technical debt difficult and they’re discussed elsewhere in this blog. This thread explores tactics for dealing with those issues from a variety of initial conditions. For example, tactics that work well for an organization that already has control of its technical debt, and which wants to keep it under control, might not work at all for an organization that’s just beginning to address a vast portfolio of runaway technical debt. The needs of these two organizations differ. The approaches they must take might then also differ.

A jumble of jigsaw puzzle pieces. Managing technical debt can be like solving a puzzle.
A jumble of jigsaw puzzle pieces. Where do we begin? With these puzzles, we usually begin with two assumptions: (a) we have all the pieces, and (b) they fit together to make coherent whole. These assumptions might not be valid for the puzzle of technical debt in any given organization.

The first three posts in this thread illustrate the differences among organization in different stages of developing technical debt management practices. In “Leverage points for technical debt management,” I begin to address the needs of strategists working in an organization just beginning to manage its technical debt, and asking the question, “Where do we begin?” In “Undercounting nonexistent debt items,” I offer an observation about a risk that accompanies most attempts to assess the volume of outstanding technical debt. Such assessments are frequently undertaken in organizations at early stages of the technical debt management effort. In “Crowdsourcing debt identification,” I discuss a method for maintaining the contents of a database of technical debt items. Data maintenance is something that might be undertaken in the context of a more advance technical debt management program.

Whatever approach is adopted, it must address factors that include technology, business objectives, politics, culture, psychology, and organizational behavior. So what you’ll find in this thread are insights, observations, and recommendations that address one or more of the issues related to these fields. “Demodularization can help control technical debt” considers mostly technical strategies. “Undercounting nonexistent debt items” is an exploration of a psychological phenomenon.  “Leverage points for technical debt management” considers the organization as a system and discusses tactics for altering it. And “Legacy debt incurred intentionally” explores how existing technical debt can grow as long as it remains outstanding.

Accounting issues also play a role. “Metrics for technical debt management: the basics” is a basic discussion of measurement issues. “Accounting for technical debt” looks into the matter of accounting for technical debt financially. And “Three cognitive biases” is a study of how technical debt is affected by the way we think about it.

Posts in this thread:

Tension between policymakers and technologists

Last updated on July 24th, 2018 at 08:17 pm

Tension between policymakers and technologists, which can manifest as misalignment of their respective priorities, is a significant contributor to uncontrolled growth of technical debt. In this thread I explore sources of this tension and introduce concepts that can assist policymakers and technologists in their efforts to control the growth of technical debt.

Effective, sustainable control of technical debt is the objective of technical debt management policy. In an enterprise that has achieved this objective, technical debt serves as a strategic tool that assists in attaining and maintaining market leadership. In such an organization, technical debt does exist, and some legacy technical debt might remain in place, but technical debt growth is managed strategically, if growth occurs at all. Any technical debt that carries significant MICs, and which compromises productivity and enterprise agility, is addressed and retired with appropriate priority. In short, technical debt is addressed not solely as a technological issue, but as a component of business strategy.

Raindrops on a grapevine
Raindrops on a grapevine. We often think of tension as a negative, destructive force. But tension—in the case of raindrops, surface tension—is what holds a raindrop together. Tension gives the raindrop structure and integrity. The tension between technologists and policymakers can also be constructive. It can ensure that the enterprise manages its technical debt in ways that balance the political imperatives of both technology and strategic health.

This stance is at odds with the historical position most enterprises have adopted vis-à-vis technical debt. Historically, technical debt has been seen as a technical problem, if it has been recognized at all. Most enterprises have left the management of technical debt to the technologists. Frequently, then, the policymaker who enters the discussion about technical debt might be seen by technologists as an interloper, arriving late to the discussion, or as a less-than-knowledgeable invader attempting to seize control of a piece of the technologists’ domain. Tensions can arise between policymakers and technologists. Such tensions complicate the problem of managing technical debt.

One possible source of this tension is revealed in a study of the literature of technical debt, which is evolving so rapidly that it has itself become a focus for research. Li et al. [Li 2015] have produced a review of the software engineering technical debt literature, from which we can extract insights useful to policymakers. Although they studied only the literature relating to technical debt in software engineering, their conclusions are, at least in part, applicable to any field in which the components of the finished product are executed within software tools before being committed to operational form. This covers a wide array of knowledge-intensive endeavors, including mechanical system design, electronic design, framing of legislation, process design, architecture, and even book authorship.

In this thread, I explore the sources of the tension between the modern reality of technical debt as an enterprise issue, and the historical situation of technical debt as a technological issue. This can serve as a guide for policymakers in reframing technical debt from a technological issue dependent for resolution on enterprise resources, to an enterprise issue dependent for resolution on technological resources.

References

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

Other posts in this thread

Cultural debt can be the primary driver of technical debt

Last updated on June 2nd, 2018 at 10:35 pm

Cultural debt can be expensive because, like technical debt, it can incur ongoing metaphorical interest charges. Schein defines organizational culture as “…a pattern of shared basic assumptions learned by a group as it solved its problems of external adaptation and internal integration…” [Schein 2016]. Following the concept of technical debt, we can regard as cultural debt the subset of shared basic assumptions comprising enterprise culture that are no longer fitting for enterprise realities. We can also include as cultural debt any assumptions that ought to be shared, but which are missing or are only partially shared. And we can include shared assumptions that conflict with each other and need to be resolved.

An example of cultural debt: the term “IT”

A tape measure calibrated in both feet/inches and meters/centimeters
A tape measure calibrated in both feet/inches and meters/centimeters. The need to possess tools that serve both measurement systems can be viewed as the metaphorical interest charges on a technical debt resulting from the failure to retire the older “English” system. But from another perspective, the debt involved is actually cultural. Retiring the older system would truly involve a cultural shift.

For most modern enterprises, one element of cultural debt is the very term IT itself — information technology. Coined in 1958 by Leavitt and Whisler [Leavitt 1958], the term was apt up to as recently as 20 years ago, when the role of IT was primarily management, storage, retrieval, manipulation, and presentation of data — information — by technological means. Although those functions remain relevant, the responsibilities of IT have expanded dramatically since 1958. In many organizations, IT is now responsible for designing, implementing, and maintaining the communication infrastructure, including Internet access, personal computers, networking, Web presence, telephones, video conferencing equipment, and television.

In modern organizations in which communication plays a critical and strategic role, an essential element for success is a clear understanding of what IT does and what it contributes. To regard IT as the “information technology” function of the enterprise is, therefore, to risk overlooking and undervaluing these more recently acquired responsibilities. And since the IT function is no longer solely responsible for enterprise information, using the name “IT” or the term information technology risks overvaluing the role of the IT organization relative to information management, while undervaluing its role relative to communications.

In Schein’s culture framework, the term IT reflects a shared assumption about the focus and span of the IT function. That assumption is that IT is responsible for information—an assumption that is no longer well aligned to the reality of the role of IT. We can regard this misalignment as a cultural debt.

The consequences of this particular kind of cultural debt can be severe. For instance, IT is typically responsible for selecting and configuring software for personal computers (PCs) — both desktop and laptop. This responsibility can arise as a consequence of two shared assumptions. First, that computers process information, and second, that IT is responsible for technology-based information processing. The result is that decisions about what many regard as a “personal” computer are not in the control of the person who uses the computer. This conflict in shared assumptions can lead to conflict between PC users and IT, when the IT decision is at variance with their personal preferences.

Worse, a centralized decision process for determining PC configurations is likely to produce outcomes less suitable than would a process more focused at the individual level, which only adds to the frustrations of PC users, and exacerbates the conflict between them and IT. To mitigate the risk that some PC users might try to circumvent IT policy, IT must deploy technology to ensure adherence to their policies. We can regard all of that activity, on the part of both IT and the PC users, as metaphorical interest charges on cultural debt.

An example of retiring cultural debt

In 1987, Edward Yourdon founded a magazine then known as American Programmer. In 1990, Cutter Information Corporation purchased the rights to American Programmer and created Cutter IT Journal, which name includes the term IT. At the time IT was more suitable than the term programmer. As noted above, the term IT, while once useful and apt, is now outmoded at best and often misleading. Just as the functional name IT in organizations constitutes cultural debt, so it does in the name of a journal.

So in the autumn of 2016, Cutter IT Journal retired the cultural debt in its name, and became Cutter Business Technology Journal. Journals rarely change their names. When they do, the impact of the journal is temporarily depressed because of the split of citations between the former title and the new title for two years or so [Tempest 2005]. But as research fields change, their journals must keep pace. Evidently Cutter felt a significant need to retire its cultural debt — significant enough to justify a temporary effect on impact.

What about cultural debt retirement in companies?

Difficulties associated with retiring cultural debt in companies depend strongly on both the nature of the culture and the nature of the debt. To provide insight into the issues that can arise, let’s continue with our exploration of the term IT and its cultural implications.

In many organizations, IT reports to an enterprise Chief Information Officer (CIO). Associated with this official’s title are some of the same cultural debts we find associated with the name of the IT organization. First, within their organizations, CIOs aren’t the only officers with information management responsibility. Second, many CIOs have responsibilities that extend beyond information management, to include, for example, the communication infrastructure. And unlike other peer titles such as CEO, CFO, CMO, and COO, the CIO title evokes separation from business-oriented decisions. That separation contributes to a cultural wall between “IT” and “the business.”

When cultures view IT as an information-centric service organization, a remnant perhaps of the middle or late 20th century, they tend to regard IT as a source of expense to be minimized, rather than as a strategic partner [Ross 2000]. Trends toward strategic acceptance of IT are nevertheless favorable, with room for improvement, according to recent surveys of CIOs [CIO 2018], probably because of reality.

The reality is that business technology must contribute to formulation and implementation of enterprise strategy. To the extent that CIOs and their organizations are viewed as separate from “the business,” their ability to help shape enterprise strategy is limited. This situation subjects CIOs to cultural assumptions about their responsibilities that in some instances conflict with each other, or with enterprise reality. That’s a significant source of the metaphorical interest charges on the cultural debt.

One possible way to retire this debt might entail retitling Chief Information Officer to Chief Business Technology Officer (CBTO). That’s precisely what happened at Forrester Research in 2011 [Plant 2014].

Unfortunately, the name CBTO conflicts with the three-word pattern of enterprise officer titles (C*O), which might create an urge to name the office Chief Technology Officer (CTO). But that role usually has responsibility for the functions that create technological products or services. Thus, for many organizations, to create a CBTO where there is already a CTO might create further sources of conflict. Using the CTO designation for the CBTO is probably impractical.

But we must find some way to retire this particular cultural debt, because it is such an effective generator of technical debt. CBTO seems to be the best available path.

References

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread

Contract restrictions can lead to technical debt

Last updated on March 22nd, 2018 at 11:18 pm

When the owner of an asset, especially a software asset, contracts to provide a capability to a customer incorporating a use of that asset, the work involved might require modification or enhancement of that asset. When the contract permits such work, without transferring ownership of the asset itself, performing it is relatively straightforward, provided the work can be done in a manner compatible with any pre-existing or anticipated future other uses of the asset. But some contract restrictions can cause the owner of the asset to incur technical debt.

A power adaptor/converter for international travelers with U.S. standard equipment.
A power adaptor/converter for international travelers with U.S. standard equipment. This device provides conversion for both hardware connection and voltage supplied.
The wide variation in electric power standards worldwide can be viewed as a technical debt. Someday, in the probably distant future, a world standard will emerge and that debt will be retired. Until then, adaptors like these are travel necessities.

But some contracts restrict such work. For example, for a government customer, ownership of the work performed might be required to transfer to the government customer. Potentially, all of the work might be classified as a national secret. In either of these cases, to retain control of the asset, the owner/contractor arranges to perform all of the work outside the periphery of the asset. To accomplish this, the owner/contractor might interface to the asset through an adaptor that can be transferred to the government customer, or which can be classified as secret as the case may be, thereby insulating the original asset from these ownership restrictions.

The result is tolerable after one such contract is completed. But over time, as the number of adaptors increases, they become a form of technical debt. Each must be maintained against any changes in the original asset. Moreover, making changes to the original asset can become a project of such scale that the temptation to create a static “clone” of the asset for each customer is irresistible. When that happens, any technical debt already present in the asset is also cloned. And any work performed to correct defects in the asset must be performed on each affected clone.

The problem is more general than suggested above. It also appears in the case of software offered for multiple platforms, or multiple versions of a single platform.

But it gets worse. Suppose the maintainers decide to update the asset to make it more extensible, or to make it more maintainable. That update, including all testing and documentation, must then be performed on each clone. If the asset owner elects not to update all clones, then the clones will begin to diverge from each other. Engineers performing tasks on one of the clones must then have knowledge of how it differs from other clones. If a new defect is discovered, it might or might not be present in every clone. Implementing a new extension or other modification might not be possible in all clones, or it might have to be implemented differently in some clones. Life can get very complicated.

Organizations entering into contracts of this kind would be wise either to include language limiting their obligations to maintain the original asset against any changes, internal or external, in its ability to perform its functions. Or they might include an explicit statement of the parties’ intentions relative to financial support for any continuing obligations to maintain that asset.

Organizations offering products for multiple platforms would be wise to consider as strategic the management of technical debt that arises from platform multiplicity. Sound management of this form of technical debt can extend their ability to support multiple platforms, which can dramatically increase returns on investment in the core asset.

References

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread

How outsourcing leads to increasing technical debt

Last updated on January 10th, 2020 at 07:36 pm

Most of the nontechnical precursors of technical debt that afflict in-house work also afflict outsourced work. For example, the planning fallacy affects internal planners, but it also afflicts the bidders for contracts offered by the enterprise in the context of outsourcing. As described in “Unrealistic optimism: the planning fallacy and the n-person prisoner’s dilemma,” Boehm, et al., [Boehm 2016] call this the “Conspiracy of Optimism.” But outsourcing engineering work can introduce pathways for incurring technical debt that are specific to outsourcing.

Green fields
Green fields. Greenfield outsourcing, also known as startup outsourcing, is the outsourcing of activity that has never been performed within the enterprise. It’s especially tricky with respect to technical debt formation, because much of the expertise necessary to perform the work being outsourced is probably not resident within the enterprise. That void in enterprise expertise makes for difficulties in managing technical debt in the outsourced artifacts.

The risks of incurring technical debt associated with outsourcing are inherently elevated, even setting aside those factors that also afflict in-house activity. When most enterprises contract for development of systems or software, the criteria for acceptance rarely include specifications for maintainability or extensibility. This happens, in part, because such qualitative specifications are so difficult to define quantitatively. That’s why the condition of deliverables relative to maintainability and extensibility is so variable. Outsourced deliverables can best be described as bearing an unknown level of technical debt.

The root cause of the problem is likely a lack of a universally accepted metrics for quantifying technical debt [Li 2015]. That’s why it’s difficult to specify in the vendor contract an acceptability threshold for technical debt. And since the consequences of the presence of technical debt in deliverables potentially do not become clear during the lifetime of the contract under which the debt was incurred, years may pass before the issue becomes evident, which complicates the task of understanding the root cause of the problem.

In what follows, I use the term vendor to denote the organization to which work has been outsourced, and the term enterprise to denote the organization that has outsourced a portion of its engineering work. The retained organization is the portion of the enterprise directly relevant to the outsourced work, but which was not itself outsourced. Among the mechanisms that lead to incurring technical debt in the outsourcing context are the six mechanisms sketched below. Gauging the size of these effects by auditing deliverables or by auditing the internal processes of the vendor could be helpful in managing levels of technical debt arising from outsourcing.

This list isn’t intended to be exhaustive. Quite possibly other phenomena also contribute to technical debt formation in the context of outsourcing.

Progressive erosion of retained organization capabilities

Over time, the enterprise can expect erosion of the engineering expertise needed to manage, evaluate, understand, or, if need be, to re-insource (or backsource) the work that has been outsourced [Willcocks 2004][Beardsell 2010]. Typically, staff who formerly performed the outsourced work do move on to other work, voluntarily or not, either within the enterprise or elsewhere. Indeed, cost savings from terminations and employee buyouts often accompany — and economically justify — outsourcing decisions. But even if the enterprise continues to employ the people who formerly performed the work that is now outsourced, those employees, filling new roles, can become less familiar with the current work and therefore less able to perform it. And since they are now engaged in other assignments, their availability is limited. In the public sector, the organizations that perform the outsourced work exacerbate this phenomenon by recruiting from the agencies they serve [Kusnet 2007]. In manufacturing, Kinkel et al. suggest that, paraphrasing, outsourcing disturbs the formation of internal competence [Kinkel 2016].

In short, outsourcing engineering efforts can erode the ability of the enterprise to perform internally the work that is outsourced, or to monitor or evaluate it when performed by the vendor. Consequently, the enterprise is less able to monitor, evaluate, or retire any technical debt that accumulates in the outsourced work products. A policy that would address this risk is one that would (a) require retained organizational capability sufficient to assess the effect on technical debt of any outsourced engineering work; (b) require attention to technical debt issues in any financial models used in making the initial outsourcing decision; (c) require financial models to include the effects of technical debt and controlling technical debt when deciding whether to extend outsourcing contracts with vendors.

Stovepiping among vendors

Most multi-vendor efforts use a separation-of-concerns approach [Laplante 2007] to dividing the work. That is, each vendor is empowered to use any approach it can, consistent with its own contract and statement of work. In some cases, two or more vendors might have overlapping needs that cause them each to produce similar capabilities as elements of their respective deliverables. Sharing of their results is of course possible, but unless both of their contracts anticipate the need for sharing, sharing is unlikely. Failure to share those results that could have been shared can lead to incurring unrecognized technical debt.

Stovepiping within vendors

With regard to the efforts of a single outsource vendor, it’s possible that different teams or individuals working for that vendor might unwittingly create elements with overlapping capabilities. That duplication could lead to technical debt, or it could constitute technical debt in itself. For example, two teams working for the same vendor might have similar testing needs, and might develop testing tools independently. As a second example, in a version of stovepiping combined with temporal displacement, suppose that one team is unaware that a previous effort for the same customer had already developed a capability that it now needs. That team then might re-create that already-existing capability.

Stovepiping within vendors is less likely when the vendor operates under a single vendor technical lead, and the enterprise interacts with that single lead through a single in-house technical lead. When either side of the relationship is managed through multiple contacts, stovepiping is more likely, and new technical debt is more likely to form.

Loss of continuity in the outsourced engineering staff

Unless specified in the agreement between the vendor and the enterprise, staff turnover or reassignment within the vendor organization, between one version of the product or service and the next, can lead to technical debt in just the same way that turnover in-house can lead to technical debt. With outsourcing, however, the vendor has little internalized motivation to control this phenomenon, and the enterprise likely has less control—and less awareness—of staff assignments within the vendor organization than it does within the enterprise. Thus, loss of continuity in the outsourced engineering staff is both more likely and more likely to lead to technical debt.

Reduced coordination of engineering approaches and business objectives

Lack of coordination between engineering approaches and business planning can cause engineers to create and deploy artifacts that must be revisited later, when they learn of business plans that were unknown to the engineers at the time they produced those artifacts. See “Failure to communicate long-term business strategy.” This scenario is more likely, and possibly irresolvable, when the enterprise withholds its long-term plans from the outsourcing vendor to protect its strategy.

Hiring and retention problems

In some instances, commonly called startup outsourcing or greenfield outsourcing, the work being outsourced has never been performed within the enterprise [Delen 2007]. In these cases, typically, the enterprise must then reassign existing employees or hire new employees to interface with the outsource vendor. These roles are analogous to remote supervisors, except that the teams they supervise are not employees of the enterprise. Hiring and retaining people for these roles can be difficult, because of startup challenges both within the enterprise and within the vendor organization. Recruitment and especially retention problems in these roles can decrease the likelihood of controlling technical debt, and increase the likelihood of incurring technical debt.

References

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

How performance management systems can contribute to technical debt

Last updated on December 26th, 2019 at 10:45 am

Few performance management systems provide guidance with respect to behaviors relating to technical debt, perhaps because technical debt is not widely understood, or perhaps because technical debt isn’t seen as a concern for the performance of anyone but engineers and their managers. Still, organizations that expect to gain control of technical debt must ensure that performance standards are clear about expectations with respect to behaviors that could affect technical debt. In organizations in which technical debt currently plays a minor role, if any, in the performance management system, policymakers can advocate for effective changes, if they understand what the appropriate role for performance management is in controlling technical debt. This post should be helpful.

A dog receiving a reward
A dog receiving a reward. Many performance management systems implement a model that assumes that the correct configuration of incentives and disincentives will produce the desired levels of performance. This theory is questionable.

A fundamental premise of many performance management systems is that incentives can encourage desirable behavior and disincentives can discourage undesirable behavior. Unfortunately, serious questions have arisen about the effectiveness of these behavioral control mechanisms in general [Kohn 1999]. The problem is that employees find ways to harvest incentives without exhibiting the desired behavior. Likewise, they find ways to circumvent disincentives while continuing to exhibit undesired behavior.

Moreover, specifically for technical debt management, behavioral control is especially problematic, because some of the behaviors that must be controlled are inherently immeasurable. For example, the design of an incentive structure to encourage legacy technical debt retirement is debatable, given the technical difficulties involved in even defining legacy technical debt, let alone measuring its size.

Managing performance vis-à-vis technical debt, therefore, presents a problem of the kind Austin calls partially supervised [Austin 1996]. Supervising engineers whose work touches on assets that bear technical debt can only be partial, because measuring technical debt is only partially practical given the state of the art. Austin shows how partial supervision frequently leads to dysfunctional performance management, but the problem is especially vexing for managing technical debt. For example, in some cases, engineers’ work can incur new technical debt that remains unrecognized for months or years after the work is completed. To fully supervise such work would require inventing retroactive incentives and disincentives, which not only do not exist, but which are of questionable legality in most jurisdictions.

Although incentives and disincentives cannot serve to manage performance relative to technical debt, a very effective model is available. Enterprise leaders could communicate their intentions relative to technical debt, and empower the people of the organization to take steps to reduce debt. In the United States military, and others as well, a doctrine that implements this approach is called commander’s intent [Mattis 2008][US Army 2010].

Gen. Mattis offers five principles that guide what the military calls “effect-based operations.” For technical debt management, the effect we seek is rational control of the technical debt portfolio. Here are his five principles, transformed to the field of technical debt.

  1. Technology development, maintenance, and cyberdefense in the future will require a balance of conventional and unconventional approaches.
  2. Technology evolves rapidly, and we must be willing to adapt our methods.
  3. Technologies are dynamic with an infinite number of variables; therefore, it is not scientifically possible to accurately predict the level of technical debt that will result from any given effort. To suggest otherwise runs contrary to historical experience and the nature of modern technological assets.
  4. We are in error when we think that what works (or does not work) in efforts involving one technology in one enterprise will be universally applicable to all technologies in all enterprises.
  5. Finally, to paraphrase General Sherman, “Every attempt to make technical debt management easy and safe will result in humiliation and disaster.”

Most organizations rely on supervisors to communicate the analog of commander’s intent to their subordinates. Currently, it’s fair to say that few supervisors outside the technology-oriented elements of the enterprise communicate much about technical debt to their subordinates.

That situation might explain why most performance management systems encourage behaviors that unwittingly expand the body of technical debt, especially for non-technologist performers. There are situations in which the widely applauded actions of the outstanding performer are such as to incur technical debt strategically and responsibly. Technical debt so incurred is what McConnell calls Type II [McConnell 2008] and what Fowler calls Deliberate and Prudent [Fowler 2009]. But most performance management systems, especially for non-technologists, say nothing about technical debt, and thus risk encouraging behaviors that indirectly exacerbate the problems associated with technical debt.

Distinguishing responsible and irresponsible behaviors is possible only if understanding of the nature of technical debt is widespread in the organization, even beyond the technologists. Here’s an example:

It was ambitious, what advocates called a “stretch goal,” but the VP of Marketing approved the plan to get the new app released by the end of the next fiscal quarter. After a month of meetings, and much jawboning, the CTO agreed to find a way to make it happen, despite serious objections from the VP of New Product Development. Engineers and testers were able to meet the date, but they had to incur significant technical debt, and when they asked for resources to retire that debt after the release, the VP of Marketing opposed the request, because she needed additional resources for the promotional campaign due to our late entry into the market.

Stories like this illustrate scenarios in which technical debt considerations are consistently assigned a lower priority than goals related to market timing, market development, and revenue generation. Standards for setting priorities closely parallel the standards defined in the performance management system. Indeed, the goal of performance management should be to support enterprise goals. In the scenario above, the organization might meet the immediate goal of a successful release, but it does so by incurring technical debt, thereby imperiling the next release. In this scenario, it’s evidently necessary to change the performance management system to achieve a better balance between immediate goals and the near-term future goals.

Since anyone in the enterprise can take actions or make decisions that lead to incurring new technical debt, or cause existing technical debt to remain in place, organizations need performance standards that guide employees with respect to technical debt. To provide guidance for distinguishing responsible behavior from irresponsible behavior, performance management systems must acknowledge the potential of any employee to affect technical debt, constructively or otherwise. Performance management systems must be reviewed with respect to alignment with technical debt policy, and adjusted to encompass a mechanism analogous to Mattis’s vision of commander’s intent.

References

[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6

Order from Amazon

Cited in:

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1

Order from Amazon

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here; Retrieved November 9, 2017.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

[US Army 2010] U.S. Army (2010) Field Manual 5.0 – The Operations Process U.S. Department of the Army.

Describes the concept, value, and importance of the doctrine of commander’s intent. See the index for “commander’s intent,” and especially paragraphs 2-90 and 2-91. Available: here; Retrieved: Dec. 22, 2019.

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

Other posts in this thread

Failure to communicate long-term business strategy

Last updated on August 25th, 2018 at 09:47 am

Failure to communicate long-term business strategy can lead to increased technical debt, because engineering decisions that aren’t aligned with business strategy can result in what later becomes technical debt. As business strategy veers away from the assumptions underlying those misaligned engineering decisions, engineers must alter implementations to track the strategy. Technical debt can form during those alteration efforts. Moreover, the expenditure of resources to support those alteration efforts might have been unnecessary if engineers had been better informed about long-term business strategy. In some cases, those resources could have been allocated to other pursuits, including technical debt retirement. To ensure alignment of engineering decisions with long-term business strategy, engineering decision-makers must be aware of long-term and intermediate-term enterprise strategy. When they’re well informed, they can anticipate the engineering needs of the enterprise. And they’re more likely to make decisions that are compatible with strategy.

A plug-in electric vehicle being recharged
Recharging a  plug-in electric vehicle. The dominance of petroleum-powered vehicles is nearing its end. Further investment in the petroleum-based fuels infrastructure is now  inconsistent with what the global economy has chosen as its strategic intent. Electric vehicles are still out of the reach of most consumers, but they would be wise to favor long range vehicles. As electric vehicles become ascendant, petroleum filling stations will become more widely spaced, because they add to the metaphorical interest charges on the technical debt of yet-to-be-retired petroleum powered vehicles. During the transition to electric power dominance, a long-range petroleum-powered vehicle offers clear advantages over its shorter-range cousins.

Moreover, the effect is bi-directional. Strategists can benefit from understanding the effect their strategies have on technological activity. For example, consider the process of choosing among strategic options. A favorable outcome is more likely if strategists know the effects of each option on the technical debt portfolio.

To gain effective control of technical debt, senior management must regard the technical elements of the enterprise as strategic partners [Woodard 2013] [Ross 2000] [Brenner 2016a]. Policymakers can make important contributions to enhancing communication between strategists and technologists.

For example, when engineers know the general direction of the enterprise, they can focus efforts on assets that are compatible with future needs. Inversely, when they’re unaware of what the business strategy might soon require, they’re more likely to make decisions that they must later rescind.

What about legacy technical debt retirement?

Analogous considerations apply to legacy technical debt  retirement efforts. Major technical debt retirement efforts are often subject to review for alignment with enterprise strategy. But we tend not to review incidental retirement efforts that occur in the context of routine maintenance or development. Consequently, engineers might allocate effort to incidental  debt retirement unnecessarily if the asset is due for overhaul or replacement. Communicating long-term strategy effectively is likely the most reliable way to prevent such misspent effort.

Some managers elect to communicate business strategy to technologists only when they “need to know.” Often, technologists needed to know long before that.

References

[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6

Order from Amazon

Cited in:

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1

Order from Amazon

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here; Retrieved November 9, 2017.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

[US Army 2010] U.S. Army (2010) Field Manual 5.0 – The Operations Process U.S. Department of the Army.

Describes the concept, value, and importance of the doctrine of commander’s intent. See the index for “commander’s intent,” and especially paragraphs 2-90 and 2-91. Available: here; Retrieved: Dec. 22, 2019.

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Other posts in this thread

How MPrin can change spontaneously

Last updated on February 18th, 2020 at 02:32 pm

Recall that our definition of the metaphorical principal of a technical debt (MPrin) at a given time is the cost of retiring the debt at that time. This cost can change with time. For example, due to ongoing maintenance and enhancements, the part of the asset in which the debt is embedded might become rather more complicated or constrained than it was earlier. That increased complexity can increase MPrin. This can happen even if the elements that comprise the debt itself have remained unchanged.

Moreover, the MPrin of any extensions to the asset in question, or to other assets, might also change. Those assets that use services provided by a subsystem manifesting the debt might also require alteration at debt retirement time. Untangling the debt from its surroundings, making necessary modifications, and testing the result, can be a delicate and complex process. The process might actually cost more at debt retirement time than whatever was saved when the debt was incurred.
On the other hand, in some circumstances, the cost of retiring the debt can decrease over time. Consider the following fictitious example.

A high pressure sodium streetlight at dusk
A high pressure sodium streetlight at dusk. Photo (cc) Famartin courtesy Wikimedia Commons.

Zion is a small city of 110,000 that’s struggling with two problems related to street lighting. Its current streetlights use High-Pressure Sodium (HPS) lights. HPS lights use almost twice as much energy as do the newer LED streetlights for the same level of illumination. Zion’s second problem is that the existing streetlights provide only one level of illumination throughout the city. This is causing a stream of complaints from many residents who have concerns about street lighting spilling onto their property at night. The bright light interferes with the sleep patterns of people and their pets.

Both of these problems have technical solutions that became available after the current HPS lights were installed. That’s why we can view them as arising from technical debt. Zion had investigated resolving the light pollution problem, but couldn’t find an affordable solution. Time passed. When LED street lights became widely available, Zion investigated retiring its HPS lights. They found an LED lighting system that was dimmable on a block-by-block basis using a wireless control system. By retiring the technical debt associated with the HPS lights, Zion was able to afford retiring the technical debt associated with its un-dimmable lighting system.

Zion was able to afford to retire both forms of technical debt at once because of the way they interacted, even though retiring them one at a time would have been too expensive.

This example shows that the MPrin of a technical debt can vary widely, depending on the assets involved, and on what other debts they carry. Such variation is far more common in the realm of technical debt than it is in the world of financial debt.

References

[Austin 1996] Robert D. Austin. Measuring and Managing Performance in Organizations. New York: Dorset House, 1996. ISBN:0-932633-36-6

Order from Amazon

Cited in:

[Beardsell 2010] Julie Beardsell. “IT Backsourcing: is it the solution to innovation?”, SMC Working Paper Series, Issue: 02/2010, Swiss Management Center University, 2010.

Available: here; Retrieved: February 15, 2018

Cited in:

[Boehm 2016] Barry Boehm, Celia Chen, Kamonphop Srisopha, Reem Alfayez, and Lin Shiy. “Avoiding Non-Technical Sources of Software Maintenance Technical Debt,” USC Course notes, Fall 2016.

Available: here; Retrieved: July 25, 2017

Cited in:

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Delen 2007] Guus Delen. “Decision and Control Factors for IT-sourcing,” in Handbook of Network and System Administration, Jan Bergstra and Mark Burgess, eds., Boston: Elsevier, 929-946, 2007.

Order from Amazon

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Kinkel 2016] Steffen Kinkel, Angela Jäger, Djerdj Horvath, and Bernhard Rieder. “The effects of in-house manufacturing and outsourcing on companies’ profits and productivity,” 23rd International Annual EurOMA Conference, At Trondheim, Volume: 23, June 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999. ISBN:0-395-71090-1

Order from Amazon

Cited in:

[Kusnet 2007] David Kusnet. “Highway Robbery II,” report of the National Association of State Highway and Transportation Unions (NASHTU).

Cited in:

[Laplante 2007] Phillip A. Laplante. What Every Engineer Should Know About Software Engineering. New York: CRC Press, 2007.

Order from Amazon

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here; Retrieved November 9, 2017.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

[US Army 2010] U.S. Army (2010) Field Manual 5.0 – The Operations Process U.S. Department of the Army.

Describes the concept, value, and importance of the doctrine of commander’s intent. See the index for “commander’s intent,” and especially paragraphs 2-90 and 2-91. Available: here; Retrieved: Dec. 22, 2019.

Cited in:

[Willcocks 2004] L. Willcocks, J. Hindle, D. Feeny, and M. Lacity. “IT and Business Process Outsourcing: The Knowledge Potential,” Information Systems Management 21:3, 7-15, 2004.

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Related posts