Blog

Posted on

The Dunning-Kruger effect can lead to technical debt

Last updated on May 31st, 2018 at 07:43 am

The Dunning-Kruger effect [Kruger 1999] can lead to formation or persistence of technical debt in two ways. First, it can cause technologists or their managers to overestimate their ability to maintain the resource focus needed for retiring technical debt in a timely fashion. Second, it can cause senior managers to be reluctant to accede to resource requests of technologists and their managers in support of technical debt management programs.

Cropped detail from Charles Robert Darwin, a painting by John Collier
Cropped detail from Charles Robert Darwin, a painting by John Collier (1850-1934), given to the National Portrait Gallery, London, in 1896. Darwin writes, in The Descent of Man (1871): “… ignorance more frequently begets confidence than does knowledge …” which is the essence of the Dunning-Kruger effect. Image courtesy WikiQuote.

Kruger and Dunning conducted experiments that yielded results consistent with the following four principles (paraphrasing):

  1. Incompetent individuals, compared to their more competent peers, tend to dramatically overestimate their own ability and performance
  2. Incompetent individuals, compared to their more competent peers, tend to be less able to gain insight into their own true levels of performance
  3. Incompetent individuals can gain insight about their shortcomings, but, paradoxically, this comes about by gaining competence
  4. Incompetent individuals, compared to their more competent peers, are less able to recognize competence when they see it

The first three principles lead to distorted assessments of one’s own capabilities. The fourth principle leads to distorted assessments of the capabilities of others.

As an example of distorted self-assessment, consider a team or its managers who must undertake retirement of some types of technical debt in the course of enhancing or repairing an asset. Such a task plan seems at first to offer efficiencies, because the engineers can readily make both kinds of changes at one go. Metaphorically, if we must go to the store for milk, we can also pick up bread while we are there, rather than making two trips.

However, modifying an existing complex technological asset is unlike shopping for bread and milk. The two kinds of modifications — debt retirement and asset enhancement or repair — might seem at first to be separable, and often they are. But if they are not separable, and the two tasks are undertaken together, testing and debugging can become extremely complicated, because of interactions between defects in the two kinds of modifications. Under some circumstances, an experienced team and its managers might be more likely to anticipate these difficulties. An inexperienced team and its managers might be more likely to underestimate the difficulties, as a consequence of the Dunning-Kruger effect. Budget and schedule overruns are possible consequences of underestimating the complexity of the problem.

As an example of the fourth principle above, the Dunning-Kruger effect can cause some decision-makers to discount the warnings and resource requests of engineers and their managers. Decision-makers who are unsophisticated in matters related to technical debt must nevertheless assess the validity of the requests for resources. In making these assessments, these decision-makers may be disadvantaged for a number of reasons, including the following:

  • Decision-makers might hold any of a number of mistaken beliefs about technical debt. For example, many believe that the main causes of technical debt are poor decisions by engineering managers. And others believe that technical debt is the result of slovenly work habits of engineers. Those who hold such beliefs might be reluctant to allocate yet more resources to engineers to address the problem of technical debt.
  • If the advocates of resources for technical debt management are not fully informed about the strategic direction of the enterprise, their requests might be inconsistent with enterprise strategy. As a result of a cognitive bias [Kahneman 2011] known as the halo effect [Thorndike 1920], decision-makers might tend to discount valid portions of the technologists’ proposals, because some portions of those proposals don’t take enterprise strategy into account properly.
  • Decision-makers might be affected by unrealistic optimism [Weinstein 1996], also known as optimism bias. It’s a cognitive bias that can cause them to discount the sometimes-vivid warnings of technologists about the unfavorable consequences of failing to provide technical debt management resources.

Investigations of the degree of correlation between burdens of technical debt and the incidence of rejected or severely curtailed proposals for resources to support technical debt management programs could determine the significance of the Dunning-Kruger effect relative to the problem of technical debt. Also rewarding would be a survey of the nearly 200 known cognitive biases, to determine which of them might be most likely to affect decision-making relative to technical debt, and how best to mitigate the risks they present.

References

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

Other posts in this thread

Posted on

Team composition volatility

Last updated on February 1st, 2018 at 07:31 am

Team composition volatility can interfere with technical debt retirement. In many organizations, project team composition is rarely fixed from beginning to end. In most teams, people who have special knowledge cycle in and out as the work requires. Although these changes in team composition might not interfere with completing a team’s primary objectives, they can affect the team’s ability to retire technical debt that the team incurs over the life of the project. Changes in team composition can also limit the team’s ability to retire specified legacy technical debt that it encounters while working toward its primary objectives.

Now we know what we should have done.
“Now we know what we should have done.” This is one kind of incremental technical debt. When the composition of a development team changes over the course of project, recognizing how things should have been done can become more difficult.

Changes in team composition can increase the likelihood of incurring non-strategic incremental technical debt, and increase the likelihood of failing to retire all legacy debt specified in the team’s objectives.

Most product development, maintenance, and enhancement is carried out in groups we call teams. In this context, team is usually defined as, “a small group of interdependent individuals who share responsibility for outcomes.” [Hollenbeck 2012] However, as Hollenbeck et al. observe, teams vary widely in both skill differentiation and composition stability. My sense is that both factors can potentially influence a team’s ability to retire incremental technical debt. They also affect its ability to achieve its objectives with respect to retiring legacy technical debt.

For example, consider what Fowler calls the Inadvertent/Prudent class of technical debt — “Now we know how we should have done it.” [Fowler 2009] In a project of significant size, some might recognize that different approaches to all or parts of it would have been more effective than the ones that were chosen. The recognition might come several months, or even years, after the work affected was conceived or even completed.

But for the moment, consider only cases in which the recognition occurs during the project, or shortly after completion. In these cases, the people who performed that work might have moved on to other teams in need of their talents and abilities. The people who now realize “how we should have done it” might not be themselves capable of making the needed changes, even if they have the budget or time to do the work. Or worse, they might not have the knowledge needed to recognize that a different approach would have been more effective. In either case, recognized or not, the work performed by the people no longer on the team comprises incremental technical debt. Because of team composition volatility, recognizing or retiring that incremental technical debt can be difficult.

Team composition volatility can also interfere with retiring legacy technical debt. Some projects are specifically charged with retiring a class or classes of legacy technical debt. But others with different objectives might also be charged with retiring instances of specific kinds of legacy technical debt as they encounter them. When team members with special knowledge required for the team’s primary objectives are reassigned, some legacy technical debt can remain un-retired, if retiring that debt from the context in which it occurs requires their special knowledge, and if the reassignment occurs before they can complete the legacy debt retirement. This mechanism is more likely to occur when the legacy debt retirement objective is viewed as subordinate to other business objectives.

Keeping team membership stable has big advantages relative the technical debt management. Said differently, organizations that must shuffle people from team to team as a consequence of controlling costs by reducing headcount can pay big penalties in terms of increasing loads of technical debt.

References

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

Other posts in this thread

Posted on

Technological communication risk

Last updated on February 1st, 2018 at 07:32 am

Technologists must convey what they know about long-term technology trends to enterprise strategists and others. In addition to strategists, the interested parties include internal customers of technology, product owners, product managers, project sponsors, or senior management. Within the enterprise, technologists tend to be among those most knowledgeable regarding the relative alignment between enterprise technological assets and long-term technology trends. Yet technologists frequently fail to communicate this knowledge effectively to those who need it, and that can lead to non-strategic technical debt. I call this phenomenon technological communication risk.

See no evil, hear no evil, speak no evil
Hear no evil, see no evil, speak no evil — the iconic representation of communication failure. Technical debt can result from communication failures due to unwillingness to inform others of what you know, and unwillingness to receive information from others more knowledgeable.

Technological communication risk is the risk that knowledgeable people within the enterprise don’t communicate important knowledge to the people who need it, or the people who need it aren’t receptive to it. Policymakers can address this problem by working to define the roles of all involved to specify the need for this communication, and the need to be receptive to it.

A clear understanding of long-term technology trends is important in managing technical debt. Any significant misalignment between enterprise technological assets and long-term technology trends creates a risk of incurring new technical debt. As technologies evolve, enterprise assets must evolve with them. The gap between those assets and the state of the art is a source of lost productivity and depressed organizational agility, which is our definition of technical debt.

Some technologists are better informed about technology trends than are their internal customers, product owners, product managers, project sponsors, or senior management. Technologists often do attempt to communicate what they know on an informal basis, but unless such communication is expected and defined as an official duty, their superiors and internal customers don’t always welcome the information, especially if they haven’t heard it elsewhere, or if it conflicts with what they’ve learned elsewhere, or if its implications conflict with established strategic positions.

Many technologists are aware that their superiors might not welcome their observations about technological trends or technology-based strategic vulnerabilities or opportunities. For example, one might understand why a technologist might be reluctant to alert an unreceptive senior manager to a suddenly revealed cybersecurity risk that would be very expensive to mitigate. This mechanism is especially strong when deploying adequate cyberdefense would compete for resources with other initiatives already underway, or when the negative consequences of the vulnerability are unlikely to materialize. And some tend to question technologists’ credibility when they blame the technologists for the vulnerability itself.

Situations like these can lead to the formation of new non-strategic technical debt in circumstances such as the following:
  • Management directs the technologists to produce capabilities using approaches known to the technologists to be technological dead ends.
  • Management directs the technologists to implement capabilities that don’t exploit known approaches that could open new and vital lines of business.
  • Management directs the technologists to focus resources on initiatives that in the view of the technologists lack sufficient technological imperative.

Policymakers can mitigate technological communication risk by establishing internal standards that encourage knowledgeable technologists to share what they know with internal customers, project sponsors, or senior management. Similarly, those standards can encourage internal customers, project sponsors, product owners, product managers, and senior management to take heed when knowledgeable technologists do speak up.

References

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

Other posts in this thread

Posted on

Failure to communicate the technical debt concept

Last updated on May 25th, 2018 at 09:46 am

The behavior of internal customers and users of enterprise technological assets can contribute to technical debt formation and persistence. Because of these contributions, introducing effective technical debt management practices requires widespread behavioral changes on the part of those internal customers and users. Accepting these changes, and the initiative and creativity they require, is possible only if people understand the technical debt concept. When they do, they can appreciate the benefits of controlling technical debt, and the consequences of failing to control it. Similarly, when they do not understand or accept the technical debt concept, progress toward effective technical debt management is unlikely. Policymakers can contribute to the planning and execution of the required organizational transformation.

Even when the engineering teams are aware of the technical debt concept, and when they do try to manage technical debt, they cannot make much progress unless they have the support and understanding of their own management, their internal customers, and their customers’ managements. Everyone must understand that controlling technical debt — and retiring it — is a necessary engineering activity that has a business purpose. Everyone must understand that technical debt arises as a result of everyone’s behavior — not just the behavior of technologists.

A tensegrity 3-prism
A tensegrity three-prism. . Read about tensegrity structures.
Image (cc) Bob Burkhardt courtesy Wikimedia.
Part of the job of Management is to see that engineers have what they need to avoid incurring technical debt unnecessarily, and that they have what they need to retire elements of legacy technical debt on a regular basis. Internal customers must understand that communicating their long-term business strategies to Engineering is essential for limiting unnecessary creation of artifacts that become non-strategic technical debt. Only by understanding the technical debt concept can internal customers learn to avoid the behaviors that lead to non-strategic technical debt, and adopt behaviors that limit new technical debt.

Tensegrity structures provide a metaphor for organizations that have mastered the technical debt concept. Tensegrity structures use isolated rigid components in compression, held by a network of strings or cables in tension. The rigid components are usually struts or masts, and they aren’t in contact with each other.

The struts correspond to the users or customers of technological assets. The cables correspond to the engineering activities required to support the customers. The organization is stable relative to technical debt only when the two kinds of elements (struts and cables) work together, each playing its own role, but each appreciating the role of the other.

Advocating for cultural transformation

Advocates of any change to organizational culture are often seen as acting in their own self-interest. That’s a common risk associated with cultural transformation. It’s a risk that can lead to failure when inserting practices related to technical debt management into the culture. The risk is greatest when advocates for change are drawn exclusively from the technical elements of the enterprise. The ideal advocates for these ideas and practices are the internal customers of the technical organizations, and senior management.

References

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

Other posts in this thread

Posted on

Failure to communicate long-term business strategy

Last updated on August 25th, 2018 at 09:47 am

Failure to communicate long-term business strategy can lead to increased technical debt, because engineering decisions that aren’t aligned with business strategy can result in what later becomes technical debt. As business strategy veers away from the assumptions underlying those misaligned engineering decisions, engineers must alter implementations to track the strategy. Technical debt can form during those alteration efforts. Moreover, the expenditure of resources to support those alteration efforts might have been unnecessary if engineers had been better informed about long-term business strategy. In some cases, those resources could have been allocated to other pursuits, including technical debt retirement. To ensure alignment of engineering decisions with long-term business strategy, engineering decision-makers must be aware of long-term and intermediate-term enterprise strategy. When they’re well informed, they can anticipate the engineering needs of the enterprise. And they’re more likely to make decisions that are compatible with strategy.

A plug-in electric vehicle being recharged
Recharging a  plug-in electric vehicle. The dominance of petroleum-powered vehicles is nearing its end. Further investment in the petroleum-based fuels infrastructure is now  inconsistent with what the global economy has chosen as its strategic intent. Electric vehicles are still out of the reach of most consumers, but they would be wise to favor long range vehicles. As electric vehicles become ascendant, petroleum filling stations will become more widely spaced, because they add to the metaphorical interest charges on the technical debt of yet-to-be-retired petroleum powered vehicles. During the transition to electric power dominance, a long-range petroleum-powered vehicle offers clear advantages over its shorter-range cousins.

Moreover, the effect is bi-directional. Strategists can benefit from understanding the effect their strategies have on technological activity. For example, consider the process of choosing among strategic options. A favorable outcome is more likely if strategists know the effects of each option on the technical debt portfolio.

To gain effective control of technical debt, senior management must regard the technical elements of the enterprise as strategic partners [Woodard 2013] [Ross 2000] [Brenner 2016a]. Policymakers can make important contributions to enhancing communication between strategists and technologists.

For example, when engineers know the general direction of the enterprise, they can focus efforts on assets that are compatible with future needs. Inversely, when they’re unaware of what the business strategy might soon require, they’re more likely to make decisions that they must later rescind.

What about legacy technical debt retirement?

Analogous considerations apply to legacy technical debt  retirement efforts. Major technical debt retirement efforts are often subject to review for alignment with enterprise strategy. But we tend not to review incidental retirement efforts that occur in the context of routine maintenance or development. Consequently, engineers might allocate effort to incidental  debt retirement unnecessarily if the asset is due for overhaul or replacement. Communicating long-term strategy effectively is likely the most reliable way to prevent such misspent effort.

Some managers elect to communicate business strategy to technologists only when they “need to know.” Often, technologists needed to know long before that.

References

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Other posts in this thread

Posted on

Non-technical precursors of non-strategic technical debt

Last updated on April 29th, 2018 at 06:36 am

Non-strategic technical debt is technical debt that appears in the asset without strategic purpose. We tend to introduce non-strategic technical debt by accident, or as the result of urgency, or from changes in standards, laws, or regulations—almost any source other than asset-related engineering purposes. In this group of posts I examine a variety of precursors of non-strategic technical debt that are not directly related to technology. Sources of these precursors include:

  • Communication between and among people
  • Organizational policies relating to job assignments
  • Cognitive biases [Kahneman 2011]
  • Performance management policy
  • Incentive structures
  • Organizational structures
  • Contract language
  • Outsourcing
  • …and approaches to dealing with budget depletion.

The cables of the Brooklyn Bridge are an example of non-strategic technical debt
Some of the suspension cables of the Brooklyn Bridge. Washington Roebling, the chief engineer, designed the cables to be composed of 19 “strands” of wire rope [McCullough 1972]. Each strand was to be made of 278 steel wires. Thus, the original design called for a total of 5,282 wires in each of the main cables. After the wire stringing began, the bridge company made an unsettling discovery. The wire supplier, J. Lloyd Haigh, had been delivering defective wire by circumventing the bridge company’s stringent inspection procedures. In all, Roebling estimated that 221 U.S. tons (200 metric tons) of rejected wire had been installed in the bridge. This was a significant fraction of the planned total weight of 3,400 U.S. tons (3,084 metric tons). Because they couldn’t remove the defective wire, Roebling decided to add about 150 wires to each main cable. That extra wire would be provided at no charge by Haigh [Talbot 2011]. I can’t confirm this, but I suspect that Roebling actually added 152 wires, which would be eight wires for each of the 19 strands, to make a total of 286 wires per strand, for a total of 5,434 wires. The presence of the defective wire in the bridge cables—which remains to this day—is an example of technical debt. The fraud perpetrated by Haigh illustrates how malfeasance can lead to technical debt.
I use the term precursor instead of cause because none of these conditions leads to technical debt inevitably. From the perspective of the policymaker, we can view these conditions as risks. It’s the task of the policymaker to devise policies that manage these risks.

McConnell has classified technical debt in a framework that distinguishes responsible forms of technical debt from other forms [McConnell 2008]. Briefly, we incur some technical debt strategically and responsibly, and we retire it when the time is right. We incur other technical debt for other reasons, some of which are inconsistent with enterprise health and wellbeing.

The distinction is lost on many. Unfortunately, most technical debt is non-strategic. We would have been better off  if we had never created it. Or if we had retired it almost immediately. In any case we should have retired it long ago.

It’s this category of non-strategic technical debt that I deal with in this group of posts. Although all technical debt is unwelcome, we’re especially interested in non-strategic technical debt, because it is usually uncontrolled. In these posts I explore the non-technical mechanisms that lead to formation of non-strategic technical debt. Schedule pressure is one exception. Because it’s so important, it deserves a thread of its own. I’ll address it later.

Common precursors of non-strategic technical debt

Here are some of the more common precursors of non-strategic technical debt.

I’ll be adding posts on these topics, so check back often, or subscribe to receive notifications when they’re available.

References

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Related posts

  • Malfeasance can be a source of technical debt
    Although deploying technical debt management policies is necessary, it might not be sufficient. Malfeasance can lead to incurring new technical debt. It can prevent retiring—or even identifying—existing technical debt. Correcting, detecting, and preventing malfeasance must be goals of technical debt management policy.
  • The Broken Windows theory of technical debt is broken
    Because there is no known controlled study providing evidence that supports the application of the Broken Windows theory to technical debt management, Broken Windows as applied to technical debt management must be regarded as an intriguing conjecture awaiting scientific examination.
  • The Tragedy of the Commons is a distraction
    The urge to control technical debt by designating one group as the “owner” of the asset doesn’t avoid the tragedy of the commons. It leads inevitably to duplication and more technical debt. A more effective approach: Communities that share specific assets must devise customs, rules, and sanctions for managing them.
  • Organizational psychopathy: career advancement by surfing the debt tsunami
    Organizational psychopathy can be the dominant contributing factor in generating technical debt when the primary beneficiary of a decision is the decision-maker or the advocate, and when he or she intends to move on to employment elsewhere before the true cost of the debt becomes evident.
  • Contract restrictions can lead to technical debt
    Contract restrictions can lead to technical debt. When contractor organizations customize an asset they own for customers, they usually mean to retain ownership of the asset. But segregating the asset from the customizations can lead to technical debt. Here's how.
Posted on

Technical debt: monochronic and polychronic cultures

Last updated on June 2nd, 2018 at 09:50 pm

In the context of today’s prevalence of global teams, the word culture evokes consideration of the differences between different national cultures. We think of the difference between Singapore and Latvia. Or we might also consider regional differences between Massachusetts and Texas. Or between San Francisco and Los Angeles. These differences are real. They do affect the way the inhabitants of these places deal with technical debt. But these cultural differences are topics for another time.

A Simmental cow and calf
A Simmental cow and calf

For now, let’s consider how a particular cultural attribute of teams or groups in organizations affects how they manage technical debt. Specifically, how we regard time correlates closely with our preferences, biases, and even the way we think about approaching technical debt.

Anthropologist Edward Twitchell Hall, Jr. (1914-2009) developed fundamental concepts for thinking about how we experience space and time. In The Silent Language [Hall 1973], he introduced the concepts of polychronic time and monochronic time. They represent two different ways human cultures relate to time.

People and cultures with a monochronic orientation (M-People) regard time as linear. M-People are most comfortable when they undertake only one task at a time. They tend to sequence tasks one after the other. Their motto is “Work on one and get it done.” When M-People must tackle more than one task in an hour, they divide the hour into blocks. They dedicate each block to one task. But time division isn’t always possible. When M-People work in groups, and when they must do more than one thing simultaneously, they divide their groups. Then each subgroup can focus on one task at a time. This is more or less how conventional (pre-Agile) project teams work.

People and cultures with a polychronic orientation (P-People) regard time differently. They define time more in terms of completed tasks than in terms specified by the hands of a clock. For example, a farmer might define time by what’s happening: it’s time for planting, for harvest, for haying, for canning, or for calving. Finer increments are also defined by what’s happening: it’s time for milking, time for breakfast, or it’s sundown or sunset. More than one thing can be happening at any given time.

In organizational cultures, the differences between monochronic and polychronic orientations are evident at every level of activity. For example, in a meeting of M-People, only one person has the floor at any one time. M-People schedule their agenda items, and after they move to the next agenda item, they don’t return. (Well, they do sometimes return, but they aren’t comfortable when they do) A meeting of P-People might have several people talking at once. They bounce from topic to topic as the discussion calls for it. M-People are very uncomfortable in P-style meetings; P-People are just as uncomfortable in M-style meetings [Brenner 2018].

M-approaches can work just fine when we know in advance what the tasks entail. We can carve up our total work into pieces and address them one at a time, because they either don’t interact much, or because we can control their interactions. And we can partition the work across the available people and groups, each working on one piece of the effort.

And that has worked well for much of the work involved in producing and maintaining today’s complex technological assets.

But M-approaches can get into trouble when we work on something that’s complex or unfamiliar. When we don’t have (and can’t devise) a detailed plan of the work, dividing the work either by time or by task can be difficult to get right. To do this kind of work, we must depend more on interpersonal relationships and close teamwork than on scheduling and task partitioning. M-style approaches can flounder. And that’s where P-style approaches have big advantages.

In polychronic work styles, we’re more focused on accomplishments than schedule, because we don’t really know enough about the work to make a detailed schedule. In some instances, estimating how much work something might take is essentially impossible because we don’t understand the task in enough detail. We must then let events guide us, in the way the seasons guide the farmer. The people doing the work must adapt as they go, which requires strong relationships, tight collaboration, and trust.

And that brings us to technical debt management. We can address some kinds of technical debt with M-style approaches. We understand those debts and we know how working on their retirement will affect other work. But many forms of technical debt are so entangled with the rest of the organization’s efforts that P-style approaches are more likely to be successful.

For example, consider a project that’s aimed at retiring some kinds of technical debt. That might require access to assets that are critical to certain revenue streams. And suppose that ongoing work unrelated to retiring that technical debt might also affect these same assets. Coordinating all these efforts is complicated, even when we’re planning for only the next few weeks. But coordinating these efforts six months or twelve months in advance is probably impossible, because we can’t create schedules reliable enough to support such planning. The schedules of the various interlocking activities aren’t well enough controlled for that, any more than a dairy farmer controls the calving schedule. The cows control the calving schedule.

For many technical debt management activities, M-style approaches are simply unworkable. We cannot answer with useful precision questions like these:
  • How much will it cost—total—to retire the technical debt of that kind?
  • How long will it take?
  • Will that debt retirement interfere with our plans for the rollout of the Marigold products next September?

Conventional project management might not work well for technical debt retirement efforts. Regarding them as “projects,” with defined schedules, defined budgets, and defined staff assignments might be a serious error. Agile approaches, which are inherently more P-style, might work better.

References

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Brenner 2018] Richard Brenner. “Polychronic Meetings,” Point Lookout 18:1, January 3, 2018.

Available here; . Forthcoming.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Hall 1973] Edward T. Hall. The Silent Language. New York: Anchor Books, 1973.

Originally published in 1959. Order from Amazon

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Other posts in this thread

Posted on

With all deliberate urgency

Last updated on June 2nd, 2018 at 09:21 pm

One of the drivers of technical debt—one of the most important generators of technical debt—is crossing the fine line from urgency to panic when it comes to deadlines.

On October 12, 2017, the Chicago Cubs and Washington Nationals met at the Nationals’ home field for Game 5 of the National League Divisional Series (American baseball playoffs). At that point, the series was tied 2-2. It was a high-pressure game that would decide the division championship. By the end of the second inning, the Nationals led 4-1. They went on to lose, 9-8.

Pressure situations are tough.

Dusty Baker as Manager of the Washington Nationals in 2017
Dusty Baker (center) as Manager of the Washington Nationals, at a game at the home field of the Baltimore Orioles, May 8, 2017. At the right is Davey Lopes, the first base coach. I can’t identify the man on the left. If you can, let me know. Photo (cc) Keith Allison courtesy Wikimedia
After losing the first game of the series, Dusty Baker, the Nationals’ manager, conducted a press conference before Game 2. A difficult situation for any manager. He’s quoted [Gonzales 2017] as saying, “There’s a fine line between urgency and panic, and the thing that you never want to do, you never want to panic.”

These are words of wisdom that apply just as well in business, especially with respect to technical debt. Consider this scenario:

Sales at Unbelievable Growth, Inc.,(UGI) have been only fair this fiscal year—far from “unbelievable.” But a new product is under development, an app for Android and iPhone called StrawIntoGold 1.0. It has an uncanny ability to predict the price movements of specific common stocks over the next 60 seconds. (This is totally fictitious—don’t bother surfing for UGI or StrawIntoGold) Unfortunately, StrawIntoGold development is far behind schedule. After the all-hands meeting yesterday, the core engineering team had a three-hour meeting of its own. They found some ways to wrap things up in the next ten days. They think they can do it, but they’ll be eliminating some testing, and performing other tests manually. And they plan to re-use some code from the beta version that they had previously decided to replace.

If the UGI engineers succeed, they will be incurring significant technical debt. They have crossed the “urgency line,” and although it’s too soon to say definitively that they’ve panicked, my personal experience suggests that the risk of reaching some degree of panic is high. And that risk will get higher as the deadline approaches.

Urgency focuses our energy and attention. As Dusty Baker says, “You have to be of the coolness of mind, but then bring desire to succeed in your heart, and then respond.” When urgency is deliberate, urgency gets the job done. Deliberate urgency is what Kotter calls healthy urgency [Kotter 2014].

Panic is something else. It can cause us to choose to cut corners, a choice commonly cited as a source of technical debt. When it makes clear thinking difficult, it impedes memory, increases error rates, reduces attention spans, and contributes to toxic conflict. In short, it makes any kind of brainwork more difficult, less effective, and less reliable.

It’s reasonable to suppose that panic isn’t helpful in avoiding or removing technical debt in any kind of technological asset. It’s just as reasonable to suppose that panic actually contributes to technical debt formation and persistence.

Urgency, good. Panic, bad. Once you let panic into an organization’s culture, the effect on technical debt is predictable. Over time, technical debt will increase out of control.

So what alternatives do the UGI engineers have? In most organizations, they would probably have no alternative. StrawIntoGold 1.0 would be offered to customers in a very sorry state that might not affect its performance, but its maintainability—its sustainability—would be poor. The prospects for version 2.0 would not be bright.

But some organizations do find alternative approaches. What they do, in effect, is redefine the word “done” as it applies to the StrawIntoGold 1.0 product. In that redefined form, “done” has two stages.

In Stage 1, UGI does release StrawIntoGold 1.0, despite its unsustainable state. But then UGI management makes a clever decision. Instead of moving the StrawIntoGold team on to begin version 2.0, or what is worse, reassigning the team members to other projects, UGI management charters the StrawIntoGold 1.0 team with retiring the technical debt they incurred to meet the version 1.0 deadline. They restrict the team’s efforts to technical debt retirement only, so that they produce a version 1.1 that is identical to version 1.0 from the customer perspective. That becomes Stage 2 of “done.” They defer any work on version 2.0, because starting 2.0 would cause fragmentation of the 1.0 team. StrawIntoGold 1.0 is thenceforth shelved, and any new orders are filled with StrawIntoGold 1.1. Then work on version 2.0 begins.

By carefully managing their technical debt, UGI can make its products more sustainable in the very dynamic mobile device app market. They exploit urgency deliberately. They do not panic. Then, at UGI, situations like the one that hit StrawIntoGold 1.0 become rare.

Do you have any teams that have crossed the fine line between urgency and panic?

References

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Brenner 2018] Richard Brenner. “Polychronic Meetings,” Point Lookout 18:1, January 3, 2018.

Available here; . Forthcoming.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gonzales 2017] Mark Gonzales. “Nationals manager Dusty Baker preaches calm vs. Cubs,” ChicagoTribune.com, October 7, 2017.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Hall 1973] Edward T. Hall. The Silent Language. New York: Anchor Books, 1973.

Originally published in 1959. Order from Amazon

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kotter 2014] John P. Kotter. “To Create Healthy Urgency, Focus on a Big Opportunity,” Harvard Business Review, February 21, 2014.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Other posts in this thread

Posted on

On assigning responsibility for creating technical debt

Last updated on June 2nd, 2018 at 08:50 pm

When we discover an issue within our organizations, two intertwined imperatives demand attention: “How did this happen?” and “What do we do about it?” As we address the former question, almost inevitably we begin to decide who was responsible for creating the problem. Even if we succeed in avoiding blamefests (see [Brenner 2005a]) we can still make gross errors.

An engineer attending a meeting
An engineer attending a meeting with 14 other engineers. You can’t see the other 14 because they’re at least 4,000 miles away in four separate locations.

Assigning responsibility for creating technical debt provides some clear examples of the many dangerous traps that await us on the path to Truth. How we assign responsibility is due, in part, to patterns of organizational culture.

The causes of growth in technical debt are numerous. They include—among many others—insufficient resources, schedule pressure, existing technical debt, changes in strategic direction, changes in law or regulations, and the risks associated with creating first-of-kind solutions to difficult problems. In most engineering activity new technical debt is inevitable. How we deal with it is up to us.

Unfortunately, many organizations don’t provide the time or resources needed to retire that new technical debt on a regular basis.

When technologists—engineers, their managers, or others in technical roles—try to alert the rest of the organization (non-technologists) to the problems that follow from continually accumulating technical debt, they often meet resistance from non-technologists. Technologists usually hope that the resistance can be resolved with an intensive education program.

Sometimes that works. Sometimes technologists do receive the additional resources, time, and cooperation they need to start retiring the accumulated technical debt, and to avoid adding more debt to the burden they already carry.

Mostly, though, education programs don’t work, for reasons beyond mere misunderstanding of the issue. One fundamental problem is the word “technical” in the term technical debt. Non-technologists must be forgiven for believing that since technical debt is inherently technical, it follows that its causes are also technical, that technologists are solely responsible for creating technical debt, and that non-technologists play no role. Those conclusions are, of course, false, but the beliefs persist, and many non-technologists adopt the view that “It’s your problem—fix it.”

A second cause of misconceptions about the causes of technical debt lies in the belief that technologists aren’t working very hard. This belief is founded on assumptions many of us make about what diligent work looks like. Many non-technologists have roles in General Management, Sales, Marketing, or Business Development. They’re working hard when they’re in contact with each other or with people external to the enterprise. They’re traveling, conversing by telephone, or attending or hosting meetings. By contrast, technologists are working hard when they’re at their desks, or attending (face-to-face or virtual) meetings. They do attend meetings off premises, but they do so at much lower rates than do many non-technologists.

When non-technologists assess the technologists’ work ethic, they tend to use the same standards and assumptions they apply to themselves. They under-estimate the technologists’ activity level because outwardly, technologists appear more often to be what non-technologists would regard as “idle”—sitting at their desks, thinking, or typing [Schein 2016].

All of this shows how language, stereotypes, and assumptions conspire to lead  us to misallocate responsibility for creating technical debt. Some believe that technologists are solely responsible for technical debt, because only they can create it, and they aren’t working very hard to do anything about it. Proceeding from that conclusion, finding a resolution of the problem will be difficult indeed. Language, stereotypes, and assumptions can be traps.

References

[Brenner 2005a] Richard Brenner. “Is It Blame or Is It Accountability?” Point Lookout 5:51, December 21, 2005.

Available here; Retrieved December 30, 2016.

Cited in:

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Brenner 2018] Richard Brenner. “Polychronic Meetings,” Point Lookout 18:1, January 3, 2018.

Available here; . Forthcoming.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gonzales 2017] Mark Gonzales. “Nationals manager Dusty Baker preaches calm vs. Cubs,” ChicagoTribune.com, October 7, 2017.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Hall 1973] Edward T. Hall. The Silent Language. New York: Anchor Books, 1973.

Originally published in 1959. Order from Amazon

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kotter 2014] John P. Kotter. “To Create Healthy Urgency, Focus on a Big Opportunity,” Harvard Business Review, February 21, 2014.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Other posts in this thread

Posted on

Organizational culture and technical debt

Last updated on June 29th, 2018 at 08:20 pm

Organizational culture affects—and is affected by—everything the organization does. It is complex beyond imagining. It both creates leaders and is shaped by them [Schein 2016]. Fortunately, for our narrow purposes—namely, improving how we manage technical debt—we need to explore neither every available model of culture nor their dynamics. All we need to know is how to adjust cultures so that they’re more amenable to technical debt management efforts. Even so, that’s an ambitious objective.

A bonsai tree
A bonsai tree. We cannot control the growth of bonsai in detail. We can only make suggestions, and hope that the tree will follow our guidance. So it is with organizational culture.

Organizational culture is the collection of values, beliefs, and principles shared among the people of the organization. Some of these elements are within the awareness of the organization’s people, but some might not be. Culture excludes behavior; behavior is a result of the combined effects of personal choices and organizational culture.

Our objective is ambitious because it is at once specific and general. It is general in the sense that as a goal, we would like to devise prescriptions for all organizational cultures as they strive to be more effective in managing technical debt. It is specific in the sense that we’re concerned only with those cultural factors that affect technical debt management.

But aren’t there enough technical problems to keep us occupied before we get involved in all that culture change stuff? The answer to that, I believe, is yes, but we can make more rapid progress if we allow our technical investigations to be guided by considerations of culture.

Some of our best minds have been working on the technical debt problem for more than two decades, mostly from the technological perspective. True, we haven’t given the issue the attention and resources it deserves. That might be changing now, but one can reasonably ask a simple question: If some of the causes of the technical debt problem are cultural, and if we begin to make some technological progress in reducing the burden of technical debt, wouldn’t the cultural causes begin to dominate at some point? If that were to happen, the effectiveness of our technical solutions would appear to decline, even though they might be “right,” because technical solutions cannot address cultural problems.

It seems only prudent to prepare for that case by examining the role of culture in creating technical debt and preventing its retirement.

A reasonable goal might be to define a set of attributes an organizational culture should have if the organization wants to limit the contributions of culture to the formation and persistence of technical debt. I suspect that for most cultures, change would be required, because most organizations now carry more technical debt than they would care to acknowledge.

As with all culture change efforts, we seek, in the end, to modify the behavior of the people who live within that culture. We want to modify aspects of the culture so as to lead its people to be more likely to make choices that are consistent with prudent technical debt management.

With that goal in mind, I’m opening a thread in this blog that I hope will stimulate your thinking and mine about how organizational culture and technical debt interact. This post will contain an outline of these ruminations. Here’s what I have so far:

References

[Brenner 2005a] Richard Brenner. “Is It Blame or Is It Accountability?” Point Lookout 5:51, December 21, 2005.

Available here; Retrieved December 30, 2016.

Cited in:

[Brenner 2016a] Richard Brenner. “The Psychology and Politics of Technical Debt: How We Incur Technical Debt and Why Retiring It Is So Difficult,” Cutter Business Technology Journal, 29:3, 2016, 21-27.

Cited in:

[Brenner 2018] Richard Brenner. “Polychronic Meetings,” Point Lookout 18:1, January 3, 2018.

Available here; . Forthcoming.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gonzales 2017] Mark Gonzales. “Nationals manager Dusty Baker preaches calm vs. Cubs,” ChicagoTribune.com, October 7, 2017.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Hall 1973] Edward T. Hall. The Silent Language. New York: Anchor Books, 1973.

Originally published in 1959. Order from Amazon

Cited in:

[Hollenbeck 2012] John R. Hollenbeck, Bianca Beersma, and Maartje E. Schouten. “Beyond Team Types and Taxonomies: A Dimensional Scaling Conceptualization for Team Description,” Academy of Management Review, 37:1, 82–106, 2012. doi:10.5465/amr.2010.0181

Available: here; Retrieved: July 8, 2017

Cited in:

[Kahneman 2011] Daniel Kahneman. Thinking, Fast and Slow. New York: Macmillan, 2011.

Order from Amazon

Cited in:

[Kotter 2014] John P. Kotter. “To Create Healthy Urgency, Focus on a Big Opportunity,” Harvard Business Review, February 21, 2014.

Available: here; Retrieved: December 13, 2017.

Cited in:

[Kruger 1999] Justin Kruger and David Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology, 77:6, 1121-1134, 1999.

Cited in:

[McConnell 2008] Steve McConnell. Managing Technical Debt, white paper, Construx Software, 2008.

Available: here; Retrieved November 10, 2017.

Cited in:

[McCullough 1972] David McCullough. The Great Bridge: The epic story of the building of the Brooklyn Bridge. New York: Simon and Schuster, 1972.

Order from Amazon

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Talbot 2011] J. Talbot. “The Brooklyn Bridge: First Steel-Wire Suspension Bridge.” Modern Steel Construction 51:6, 42-46, 2011.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Thorndike 1920] Edward L. Thorndike. “A constant error in psychological ratings,” Journal of Applied Psychology, 4:1, 25-29, 1920. doi:10.1037/h0071663

The first report of the halo effect. Thorndike found unexpected correlations between the ratings of various attributes of soldiers given by their commanding officers. Although the halo effect was thus defined only for rating personal attributes, it has since been observed in assessing the attributes of other entities, such as brands. Available: here; Retrieved: December 29, 2017

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15:1, 1-8, 1996. doi:10.1521/jscp.1996.15.1.1

Cited in:

[Woodard 2013] C. Jason Woodard, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy. “Design Capital and Design Moves: the Logic of Digital Business Strategy,” MIS Quarterly 37:2, 537-564, 2013.

Cited in:

Related posts

  • Technical debt use disorder
    Technical debt use disorder is a pattern in which the organization can’t seem to retire much of its technical debt, or can’t stop incurring increasing burdens of technical debt, even though everyone in the organization realizes—or almost everyone realizes—that technical debt is harming the organization.
  • Technical debt smell
    Technical debt smell would be a useful indicator of severe technical debt problems. It doesn’t have a real smell, but it might have a metaphorical smell.
  • Using SMART goals for technical debt reduction
    Reducing the enterprise burden of technical debt by setting “SMART goals” in a naïve way might not work. Measuring technical debt is notoriously difficult, and in any case the interest charges matter more than debt volume. Effective SMART goals for technical debt reduction must be closely related to behavior.
  • Cultural debt can be the primary driver of technical debt
    Cultural debt can be expensive, because, like technical debt, it can incur ongoing metaphorical interest charges. Cultural debt is the unfulfilled obligation to alter organizational culture so as to better align it with organizational imperatives.
  • Technical debt: monochronic and polychronic cultures
    Conventional project management might not be applicable to technical debt retirement efforts. Regarding them as “projects,” with defined schedules, defined budgets, and defined staff assignments might be a serious error, culturally speaking. Agile approaches, which are inherently more P-style in nature, might be required.
Show Buttons
Hide Buttons