How performance management systems can contribute to technical debt

Few performance management systems provide guidance with respect to behaviors relating to technical debt, perhaps because technical debt is not widely understood, or perhaps because technical debt isn’t seen as a concern for the performance of anyone but engineers and their managers. Still, organizations that expect to gain control of technical debt must ensure that performance standards are clear about expectations with respect to behaviors that could affect technical debt.. In organizations in which technical debt currently plays a minor role, if any, in the performance management system, policymakers can advocate for effective changes, if they understand what the appropriate role for performance management is in controlling technical debt. This post should be helpful.

A dog receiving a reward
A dog receiving a reward. Many performance management systems implement a model that assumes that the correct configuration of incentives and disincentives will produce the desired levels of performance. This theory is questionable.

A fundamental premise of many performance management systems is that incentives can encourage desirable behavior and disincentives can discourage undesirable behavior. Unfortunately, serious questions have arisen about the effectiveness of these behavioral control mechanisms in general [Kohn 1999]. The problem is that employees find ways to harvest incentives without exhibiting the desired behavior. Likewise, they find ways to circumvent disincentives while continuing to exhibit undesired behavior.

Moreover, specifically for technical debt management, behavioral control is especially problematic, because some of the behaviors that must be controlled are inherently immeasurable. For example, the design of an incentive structure to encourage legacy technical debt retirement is debatable, given the technical difficulties involved in even defining legacy technical debt, let alone measuring its size.

Managing performance vis-à-vis technical debt, therefore, presents a problem of the kind Austin calls partially supervised [Austin 1996]. Supervising engineers whose work touches on assets that bear technical debt can only be partial, because measuring technical debt is only partially practical given the state of the art. Austin shows how partial supervision frequently leads to dysfunctional performance management, but the problem is especially vexing for managing technical debt. For example, in some cases, engineers’ work can incur new technical debt that remains unrecognized for months or years after the work is completed. To fully supervise such work would require inventing retroactive incentives and disincentives, which not only do not exist, but which are of questionable legality in most jurisdictions.

Although incentives and disincentives cannot serve to manage performance relative to technical debt, a very effective model is available. Enterprise leaders could communicate their intentions relative to technical debt, and empower the people of the organization to take steps to reduce debt. In the United States military, and others as well, a doctrine that implements this approach is called commander’s intent [Mattis 2008].

Gen. Mattis offers five principles that guide what the military calls “effect-based operations.” For technical debt management, the effect we seek is rational control of the technical debt portfolio. Here are his five principles, transformed to the field of technical debt.

  1. Technology development, maintenance, and cyberdefense in the future will require a balance of conventional and unconventional approaches.
  2. Technology evolves rapidly, and we must be willing to adapt our methods.
  3. Technologies are dynamic with an infinite number of variables; therefore, it is not scientifically possible to accurately predict the level of technical debt that will result from any given effort. To suggest otherwise runs contrary to historical experience and the nature of modern technological assets.
  4. We are in error when we think that what works (or does not work) in efforts involving one technology in one enterprise will be universally applicable to all technologies in all enterprises.
  5. Finally, to paraphrase General Sherman, “Every attempt to make technical debt management easy and safe will result in humiliation and disaster.”

Most organizations rely on supervisors to communicate the analog of commander’s intent to their subordinates. Currently, it’s fair to say that few supervisors outside the technology-oriented elements of the enterprise communicate much about technical debt to their subordinates.

That situation might explain why most performance management systems encourage behaviors that unwittingly expand the body of technical debt, especially for non-technologist performers. There are situations in which the widely applauded actions of the outstanding performer are such as to incur technical debt strategically and responsibly. Technical debt so incurred is what McConnell calls Type II [McConnell 2008] and what Fowler calls Deliberate and Prudent [Fowler 2009]. But most performance management systems, especially for non-technologists, say nothing about technical debt, and thus risk encouraging behaviors that indirectly exacerbate the problems associated with technical debt.

Distinguishing responsible and irresponsible behaviors is possible only if understanding of the nature of technical debt is widespread in the organization, even beyond the technologists. Here’s an example:

It was ambitious, what advocates called a “stretch goal,” but the VP of Marketing approved the plan to get the new app released by the end of the next fiscal quarter. After a month of meetings, and much jawboning, the CTO agreed to find a way to make it happen, despite serious objections from the VP of New Product Development. Engineers and testers were able to meet the date, but they had to incur significant technical debt, and when they asked for resources to retire that debt after the release, the VP of Marketing opposed the request, because she needed additional resources for the promotional campaign due to our late entry into the market.

Stories like this illustrate scenarios in which technical debt considerations are consistently assigned a lower priority than goals related to market timing, market development, and revenue generation. Standards for setting priorities closely parallel the standards defined in the performance management system. Indeed, the goal of performance management should be to support enterprise goals. In the scenario above, the organization might meet the immediate goal of a successful release, but it does so by incurring technical debt, thereby imperiling the next release. In this scenario, it’s evidently necessary to change the performance management system to achieve a better balance between immediate goals and the near-term future goals.

Since anyone in the enterprise can take actions or make decisions that lead to incurring new technical debt, or cause existing technical debt to remain in place, organizations need performance standards that guide employees with respect to technical debt. To provide guidance for distinguishing responsible behavior from irresponsible behavior, performance management systems must acknowledge the potential of any employee to affect technical debt, constructively or otherwise. Performance management systems must be reviewed with respect to alignment with technical debt policy, and adjusted to encompass a mechanism analogous to Mattis’s vision of commander’s intent.

References

[Fowler 2009] Fowler, Martin. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009. here . Retrieved January 10, 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999.

Order from Amazon

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here Retrieved November 9, 2017.

Cited in:

[McConnell 2008] McConnell, Steve. Managing Technical Debt, white paper, Construx Software, 2008.

Available at: www.construx.com/Page.aspx?cid=2801 Retrieved November 10, 2017.

Cited in:

Related posts

Self-sustaining technical knowledge deficits during contract negotiations

Enterprises that grow by acquisition find themselves acquiring the technological assets of the organizations they acquire. And most enterprises acquire technological assets by other means as well. In either case, the contract negotiation teams need technical knowledge to evaluate and project the effects of these acquisitions on total enterprise technical debt. But as total enterprise technical debt grows, the capacity of enterprise technologists to support new contract negotiations declines, which leads to a self-sustaining cycle of technical knowledge deficits. Policymakers and strategic planners are likely the most effective possible advocates for breaking the cycle by hiring more technologists.

Avoid technical knowledge deficits in important contract negotiations
Contract negotiations can be complex

Negotiating contracts with vendors that provide outsourcing services or subcontracting work, or with organizations to be acquired, requires a sophisticated appreciation of the technical debt status of the assets acquired or to be acquired. The technical debt in question includes more than just the debt borne by the asset as it stands in its pre-acquisition context. It also includes the debt that the asset will carry after it’s inserted into the asset portfolio of the acquiring enterprise.

These two debts — pre-acquisition and post-acquisition — can differ, because the interfaces, standards, and approaches of the acquiring organization likely differ from those prevailing within the vendor organization or the acquired organization. Knowledge of the interfaces, standards, and approaches of both parties to the transaction is therefore required to make a valid assessment of the total post-acquisition levels of technical debt.

The enterprise negotiation team therefore requires the services of technologists who are familiar with the maintenance, extension, and cybersecurity work that will be performed on the acquired assets post-acquisition. When the technical debt situation in the enterprise reaches a level so serious that it requires the full attention of all available technologists, they cannot be spared for negotiating contracts. If this happens, then contract negotiation teams could experience a deficit of knowledge concerning the consequences of acquiring assets laden with technical debt. That leads to increasing levels of non-strategic technical debt, which then has the potential to exacerbate the technical knowledge deficit for the negotiating teams.

This situation is an example of what’s commonly called a vicious cycle. After technical debt has reached a critical level, there are really only two tactics that can break the cycle — get more engineers, or suspend some work.

References

[Fowler 2009] Fowler, Martin. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009. here . Retrieved January 10, 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999.

Order from Amazon

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here Retrieved November 9, 2017.

Cited in:

[McConnell 2008] McConnell, Steve. Managing Technical Debt, white paper, Construx Software, 2008.

Available at: www.construx.com/Page.aspx?cid=2801 Retrieved November 10, 2017.

Cited in:

Related posts

The Dunning-Kruger effect can lead to technical debt

The Dunning-Kruger effect [Kruger 1999] can lead to formation or persistence of technical debt in two ways. First, it can cause technologists or their managers to overestimate their ability to maintain the resource focus needed for retiring technical debt in a timely fashion. Second, it can cause senior managers to be reluctant to accede to resource requests of technologists and their managers in support of technical debt management programs.

Cropped detail from Charles Robert Darwin, a painting by John Collier
Cropped detail from Charles Robert Darwin, a painting by John Collier (1850-1934), given to the National Portrait Gallery, London, in 1896. Darwin writes, in The Descent of Man (1871): “… ignorance more frequently begets confidence than does knowledge …” which is the essence of the Dunning-Kruger effect. Image courtesy WikiQuote.

Kruger and Dunning conducted experiments that yielded results consistent with the following four principles (paraphrasing):

  1. Incompetent individuals, compared to their more competent peers, tend to dramatically overestimate their own ability and performance
  2. Incompetent individuals, compared to their more competent peers, tend to be less able to gain insight into their own true levels of performance
  3. Incompetent individuals can gain insight about their shortcomings, but, paradoxically, this comes about by gaining competence
  4. Incompetent individuals, compared to their more competent peers, are less able to recognize competence when they see it

The first three principles lead to distorted assessments of one’s own capabilities. The fourth principle leads to distorted assessments of the capabilities of others.

As an example of distorted self-assessment, consider a team or its managers who must undertake retirement of some types of technical debt in the course of enhancing or repairing an asset. Such a task plan seems at first to offer efficiencies, because the engineers can readily make both kinds of changes at one go. Metaphorically, if we must go to the store for milk, we can also pick up bread while we are there, rather than making two trips.

However, modifying an existing complex technological asset is unlike shopping for bread and milk. The two kinds of modifications — debt retirement and asset enhancement or repair — might seem at first to be separable, and often they are. But if they are not separable, and the two tasks are undertaken together, testing and debugging can become extremely complicated, because of interactions between defects in the two kinds of modifications. Under some circumstances, an experienced team and its managers might be more likely to anticipate these difficulties. An inexperienced team and its managers might be more likely to underestimate the difficulties, as a consequence of the Dunning-Kruger effect. Budget and schedule overruns are possible consequences of underestimating the complexity of the problem.

As an example of the fourth principle above, the Dunning-Kruger effect can cause some decision-makers to discount the warnings and resource requests of engineers and their managers. Decision-makers who are unsophisticated in matters related to technical debt must nevertheless assess the validity of the requests for resources. In making these assessments, these decision-makers may be disadvantaged for a number of reasons, including the following:

  • Decision-makers might hold any of a number of mistaken beliefs about technical debt. For example, many believe that the main causes of technical debt are poor decisions by engineering managers. And others believe that technical debt is the result of slovenly work habits of engineers. Those who hold such beliefs might be reluctant to allocate yet more resources to engineers to address the problem of technical debt.
  • If the advocates of resources for technical debt management are not fully informed about the strategic direction of the enterprise, their requests might be inconsistent with enterprise strategy. As a result of a cognitive bias known as the halo effect [Thorndike 1920], decision-makers might tend to discount valid portions of the technologists’ proposals, because some portions of those proposals don’t take enterprise strategy into account properly.
  • Decision-makers might be affected by unrealistic optimism [Weinstein 1996], also known as optimism bias. It’s a cognitive bias that can cause them to discount the sometimes-vivid warnings of technologists about the unfavorable consequences of failing to provide technical debt management resources.

Investigations of the degree of correlation between burdens of technical debt and the incidence of rejected or severely curtailed proposals for resources to support technical debt management programs could determine the significance of the Dunning-Kruger effect relative to the problem of technical debt. Also rewarding would be a survey of the nearly 200 known cognitive biases, to determine which of them might be most likely to affect decision-making relative to technical debt, and how best to mitigate the risks they present.

References

[Fowler 2009] Fowler, Martin. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009. here . Retrieved January 10, 2016.

Cited in:

[Kohn 1999] Alfie Kohn. Punished by rewards: The trouble with gold stars, incentive plans, A's, praise, and other bribes. Boston: Houghton Mifflin Harcourt, 1999.

Order from Amazon

Cited in:

[Kruger 1999] J. Kruger and D. Dunning. “Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments,” Journal of Personality and Social Psychology 77(6), 1121-1134 (1999).

Cited in:

[Mattis 2008] James N. Mattis. “USJFCOM Commander’s Guidance for Effects-based Operations,” Joint Force Quarterly 51, Autumn 2008 105-108.

Available: here Retrieved November 9, 2017.

Cited in:

[McConnell 2008] McConnell, Steve. Managing Technical Debt, white paper, Construx Software, 2008.

Available at: www.construx.com/Page.aspx?cid=2801 Retrieved November 10, 2017.

Cited in:

[Thorndike 1920] E.L. Thorndike “A constant error in psychological ratings,” Journal of Applied Psychology, 4(1), 25-29 (1920). doi:10.1037/h0071663

Cited in:

[Weinstein 1996] Neil D. Weinstein and William M. Klein. “Unrealistic Optimism: Present and Future,” Journal of Social and Clinical Psychology 15(1), 1-8 (1996). doi:10.1521/jscp.1996.15.1.1

Cited in:

Related posts