Tag: balance

Posted on

Accounting for technical debt

Last updated on July 11th, 2021 at 04:57 pm

Accounting for technical debt isn’t the same as measuring it
Accounting for technical debt isn’t the same as measuring it. We usually regard our accounting system as a way of measuring and tracking enterprise financial attributes. We think of those financial attributes as representations of money. Technical debt is different. It isn’t real, and it isn’t a representation of money. It’s a representation of resources. Money is just one of those resources. Money is required to retire technical debt. We use money when we carry technical debt, and when we retire it. But we also use other kinds of resources when we do these things. Sometimes we forget this when we account for technical debt.

We need a high-caliber discussion of accounting for technical debt [Conroy 2012]. It’s a bit puzzling why there’s so little talk of it in the financial community. Perhaps one reason for this is the social gulf between the financial community and the technologist community. But another possibility is the set of pressures compelling technologists to leave technical debt in place and move on to other tasks.

Here’s an example. One common form of technical debt is the kind first described by Cunningham [Cunningham 1992]. Essentially, when we complete a project, we often find that we’ve advanced our understanding of what we actually needed to reach our goals. Because of our advanced understanding, we recognize that we should have taken a different approach. Fowler described this kind of technical debt as, “Now we know how we should have done it.” [Fowler 2009] At this point, typically, we disband the team and move on to other things, leaving the technical debt outstanding, and often, undocumented and soon to be forgotten.

Echo releases and management decision-making

A (potentially) lower-cost approach involves immediately retiring the debt and re-releasing the improved asset. I call this an “echo release.” An echo release is one in which the asset no longer carries the technical debt we just incurred and immediately retired. But echo releases usually offer no immediate, evident advantage to the people and assets that interact with the asset in question. That’s why decision makers have difficulty allocating resources to echo releases.

This problem arises, in part, from the effects of a what I regard as a shortcoming in management accounting systems. Most enterprise management accounting systems track effectively the immediate costs associated with technical debt retirement projects. They do a much less effective job of representing the effects of failing to execute echo releases, or failing to execute debt retirement projects in general. The probable cause of this deficiency is the distributed nature of the MICs—the metaphorical interest charges associated with carrying a particular technical debt. MICs appear in multiple forms: lower productivity, increased time-to-market, lost market share, elevated turnover of technologists, and more (see “MICs on technical debt can be difficult to measure”). Enterprise accounting systems don’t generally represent these phenomena very well.

The cost of not accounting for the cost of not retiring technical debt

Decision makers then adopt the same bias that afflicts the accounting system. In their deliberations regarding resource allocation, they emphasize only the cost of debt retirement. These discussions usually omit from consideration altogether any mention of the cost of not retiring the debt. That cost can be enormous, because it is a continuously recurring periodic charge with no end date. Those costs are the costs of not accounting for the cost of not retiring technical debt.

If we do make long-term or intermediate-term projections of MICs or costs related to echo releases, we do so to evaluate proposals for retiring the debt. Methods vary from proposal to proposal. Few organizations have standard methods for making these projections. And lacking a standard method, comparing the benefits of different debt retirement proposals is difficult. This ambiguity and variability further encourages decision makers to base decisions solely on current costs, omitting consideration of projected future benefits.

Dealing with accounting for technical debt

Relative to technical debt, the accounting practice perhaps most notable for its absence is accounting for outstanding technical debts as liabilities. We do recognize outstanding financial debt. But few balance sheets mention outstanding technical debt. Ignorance of the liabilities outstanding technical debt represents creates an impression that the enterprise has capacity that it doesn’t actually have. That’s why tracking our technical debts as liabilities would alleviate many of the problems associated with high levels of technical debt.

But other shortcomings in accounting practices can create additional problems almost as severe.

Addressing the technical-debt-related shortcomings of accounting systems requires adopting enterprise-standard patterns for debt retirement proposals. Such standards would make possible meaningful comparisons between different technical debt retirement options and between technical debt retirement options and development or maintenance options. One area merits focused and immediate attention: estimating MPrin and estimating MICs.

Standards for estimating MPrin are essential for estimating the cost of retiring technical debt. Likewise, standards for estimating MICs, at least in the short term, are essential for estimating the cost of not retiring technical debt. Because both MPrin and MICs can include contributions from almost any enterprise component, merely determining where to look for contributions to MPrin or MICs can be a complex task. So developing a checklist of potential contributions can help proposal writers develop a more complete and consistent picture of the MICs or MPrin associated with a technical debt. Below are three suggestions of broad areas worthy of close examination.

Revenue stream disruption

Technical debt can disrupt revenue streams either in the course of retirement projects, or when defects in production systems need attention. When those systems are out of production for repairs or testing, revenue capture might undergo short disruptions. Technical debt can extend those disruptions or increase their frequency.

For example, a technical debt consisting of the absence of an automated test can lengthen a disruption while the system undergoes manual tests. Technical debt consisting of misalignment between the testing and production environments can allow defects to slip through. Undetected defects can create new disruptions. Even a short disruption of a high-volume revenue stream can be expensive.

In advance of any debt retirement effort, we can identify some associations between classes of technical debt and certain revenue streams. This knowledge is helpful in estimating the contributions to MICs or MPrin from revenue stream disruption.

Extended time-to-market

Although technologists are keenly aware of productivity effects of technical debt, these effects can be small compared to the costs of extended time-to-market. In the presence of outstanding technical debt, time-to-market expands not only as a result of productivity reduction, but also from resource shortages and resource contention. Extended time-to-market can lead to delays in realizing revenue potential. And it can cause persistent and irreparable reductions in market share. To facilitate comparisons between different technical debt retirement proposals, estimates of these effects are invaluable.

Data flow disruption

All data flow disruptions aren’t created equal. Some data flow processes can detect their own disruptions and backfill as needed. For these flows, the main contribution to MICs or MPrin is delay. And the most expensive of these are delays in receiving or processing orders. Less significant but still important are delays in responding to anomalous conditions. Data flows that cannot detect disruptions are usually less critical. But they nevertheless have costs too. All of these consequences can be modeled and estimated. We can develop standard packages for doing so. And we can apply them repeatedly to MICs or MPrin estimates for different kinds of technical debt.

Last words

Estimates of MICs or MPrin are helpful in estimating the costs of retiring technical debt. They’re also helpful in estimating the costs of not retiring technical debt. In either case, they’re only estimates. They have error bars and confidence limits. The accounting systems we now use have no error bars. That, too, is a shortcoming that must be addressed.

References

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

Other posts in this thread

Posted on

Feature bias: unbalanced concern for capability vs. sustainability

Last updated on July 7th, 2021 at 09:56 pm

Alaska crude oil production 1990-2015
Alaska crude oil production 1990-2015. This chart [Yen 2015] displays Alaska crude oil produced and shipped through the Trans Alaska Pipeline System (TAPS) from 1990 to 2015. Production had dropped by 75% in that period, and the decline is projected to continue. In January 2018, in response to pressure from Alaskan government officials and the energy industry, the U.S. Congress passed legislation that opened the Arctic National Wildlife Refuge to oil exploration, despite the threat to ecological sustainability that exploration poses. If we regard TAPS as a feature of the U.S. energy production system, we can view its excess capacity as a source of feature bias. It creates pressure on decision makers to add features to the U.S. energy system. Alternatively, they could act to enhance the sustainability of Alaskan and global environmental systems [Wight 2017].

<

Enterprise decision makers affected by feature bias tend to harbor distorted views of the importance of new capability development compared to technical debt management. This tendency is likely due to the customer’s relative sensitivity to features, and relative lack of awareness of sustainability. Whatever the cause, customers tend to be more attracted to features than they are to indicators of sound technical debt management and other product sustainability practices. This tendency puts decision makers at risk of feature bias: unbalanced concern for capability vs. sustainability.

h4>Accounting changes can help

Changes in cost accounting could mitigate feature bias effects by projecting more accurately total MICs based on historical data and sound estimation. I explore possible accounting changes later in this post, and in future posts; meanwhile, let’s explore the causes and consequences of the distorted perspective I’m calling feature bias.

Causes and consequences of feature bias

For products or services offered outside the enterprise, the sales and marketing functions of the enterprise represent the voice of the customer [Gaskin 1991]. But customers are generally unaware of product or service attributes that determine maintainability, extensibility, or cybersecurity. These factors, the sustainability factors, affect the MICs for technical debt. But customers are acutely aware of capabilities—or missing or defective capabilities. Customer comments and requests are therefore unbalanced in favor of capability over sustainability. The sales and marketing functions tend to accurately transmit this unbalanced perspective to decision makers and technologists.

An analogous mechanism prevails with respect to infrastructure and its internal customers. Internal customers tend to be more concerned with capabilities than they are with sustainability of the processes and systems that deliver those capabilities. Thus, pressure from internal customers tends to emphasize capability at the expense of sustainability. The result of this imbalance is pressure to allocate excessive resources to capability enhancement, compared to activities that improve sustainability. And therefore controlling or reducing technical debt and its MICs gets less attention.

Nor is this the only consequence of feature bias. It provides unrelenting pressure for increasing numbers of features, despite the threats to architectural coherence and overall usability that such “featuritis” or “featurism” present. Featurism leads, ultimately, to feature bloat, and to difficulties for users, who can’t find what they need among the clutter of features that are often too numerous to document. For example, in Microsoft Word, many users are unaware that Shift+F5 moves the insertion point and cursor to the point in the active document that was last edited, even if the document has just been freshly loaded into Word. Useful, but obscure.

Feature bias bias

Feature bias, it must be noted, is subject to biases itself. The existing array of features appeals to a certain subset of all potential customers. Because it is that subset that’s most likely to request repair of existing features. And they’re also the most likely to suggest additional features. The pressure for features tends to be biased in favor of the needs of the most vociferous users. That is, there is pressure to evolve to better meet the needs of existing users. That pressure can force to lower priority any efforts toward meeting the needs of other stakeholders or potential stakeholders. These other stakeholders might be even more important to the enterprise than are the existing users. This bias in feature bias presents another risk that can affect decision makers.

Organizations can take steps to mitigate the risks of feature bias. An example of such a measure might be using focus groups to study how educating customers in sustainability issues affects their perspectives relative to feature bias. Educating decision makers about feature bias can also reduce this risk.

At the enterprise scale, awareness of feature bias would be helpful. But awareness alone is unlikely to counter its detrimental effects. These effects include underfunding technical debt management efforts. Eliminating the source of feature bias is extraordinarily difficult, because customers and potential customers aren’t subject to enterprise policy. Feature bias and feature bias bias are therefore givens. To mitigate the effects of feature bias, we must adopt policies that compel decision makers to consider the need to deal with technical debt.

A possible corrective action

One possible corrective action might be improving accounting practices for MICs, based on historical data. For example, there’s a high probability that any project might produce new technical debt. It might be prudent to fund the retirement of that debt in the form of reserves when we fund projects. And if we know that a project has encountered some newly recognized form of technical debt, it might be prudent to reserve resources to retire that debt as soon as possible. Ideas such as these can rationalize resource allocations with respect to technical debt.

These two examples illustrate what’s necessary if we want to mitigate the effects of feature bias. They also illustrate just how difficult such a task will be.

References

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Other posts in this thread

Posted on

Technical debt and engineering resources

Last updated on July 7th, 2021 at 10:34 am

Flooding from Hurricane Katrina in New Orleans, 2005.
Flooding from Hurricane Katrina in New Orleans, 2005. The forces of Nature can overtop or undermine any levee humans can build. So it is with technology. Organizational policy and politics can overcome or undermine any technology humans can devise to attain mastery over technical debt. To master technical debt, technology isn’t enough—we must also deal with policy and politics.

Improving organizational effectiveness in technical debt management—or avoiding incurring new technical debt—should create significant savings and competitive advantages. These benefits arise from reductions in metaphorical interest charges (MICs) that result from retiring technical debt. But these benefits become available only if engineering capacity increases relative to the total debt-related workload. After the technical debt management program is in place, if the balance between engineering resources and debt-related workload becomes more favorable, then organizational effectiveness can improve. But if the balance becomes less favorable, as a result of reductions in engineering resources, organizational effectiveness won’t improve, even at lower levels of technical debt.

Unfortunately, some organizations adopt advanced technical debt management practices while reducing engineering capacity. If reductions are dramatic enough, engineering effectiveness is no better than it was before initiating the technical debt management program. The reason for this is that the engineering process isn’t the sole cause of technical debt. Improving the engineering process to eliminate technical causes of technical debt leaves nontechnical causes in place. That’s why technological solutions to the technical debt management problem might not produce benefits in organizational effectiveness and agility.

The focus of technical debt research has been technology

The focus of research in technical debt management has been on technology—recognition of technical debt, its measurement, representation, retirement, and so on. Progress on improving the engineering process has been significant, especially in software engineering, where a clear “research roadmap” has appeared [Izurieta 2017]. Effective tools for automating or partially automating technical debt detection and retirement will be widely available and very generally effective in the not-too-distant future, at least for software. But progress has transcended debt detection and retirement. Avoiding technical debt formation to the extent possible is much preferable, and in some contexts, it’s practical even today, as Trumler and Paulisch suggest [Trumler 2016].

Such developments might or might not have much impact on the limiting the effects of carrying technical debt. Given the necessary resources, engineering organizations could retire much of the technical debt now extant. That is, the will and the capacity to invest in debt retirement determines debt retirement rates. Currently, the levels of will and capacity for such activity are insufficient. But if new methods for managing technical debt become available, one might wonder whether organizations will apply resources sufficient to ensure that they actually experience a reduction in the limiting effects of technical debt.

Technological development isn’t enough

The open question is this: will technological developments alone give us control of the problem of technical debt? Perhaps not. Advancements in technical debt management do benefit organizations. But they could use that benefit to execute reductions in engineering staffing. If they do, they could divert savings to other parts of the enterprise. That would allow technical debt to remain at reduced levels that could still compromise the effectiveness of that reduced engineering staff.

For example, research has shown that schedule pressure contributes to technical debt formation and persistence. Suppose that the engineering groups of an organization become more adept at managing and preventing technical debt. Suppose further that the organization’s marketing and sales groups don’t improve their own intelligence and planning processes. Then Marketing might demand new capabilities with ever shorter timelines. That could lead to increased schedule pressure for the engineering groups. Then the enterprise might not benefit from the new technical debt management capabilities, even though the burden of technical debt has been reduced.

Until we have evidence of significant change in the behavior of non-technologists—or even acknowledgment that their behavior contributes to debt formation—we can expect the effects of nontechnical causes of technical debt to persist, and possibly even to grow.

This blog focuses on the nontechnical etiology of technical debt formation and persistence, and approaches for managing it. Watch this space.

References

[Conroy 2012] Patrick Conroy. “Technical Debt: Where Are the Shareholders' Interests?,” IEEE Software, 29, 2012, p. 88.

Available: here; Retrieved: August 15, 2018.

Cited in:

[Cunningham 1992] Ward Cunningham. “The WyCash Portfolio Management System.” Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

Cited in:

[Fowler 2009] Martin Fowler. “Technical Debt Quadrant.” Martin Fowler (blog), October 14, 2009.

Available here; Retrieved January 10, 2016.

Cited in:

[Gaskin 1991] Steven P. Gaskin, Abbie Griffin, John R. Hauser, Gerald M. Katz, and Robert L. Klein. “Voice of the Customer,” Marketing Science 12:1, 1-27, 1991.

Cited in:

[Izurieta 2017] Clemente Izurieta, Ipek Ozkaya, Carolyn Seaman, and Will Snipes. “Technical Debt: A Research Roadmap: Report on the Eighth Workshop on Managing Technical Debt (MTD 2016),” ACM SIGSOFT Software Engineering Notes, 42:1, 28-31, 2017. doi:10.1145/3041765.3041774

Cited in:

[Trumler 2016] Wolfang Trumler and Frances Paulisch. “How ‘Specification by Example’ and Test-driven Development Help to Avoid Technical Debt,” IEEE 8th International Workshop on Managing Technical Debt. IEEE Computer Society, 1-8, 2016. doi:10.1109/MTD.2016.10

Cited in:

[Wight 2017] Philip Wight. “How the Alaska Pipeline Is Fueling the Push to Drill in the Arctic Refuge,” YaleE360, Yale School of Forestry & Environmental Studies, November 16, 2017.

Available: here; Retrieved: February 8, 2018

Cited in:

[Yen 2015] Terry Yen, Laura Singer. “Oil exploration in the U.S. Arctic continues despite current price environment,” Today in Energy blog, U.S. Energy Information Administration, June 12, 2015.

Available: here; Retrieved: February 8, 2018.

Cited in:

Related posts

  • The metaphorical principal of a technical debt
    The principal amount of a financial debt and the metaphorical principal of a technical debt have very different properties. They are so different that it’s wise to avoid using the term “principal” to refer to the metaphorical principal of a technical debt. We use the term MPrin.
Show Buttons
Hide Buttons