Posted on

Technical debt use disorder

The American Psychiatric Association has identified a disorder they call Substance Use Disorder (SUD), which includes alcoholism, drug addiction, and other patterns related to substance use [APA 2013]. The research they’ve done can serve well as a model for understanding organizational behavior related to technical debt. In this post I’ll show how to use that model to describe a disorder of organizations that we could call Technical Debt Use Disorder (TDUD). It’s a pattern in which the organization can’t seem to retire much of its technical debt, or can’t stop incurring increasing burdens of technical debt, even though everyone in the organization realizes—or almost everyone realizes—that technical debt is harming the organization.

A ball and chain, with shackle
A ball and chain, with shackle.  Attaching this device to the legs of prisoners or slaves limits their ability to run, just as technical debt limits the ability of organizations to exploit new opportunities, or even to maintain their current market positions.

A brief description of the American Psychiatric Association’s publication, DSM-5®, might help explain the connection between SUD and TDUD. DSM-5 is the fifth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM). It’s used by health care professionals in the United States and much of the rest of the world as a guide to diagnosing mental disorders, and as a framework for further research. First published in 1952, the current revision, DSM-5, was released in 2013.

So what does DSM-5 have to do with technical debt?

What distinguishes responsible use of technical debt from irresponsible use is a topic that has generated many papers, conference presentations, and hallway debates over the years. Although there is consensus about the distinction in many cases, the debate continues. Sometimes, though, research in one field suggests paths forward in seemingly unrelated fields. So much thought and study has been invested in DSM-5 that it’s worth a look to see if the technical debt community can harvest something useful from the research in psychiatry.

I looked at DSM because I noticed that organizations that carry significant volumes of technical debt seem to have difficulty retiring it. In some cases, they also have difficulty halting accumulation of technical debt, or even slowing the rate of accumulation. This struck me as similar to the substance use problems some people encounter. I began to wonder whether there might be parallels between the substance use disorders that afflict people—alcoholism, drug addiction, and so on—and the technical debt problems that afflict organizations.

In the table below is one set of parallels I’ve found. The left column of the table is the list of diagnostic criteria for Substance Use Disorder provided in the DSM. The right column is my rewording of those criteria in an attempt to make them apply to how organizations deal with technical debt. I had thought initially that the rewording exercise might be difficult—that it might be a stretch. And here and there, it was a bit of a stretch. But overall, the SUD framework is a very good fit.

Diagnosing technical debt use disorder

Have a look at the table, and then check out the comments below it about how health care professions use the criteria.

DSM-5 Criteria for Substance Use Disorder (SUD)Criteria for Enterprise Technical Debt Use Disorder (TDUD)
1. Taking the substance in larger amounts or for longer than you’re meant to.
1. Incurring technical debt in larger amounts than you intended and carrying it for longer than you intended.
2. Wanting to cut down or stop using the substance but not managing to.2. Wanting to retire your technical debt or reduce the rate of incurring it but not managing to.
3. Spending a lot of time getting, using, or recovering from use of the substance.3. Spending a lot of time dealing with the consequences of the technical debt you’ve already incurred.
4. Cravings and urges to use the substance.4. Insistent demands on precious resources, causing the enterprise to incur “just a little more” technical debt.
5. Not managing to do what you should at work, home, or school because of substance use.5. Not managing to attend to the needs of existing products, services, or technological infra­structure because of the demands resulting from metaphorical interest charges on technical debt.
6. Continuing to use, even when it causes problems in relationships.6. Continuing to carry technical debt, or continuing to incur yet more technical debt, even though it causes toxic conflict among employees, and problems in customer relationships and strategic partnerships.
7. Giving up important social, occupational, or recreational activities because of substance use.7. Giving up developing important new products or services, or upgrading critical infrastructure, or pursuing new initiatives because of resource deficits traceable to technical debt service.
8. Using substances again and again, even when it puts you in danger.8. Incurring technical debt again and again, even when it puts the enterprise in fiscal danger or danger of losing market position.
9. Continuing to use, even when you know you have a physical or psychological problem that could have been caused or made worse by the substance.9. Continuing to incur technical debt, even when you know you have a fiscal or market leadership problem that could have been caused or made worse by technical debt.
10. Needing more of the substance to get the effect you want (tolerance).10. Needing to incur more technical debt to get the fiscal effect you need—a product delivered or a contract completed.
11. Development of withdrawal symptoms, which can be relieved by taking more of the substance.11. Upon attempting to retire existing technical debt, or halting incurring yet more technical debt, fiscal or market position problems develop in short order, which can be relieved only by incurring yet more debt.

In health care, two or three symptoms indicate a mild substance use disorder; four or five symptoms indicate a moderate substance use disorder, and six or more symptoms indicate a severe substance use disorder. Have a look at the right-hand column. How would you score your organization? Can we categorize the severity of an organization’s problem with technical debt using a scale similar to the one health care professionals use for SUD?

Conclusion

Technical debt isn’t inherently evil. Its existence among technological assets isn’t proof of engineering malpractice. For example, we can decide—responsibly—to deliver a system that carries technical debt, provided that we’ve carefully weighed the consequences of incurring that debt against the consequences of delayed delivery, and provided that we have a workable plan for retiring that debt, or for carrying the burden of its MICs.

But organizations can nevertheless become trapped in a cycle of technical debt, unable to make much progress in reducing it. In some cases, business as usual won’t work. Drastic action is required.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

Other posts in this thread

Posted on

Technical debt smell

Technical debt smell would be a useful indicator of the presence of a severe problem of technical debt in an organization. Unfortunately, technical debt doesn’t usually have a smell, as such. But speaking metaphorically, maybe it does have indirect indicators, in the same way that we might say, “I smell a rat” when probing a mystery. Actually, the idea of indirect indicators has a long and storied tradition.  In one scene midway through James Fenimore Cooper’s The Last of the Mohicans, Uncas (who is the actual last of the Mohicans) demonstrates his tracking skills [Cooper 1857]:

The young Mohican bent over the track, and removing the scattered leaves from around the place, he examined it with much of that sort of scrutiny that a money-dealer, in these days of pecuniary doubts, would bestow on a suspected due-bill. At length he arose from his knees, satisfied with the result of the examination.

“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872)
“The Young Chief Uncas,” 1869 chromolithograph by John Mix Stanley (1814–1872). Photo courtesy Wikimedia Commons.
Uncas then deduces amazing details about the man who left the track he examined—150 years ahead of Sherlock Holmes. Natty Bumppo, Cooper’s Hawkeye character, calls the signs Uncas uses tell-tales.

Nature abounds with examples of such skill at noticing tell-tales. Lions, tigers, bears, and all sorts of fauna use their olfactory senses to detect food, predators, mates, offspring, weather, and even the change of seasons. Smell gives them access to information they need, often before sight or hearing can.

That’s probably a part of why smell has become a useful metaphor in software engineering. The technical literature about code smells is vast and growing [Haque 2018]. In a blog post titled “CodeSmell,” Martin Fowler defines code smell as, “…a surface indication that usually corresponds to a deeper problem in the system.” [Fowler 2006] Code smells are traits that are easy to recognize, and often—but not always—indicators of problems.

The concept is also useful in the business domain, though there we use a different metaphor and a different term. In the business context, we call smells red flags. Investopedia defines a red flag as, “…an indicator of potential problems with a security, such as any undesirable characteristic that stands out to an analyst as it pertains to a company’s stock, financial statements or negative news reports.” But I’ve heard the term red flag used in the context of evaluating proposals, assessments, status reports, personnel, and intelligence of all kinds.

Whether called tell-tales, smells, red flags, or just indicators, their value is that they suggest the outlines of something we haven’t yet seen clearly enough to identify with certainty. Their principal attributes are that they’re available at the surface of the domain we’re surveying, they’re relatively cheap to obtain, and, if found, they suggest trouble, and deeper investigation might be worthwhile.

In the software engineering community, technical debt is regarded as a smell that indicates trouble in the system’s software. So we might ask, “Among policymakers, what are the smells that indicate trouble in the organization?” If technical debt is the trouble we’re looking for, what are the cultural smells that indicate that technical debt might be a problem?

Said differently, can we find, or can we develop, a set of attributes of enterprise culture that indicate the degree of severity of an organization’s problems with technical debt?

Here are some possible “technical debt smells”—aspects of enterprise culture that could indicate problems with technical debt:
  • There is a general belief that technical debt afflicts some organizations, but not ours
  • We’re a new startup—just a year old—so we have no technical debt.
  • We don’t build software, therefore no technical debt
  • Non-technical members of our executive team have little if any knowledge of the concept of technical debt
  • No enterprise resources are allocated to educating non-technical employees about technical debt
  • The VP of Marketing doesn’t believe that anything she does could possibly contribute to technical debt
  • There is a general belief that if we have technical debt, it’s due solely to malpractice on the part of engineers
  • We’ve tried to assess the total cost of eliminating all of our technical debt, and found the estimates so unreliable that we decided to leave well enough alone
  • We do believe that technical debt does have costs, but because it only affects the productivity of engineers, we just hired more engineers and decided to live with it

Clearly we could assemble a list of technical debt smells—beliefs about technical debt and behaviors that affect it—and check for their presence in a given organization. But fortunately, some of that work has already been done, albeit in a very different context; That context is a malady psychiatrists call “Substance Use Disorder.” More about that next time.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Posted on

Using SMART goals for technical debt reduction

Last updated on May 9th, 2019 at 02:17 pm

Attempting to reduce the enterprise burden of technical debt by setting so-called “SMART goals” in the obvious way can often produce disappointing results. SMART, due to George T. Doran [Doran 1981], is a widely used approach for expressing management goals. “SMART” is an acronym for “Specific, Measurable, Attainable, Realistic, and Time-boxed,” though the last three words (the “ART”) are chosen in various alternative ways. Doran himself used “assignable, realistic, and time-related.”

Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives
Prof. George T. Doran (1939-2011), creator of the S.M.A.R.T acronym for setting management objectives. Watch a 2010 interview of Prof. Doran at YouTube.
SMART is so embedded in management culture that many assume without investigation that expressing technical debt reduction goals directly using the SMART pattern will produce the desired results. Also embedded in management culture is the aphorism, “You get what you measure.” [Ariely 2010]  [Bouwers 2010] Combining these two ideas in a straightforward way, one might express the technical debt reduction goal as, “Reduce the burden of technical debt by 20% per year for each of the next five years.”

There is ample support for a claim that this “direct” approach to applying the SMART technique will be ineffective. The fundamental issue is that so much of employee behavior affects technical debt indirectly that it overwhelms the effects of employee behaviors that affect technical debt directly. The result is that although the direct approach does cause some employees to adopt desirable behaviors, their impact is not significant enough compared to the effects of the behaviors of employees who see little connection between their own activities and the burden of technical debt, or who are subject to competing constraints on their behaviors that then cause them to act in ways that increase technical debt.

That’s why it’s necessary for management to develop a series of SMART goals that affect behaviors that have indirect effects on technical debt. In the first part of this post, “Setting a direct SMART goal for technical debt reduction is problematic,” I explore the problems inherent in the direct approach. In the second part, “How to set SMART goals for technical debt,” I provide examples of SMART goals that touch on behaviors that have indirect effects on technical debt.

Setting a direct SMART goal for technical debt reduction is problematic

Let’s begin by exploring some of the problems with the direct approach. In this section, I assume that management has set a SMART goal for the enterprise in the form, “Reduce the burden of technical debt by 20% per year for each of the next five years.” But there’s nothing special about the numbers. My comments below apply to the form of the goal, rather than the specific numbers.

The direct approach assumes measurability

To attain a goal of a 20% reduction in technical debt in a given year, we must be able to measure the level of technical debt at the beginning of the year and the level at the end of the year, presumably with confidence in the 90% range or better. Such a measurement with the precision required might not be possible. Moreover, in most cases the probability that such a measurement is possible is low. For these reasons, setting periodic goals for total technical debt is not a useful management tool.

Consider a simple example. One form of technical debt—and it’s a common form—is missing or incompletely implemented capability. In some instances, the metaphorical principal (MPrin) of a given instance of this debt in the current year can change spontaneously to a dramatically larger value in the following year (or even the following week), due to changes in the underlying asset unrelated to the technical debt, or due to debt contagion, or due to any number of other reasons. When this happens, the technical debt retirement effort for that year can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well.

The direct approach assumes a static principal

With most financial debts, the principal amount is specified at the time of loan origination. Moreover, we can compute the principal at any time given the mathematical formulas specified in the loan agreement.

By contrast, in many cases, the metaphorical principal amount of a technical debt might be neither fixed nor readily computable. We can estimate the MPrin of a given kind of technical debt at a given time, and we can even make forward projections of those estimates. But they are merely estimates, and their error bars can be enormous. See “Policy implications of the properties of MPrin” and “Useful projections of MPrin might not be attainable.”

The direct approach focuses on MPrin, not MICs

Objectives expressed in terms of the volume of technical debt—the total MPrin—are at risk of missing the point. Total MPrin is not what matters most. What matters is MICs—the total cost of carrying the debt. Even more important is the timing of arrival of the MICs.

And like MPrin, MICs can vary in wild and unpredictable ways. For example, the MICs for a piece of technical debt borne by an asset that isn’t undergoing maintenance or enhancement can be zero; in a later time period, when that asset is undergoing enhancement, the MICs can be very high indeed. See  “MICs on technical debt can be unpredictable” for a detailed discussion.

Priority setting for technical debt retirement is most effective when it takes into account the timing of MICs. For example, if we know that we must enhance a particular asset by FY21 Q3, and if we know that it bears technical debt that adds to the cost of the enhancement, retiring that debt in FY20 would be advisable. On the other hand, if that form of technical debt has no effect on MICs for the foreseeable future, retiring that technical debt might not be worth the effort.

The direct approach fails to distinguish legacy technical debt from incremental technical debt

Unless policies are already in place governing the formation of new technical debt—what I call incremental technical debt—technical debt retirement programs might encounter severe difficulty meeting their goals. The technical debt retirement program might simply be unable to keep up with the formation of new technical debt resulting from new development or from ongoing maintenance and enhancement of existing assets.

The direct approach fails to anticipate the formation of enterprise-exogenous technical debt

Technical debt can sometimes form as a result of innovations, changes in standards, or changes in regulations that occur entirely external to the enterprise. I call such technical debt enterprise-exogenous. When this happens, the technical debt retirement effort can appear to have suffered a serious setback, even though the technical debt retirement teams might have been performing perfectly well. Before initiating a technical debt reduction program, it’s wise to first deploy a program that’s capable of retiring technical debt at a pace that at least equals the pace of formation of enterprise-exogenous technical debt.

Incurring technical debt is sometimes the responsible thing to do

At times, incurring technical debt is prudent. In some situations, accepting the debt you’ve incurred—even for the long term—might be called for. Because strict goals about total technical debt can lead to reluctance to incur debt that has a legitimate  business purpose, whatever goals are set for total technical debt must be nuanced enough to deal with these situations. Goals for total technical debt that adhere strictly to the SMART goal pattern sometimes lack the necessary level of nuance.

How to set SMART goals for technical debt

SMART goals can work for technical debt management, but we must express them in ways that are more closely related to behavioral choices. Here are some examples of SMART goals that can be effective elements of the technical debt management program. Some of these examples are admittedly incomplete. For example,  I offer no proof of assignability, attainability, or realism, because they can vary from organization to organization, or because the goal in question must be distributed across multiple organizational elements in ways peculiar to the organization.

At least 30% of incremental technical debt will be secured technical debt at the end of Year 1; 60% by the end of Year 2

Incremental technical debt is technical debt that’s incurred in the course of work currently underway or just recently completed. Because it’s so well understood, its MPrin can be estimated with higher precision than is usually possible with legacy technical debt. That precision is needed for defining the collateral and resources used to secure the debt.

A secured technical debt, like a secured financial debt, is one for which the enterprise reserves the resources needed to retire the debt. However, unlike a financial debt, the resources required to retire a technical debt might not be purely financial. Beyond financial resources, they might include particular staff, equipment, test beds, and downtime. The commitment might extend beyond the current fiscal period. Secured technical debt is a powerful means of driving down total technical debt burden, but it might require modification of internal budget management processes and fiscal reporting. Policymakers can help in designing and deploying the necessary changes.

Within one year, at least 50% of all incremental technical debt will be retired within one year of its origination; 70% within 18 months

This goal also exploits the fact that incremental technical debt can be estimated with relatively high precision. As a goal, it builds on the goal above by requiring that the resources pledged to retire incremental debts actually be expended as intended.

Within one year, all engineers and their direct supervisors will be educated in basic technical debt concepts

The educational materials will be developed in the next five months and piloted with 10% of the technical staff within seven months. The material will include an online proficiency test that 90% of engineers will have successfully passed within a year.

Within one year, 90% of project plans will include projections of the MPrin of the incremental technical debt they expect to generate for each delivery cycle

This information is useful for making forward projections of resources needed to secure incremental technical debt. Tracking the accuracy of these projections helps project planners improve their estimates.

Within one year, initiate a practice of identifying the top five forms of legacy technical debt, ranked by the volume of the contagion

Debt contagion is the propagation of a given form of technical debt by creating new system elements or assets in forms compatible with elements already identified as technical debt. By examining the body of incremental technical debt created enterprise-wide in a given time period (say, by fiscal quarter), we can determine the portion of that incremental debt that results from contagion, for each type of contagious legacy technical debt. This data is needed to identify the most contagious forms of legacy technical debt. They are prime candidates for debt retirement.

Within one year, initiate an industrial intelligence practice that is responsible for projecting the formation of enterprise-exogenous technical debt

This group must have a sophisticated grasp of the technologies in use within the enterprise that already bear enterprise-exogenous technical debt, or which could be subject to the formation of enterprise-exogenous technical debt. Its responsibilities cover enterprise products and services, as well as enterprise infrastructure. It issues advisories as needed, and an annual forecast. The group is also responsible for recommending and monitoring participation in industrial standards organizations. The group reports to the CIO or CTO.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Posted on

Exogenous technical debt

Last updated on July 24th, 2018 at 07:30 pm

Mastering understanding of exogenous technical debt—debt that arises from causes not directly related to the asset that bears the debt—is essential to controlling technical debt formation. Exogenous technical debt is particularly troublesome to those who work on the affected assets. They can’t control its formation, and they’re rarely responsible for creating it. But their internal customers and those who control resources often fail to understand this. Indeed, those who work on the affected assets are often blamed for the formation of exogenous technical debt even though they had no role in its formation, and could have done nothing to prevent its formation.

Asbestos with muscovite.
Asbestos with muscovite. Asbestos is a family of six minerals that occur naturally in fibrous form. The fibers are all known carcinogens. Until 1990, it was widely used in many common building materials, including insulation, plaster, and drywall joint compound. It is now banned, but it’s present in many existing structures, including homes and offices. The installation of the ban caused these structures to incur exogenous technical debt. Photo by Aramgutang courtesy Wikipedia.

Technical debt is exogenous when it’s brought about by an activity not directly related to the assets in which the debt appears. The word exogenous comes from the Greek exo– (outside) + –genous (related to producing). So exogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that don’t involve the asset directly.

Because so much technical debt is produced indirectly, controlling its direct formation—for example, by engineering teams—isn’t sufficient for achieving enterprise control of technical debt formation. To control technical debt formation, we must track which activities produce it, including both direct and indirect effects. Allocating technical debt retirement costs to the activities that brought that debt about, even if the allocation doesn’t affect budget authority for those activities, is therefore a useful practice. Knowledge about which past activities created technical debt, and how much, is helpful for long-term reduction in the rate of technical debt formation.

When we think of technical debt, we tend to think of activities that produce it relatively directly. We often imagine it as resulting solely from engineering activity, or from decisions not to undertake engineering activity. In either case the activity involved, whether undertaken or not, is activity directly involving the asset that carries—or will be carrying—the technical debt. This kind of technical debt is endogenous technical debt. The word endogenous comes from the Greek endo– (within or inside) + –genous (related to producing).  So endogenous technical debt is that portion of an asset’s debt that comes about from activities or decisions that directly involve the asset.

More about endogenous technical debt in future posts. For now, let’s look more closely at exogenous technical debt, and its policy implications.

Examples of exogenous technical debt

In “Spontaneous generation,” I examined one scenario in which technical debt formation occurs spontaneously—that is, in the absence of engineering activity. Specifically, I noted how the emergence of the HTML5 standard led to the formation of technical debt in some (if not all) existing Web sites, in the sense that they didn’t exploit capabilities that had become available in HTML5. Moreover, some sites whose developers had elected to emulate capabilities of the new standard by exploiting alternative technologies needed rehabilitation to remove the emulation and replace it with use of facilities in the HTML5 standard. All of these artifacts—including those that existed, and those that didn’t—comprised technical debt. This scenario thus led to the formation of exogenous technical debt.

In a second example, AMUFC, A Made-Up Fictitious Corporation, incurs technical debt when the vendor that supplies the operating system (OS) for AMUFC’s desktop computers announces the date of the end of extended support for the version of the OS in use at AMUFC. Because the end of extended support brings an end to security updates, AMUFC must retire that debt by migrating to the next version of that vendor’s OS before extended support actually ends.

In both of these examples, the forces that lead to formation of exogenous technical debt are external to both the enterprise and the enterprise’s assets. But what makes technical debt exogenous is that the forces that led to its formation are unrelated to any of the engineering work being performed on the asset that carries the debt. This restriction is loose enough to also include technical debt that arises from any change or activity external to the asset, but within the enterprise.

Exogenous technical debt arising from actions within the enterprise

Exogenous technical debt can arise from activities or decisions that take place entirely within the enterprise.

For example, consider a line of mobile devices developed and marketed by AMUFC (A Made-Up Fictitious Corporation). Until this past year, AMUFC has been developing ever more capable devices, thereby extending its line of offerings at the high end—the more expensive and capable members of the line. But this past quarter, AMUFC developed a low-end member of the line, and as often happens, price constraints led to innovations that could produce considerable savings in manufacturing costs if those innovations were applied to all members of the line. In effect, then, the designs of the previously developed models in this line of devices have incurred exogenous technical debt. The debt is exogenous because the activity that led to debt formation was not performed on the assets that now carry the debt, even though the activity that led to debt formation occurred within the enterprise. This kind of exogenous technical debt might be termed asset-exogenous. Exogenous technical debt of the kind that’s incurred by activity beyond the enterprise might be termed enterprise-exogenous.

Exogeneity versus endogeneity

For asset-exogenous technical debt, ambiguity between endogeneity and exogeneity can arise. The example above regarding the line of mobile devices produced by AMUFC provides an illustration.

For convenience, call the team that developed one of the high-end devices Team High. Call the team that developed the low-end device Team Low. From the perspective of Team High, the technical debt due to the innovations discovered by Team Low is exogenous. But from the perspective of the VP Mobile Devices, that same technical debt might be regarded as endogenous. The debt can be endogenous at VP level because it’s possible to regard the entire product line as a single asset, and that might actually be the preferred perspective of VP Mobile Devices.

Exogeneity and legacy technical debt

The technical debt portfolio of a given asset can contain a mix a technical debt that arose from various past incidents. In assessing the condition of the asset, it’s useful to distinguish this existing debt from debt that’s incurred as a consequence of any current activity or decisions. Call this pre-existing technical debt legacy technical debt.

The legacy technical debt carried by an asset is technical debt associated with that asset, and which exists in that asset in any form prior to undertaking work on that asset. For example, in planning a project to renovate the hallways and common areas of a high-rise apartment building, workers discover that beneath the existing carpeting is a layer of floor tile containing asbestos. Management has decided to remove the tile. In this context, the floor tile can be viewed as legacy technical debt. It isn’t directly related to the objectives of the current renovation, but removing it will enhance the safety of future renovations, enable certification of the building as asbestos-free, increase the property value, and reduce the cost of eventual demolition. In this situation asbestos removal amounts to retirement of legacy technical debt, and accounting for it as part of the common-area renovation would be misleading.

When contemplating efforts to retire legacy technical debt, exogeneity becomes a factor in allocating the necessary resources. If the debt in question is enterprise-exogenous, then we can justifiably budget the effort from enterprise-level accounts if appropriate. For other cases, other pools of resources become relevant depending on what actions created the debt. For example, if the exogenous technical debt arose because of a departmental change in standards, debt retirement costs can justifiably be allocated to the standards effort. If the exogenous technical debt arose from innovations in other members of the asset’s product line, those debt retirement costs can justifiably be allocated to the product line.

Policy insights

Understanding the properties of exogenous technical debt can be a foundation for policy innovations that enhance enterprise agility.

Culture transformation

Widespread understanding the distinction between exogenous and endogenous technical debt is helpful in controlling blaming behavior that targets the engineering teams responsible for developing and maintaining technological assets.

Understanding of asset-exogenous technical debt helps non-engineers understand how their actions and decisions can lead to technical debt formation, even when there is no apparent direct connection between those actions or decisions and the assets in question.

Resource allocation

Data about the technical debt creation effects of enterprise activities is helpful in allocating technical debt retirement costs. For example, when we know all the implications of reorganization, including its impact on internal data about the enterprise itself, we can charge data-related activity to the reorganization instead of to general accounts of the Information Technology function. This helps the enterprise understand the true costs of reorganization.

Similarly, data about enterprise-exogenous technical debt helps planners understand how to deploy resources to gather external intelligence about trends that can affect internal assets. Such data is also useful for setting levels of support and participation in industrial standards organizations or in lobbying government officials.

Knowing the formation history of exogenous technical debt provides useful guidance for those charged with allocating the costs of retiring technical debt or preventing its formation.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

Other posts in this thread

Posted on

Adopt an enterprise-wide definition of technical debt

An enterprise-wide definition of technical debt is essential because effective technical debt management requires cooperation from almost everyone. Absent a shared definition of technical debt, controversy can develop, especially among those who have previously encountered the concept—namely, among technologists. Policymakers can make invaluable contributions to the design of the cultural transformation that will enable control of technical debt.

A physical expression of shared commitment, essential to adopting an enterprise-wide definition of technical debt
A physical expression of shared commitment. Effective management of technical debt requires both a shared understanding of what it is and a shared commitment to do what’s required to get control of it.

Li et al. [Li 2015] found that defining what is and what is not technical debt remains a question at issue in the software engineering literature. Even if we restrict the discussion to software constructed in-house, opinions about what constitutes technical debt do differ. The authors found that in the literature of technical debt, “The term ‘debt’ has been used in different ways by different people, which leads to ambiguous interpretation of the term.”

This finding is perhaps the most significant for policymakers, because it suggests that implementing a technical debt management regime will require forging an organizational consensus about the meaning of the term technical debt. The people of most organizations come from a broad array of different backgrounds. Some have little knowledge of technical debt, and therefore have no preconceptions. But those who are aware of the issue, who are mainly technologists and their managers, probably interpret the term technical debtin a variety of ways. Because some of those who do have awareness of the term are likely to have strong opinions about its meaning, one can anticipate a need to resolve these differences of opinion early in the effort to gain control of technical debt.

Some technical terms, like RAID,byte, compiler, and kilowatt,have standard definitions that are widely accepted. Although the term technical debthas found wide use, there is no standard definition for it. What some people categorize as technical debt, others do not. Those who are accustomed to working with terms that have precise, widely accepted definitions might tend to assume that the term technical debtdoes have (or should have) one as well. This assumption can create some difficulty for people who do not realize that others might not share their views as to the definition of the term.

Policymakers must be aware that there is a lack of consensus about the definition of technical debt. Our definition, crafted specially for the use of policymakers, might seem unusually broad to technologists and engineers. For that reason alone, it’s advisable to become familiar with the various ways the term is used by technologists, because understanding their perspective is essential to formulating sound policy deserving of their respect.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

Other posts in this thread

Posted on

Tension between policymakers and technologists

Last updated on July 24th, 2018 at 08:17 pm

Tension between policymakers and technologists, which can manifest as misalignment of their respective priorities, is a significant contributor to uncontrolled growth of technical debt. In this thread I explore sources of this tension and introduce concepts that can assist policymakers and technologists in their efforts to control the growth of technical debt.

Effective, sustainable control of technical debt is the objective of technical debt management policy. In an enterprise that has achieved this objective, technical debt serves as a strategic tool that assists in attaining and maintaining market leadership. In such an organization, technical debt does exist, and some legacy technical debt might remain in place, but technical debt growth is managed strategically, if growth occurs at all. Any technical debt that carries significant MICs, and which compromises productivity and enterprise agility, is addressed and retired with appropriate priority. In short, technical debt is addressed not solely as a technological issue, but as a component of business strategy.

Raindrops on a grapevine
Raindrops on a grapevine. We often think of tension as a negative, destructive force. But tension—in the case of raindrops, surface tension—is what holds a raindrop together. Tension gives the raindrop structure and integrity. The tension between technologists and policymakers can also be constructive. It can ensure that the enterprise manages its technical debt in ways that balance the political imperatives of both technology and strategic health.

This stance is at odds with the historical position most enterprises have adopted vis-à-vis technical debt. Historically, technical debt has been seen as a technical problem, if it has been recognized at all. Most enterprises have left the management of technical debt to the technologists. Frequently, then, the policymaker who enters the discussion about technical debt might be seen by technologists as an interloper, arriving late to the discussion, or as a less-than-knowledgeable invader attempting to seize control of a piece of the technologists’ domain. Tensions can arise between policymakers and technologists. Such tensions complicate the problem of managing technical debt.

One possible source of this tension is revealed in a study of the literature of technical debt, which is evolving so rapidly that it has itself become a focus for research. Li et al. [Li 2015] have produced a review of the software engineering technical debt literature, from which we can extract insights useful to policymakers. Although they studied only the literature relating to technical debt in software engineering, their conclusions are, at least in part, applicable to any field in which the components of the finished product are executed within software tools before being committed to operational form. This covers a wide array of knowledge-intensive endeavors, including mechanical system design, electronic design, framing of legislation, process design, architecture, and even book authorship.

In this thread, I explore the sources of the tension between the modern reality of technical debt as an enterprise issue, and the historical situation of technical debt as a technological issue. This can serve as a guide for policymakers in reframing technical debt from a technological issue dependent for resolution on enterprise resources, to an enterprise issue dependent for resolution on technological resources.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

Other posts in this thread

Posted on

Cultural debt can be the primary driver of technical debt

Last updated on June 2nd, 2018 at 10:35 pm

Cultural debt can be expensive because, like technical debt, it can incur ongoing metaphorical interest charges. Schein defines organizational culture as “…a pattern of shared basic assumptions learned by a group as it solved its problems of external adaptation and internal integration…” [Schein 2016]. Following the concept of technical debt, we can regard as cultural debt the subset of shared basic assumptions comprising enterprise culture that are no longer fitting for enterprise realities. We can also include as cultural debt any assumptions that ought to be shared, but which are missing or are only partially shared. And we can include shared assumptions that conflict with each other and need to be resolved.

An example of cultural debt: the term “IT”

A tape measure calibrated in both feet/inches and meters/centimeters
A tape measure calibrated in both feet/inches and meters/centimeters. The need to possess tools that serve both measurement systems can be viewed as the metaphorical interest charges on a technical debt resulting from the failure to retire the older “English” system. But from another perspective, the debt involved is actually cultural. Retiring the older system would truly involve a cultural shift.

For most modern enterprises, one element of cultural debt is the very term IT itself — information technology. Coined in 1958 by Leavitt and Whisler [Leavitt 1958], the term was apt up to as recently as 20 years ago, when the role of IT was primarily management, storage, retrieval, manipulation, and presentation of data — information — by technological means. Although those functions remain relevant, the responsibilities of IT have expanded dramatically since 1958. In many organizations, IT is now responsible for designing, implementing, and maintaining the communication infrastructure, including Internet access, personal computers, networking, Web presence, telephones, video conferencing equipment, and television.

In modern organizations in which communication plays a critical and strategic role, an essential element for success is a clear understanding of what IT does and what it contributes. To regard IT as the “information technology” function of the enterprise is, therefore, to risk overlooking and undervaluing these more recently acquired responsibilities. And since the IT function is no longer solely responsible for enterprise information, using the name “IT” or the term information technology risks overvaluing the role of the IT organization relative to information management, while undervaluing its role relative to communications.

In Schein’s culture framework, the term IT reflects a shared assumption about the focus and span of the IT function. That assumption is that IT is responsible for information—an assumption that is no longer well aligned to the reality of the role of IT. We can regard this misalignment as a cultural debt.

The consequences of this particular kind of cultural debt can be severe. For instance, IT is typically responsible for selecting and configuring software for personal computers (PCs) — both desktop and laptop. This responsibility can arise as a consequence of two shared assumptions. First, that computers process information, and second, that IT is responsible for technology-based information processing. The result is that decisions about what many regard as a “personal” computer are not in the control of the person who uses the computer. This conflict in shared assumptions can lead to conflict between PC users and IT, when the IT decision is at variance with their personal preferences.

Worse, a centralized decision process for determining PC configurations is likely to produce outcomes less suitable than would a process more focused at the individual level, which only adds to the frustrations of PC users, and exacerbates the conflict between them and IT. To mitigate the risk that some PC users might try to circumvent IT policy, IT must deploy technology to ensure adherence to their policies. We can regard all of that activity, on the part of both IT and the PC users, as metaphorical interest charges on cultural debt.

An example of retiring cultural debt

In 1987, Edward Yourdon founded a magazine then known as American Programmer. In 1990, Cutter Information Corporation purchased the rights to American Programmer and created Cutter IT Journal, which name includes the term IT. At the time IT was more suitable than the term programmer. As noted above, the term IT, while once useful and apt, is now outmoded at best and often misleading. Just as the functional name IT in organizations constitutes cultural debt, so it does in the name of a journal.

So in the autumn of 2016, Cutter IT Journal retired the cultural debt in its name, and became Cutter Business Technology Journal. Journals rarely change their names. When they do, the impact of the journal is temporarily depressed because of the split of citations between the former title and the new title for two years or so [Tempest 2005]. But as research fields change, their journals must keep pace. Evidently Cutter felt a significant need to retire its cultural debt — significant enough to justify a temporary effect on impact.

What about cultural debt retirement in companies?

Difficulties associated with retiring cultural debt in companies depend strongly on both the nature of the culture and the nature of the debt. To provide insight into the issues that can arise, let’s continue with our exploration of the term IT and its cultural implications.

In many organizations, IT reports to an enterprise Chief Information Officer (CIO). Associated with this official’s title are some of the same cultural debts we find associated with the name of the IT organization. First, within their organizations, CIOs aren’t the only officers with information management responsibility. Second, many CIOs have responsibilities that extend beyond information management, to include, for example, the communication infrastructure. And unlike other peer titles such as CEO, CFO, CMO, and COO, the CIO title evokes separation from business-oriented decisions. That separation contributes to a cultural wall between “IT” and “the business.”

When cultures view IT as an information-centric service organization, a remnant perhaps of the middle or late 20th century, they tend to regard IT as a source of expense to be minimized, rather than as a strategic partner [Ross 2000]. Trends toward strategic acceptance of IT are nevertheless favorable, with room for improvement, according to recent surveys of CIOs [CIO 2018], probably because of reality.

The reality is that business technology must contribute to formulation and implementation of enterprise strategy. To the extent that CIOs and their organizations are viewed as separate from “the business,” their ability to help shape enterprise strategy is limited. This situation subjects CIOs to cultural assumptions about their responsibilities that in some instances conflict with each other, or with enterprise reality. That’s a significant source of the metaphorical interest charges on the cultural debt.

One possible way to retire this debt might entail retitling Chief Information Officer to Chief Business Technology Officer (CBTO). That’s precisely what happened at Forrester Research in 2011 [Plant 2014].

Unfortunately, the name CBTO conflicts with the three-word pattern of enterprise officer titles (C*O), which might create an urge to name the office Chief Technology Officer (CTO). But that role usually has responsibility for the functions that create technological products or services. Thus, for many organizations, to create a CBTO where there is already a CTO might create further sources of conflict. Using the CTO designation for the CBTO is probably impractical.

But we must find some way to retire this particular cultural debt, because it is such an effective generator of technical debt. CBTO seems to be the best available path.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread

Posted on

The Broken Windows theory of technical debt is broken

In the United States, the Broken Windows theory of crime control first appeared in the public conversation in 1982, when Kelling and Wilson described it in The Atlantic (then known as The Atlantic Monthly) [Kelling 1982]. Briefly, the theory suggests that in urban environments, by applying police resources to preventing small crimes such as vandalism, public drinking, and toll jumping, one can prevent serious crime and create an atmosphere of order and lawfulness. Gladwell popularized the idea in his explosive best seller The Tipping Point [Gladwell 2000].

Broken windows in an old abandoned factory
Broken windows in an old abandoned factory. To work in an environment dominated by properties like this must certainly be demoralizing. But whether existing technical debt actually causes people to make choices that incur new technical debt is another question. At this point, it’s an open question.

In the year before Gladwell’s work appeared, Hunt and Thomas incorporated the Broken Windows theory into their work, The Pragmatic Programmer, suggesting it as a justification for the importance of retiring technical debt immediately upon discovering it [Hunt 1999]. Briefly, the theory as applied to technical debt in software is that tolerating low quality and technical debt in a given asset encourages further degradation of quality and incurring additional technical debt. Within the software community, the Broken Windows theory of managing technical debt is widely accepted [Note a].

However, between Kelling’s work in 1982 and the work of Hunt and Thomas in 1999, criminologists and sociologists had become skeptical of the Broken Windows theory as applied to crime prevention. As far back as 1998, investigations had begun to cast doubt on the Broken Windows theory [Harcourt 1998]. In 2006, Eck and Maguire assembled a review of the escalating controversy [Eck 2006]. Research by O’Brien, Sampson, and Winship, based on “big data” analyses, failed to produce evidence of validity of the Broken Windows theory beyond a weak positive correlation between social orderliness and lawful behavior [O’Brien 2015]. Indeed, their research instead showed a very strong positive correlation between private violent behavior and major crimes. Others have noted that what appeared to be positive results for the application of the Broken Windows approach to crime prevention in the 1990s was actually explainable by other phenomena [Note b].

Social scientists and criminologists have taken these findings seriously enough to have founded the Center for Evidence-Based Crime Policy at George Mason University, which maintains an evidence-based policing matrix to assist law enforcement organizations in evaluating the validity of claims about the efficacy of specific tactics and strategies, such as the Broken Windows theory. (See their review of Broken Windows Policing.)

But even as doubts developed about the efficacy of Broken Windows policing for crime prevention, Broken Windows continued to find adherents relative to managing technical debt in software assets. The software engineering community thus finds itself, perhaps, in the same position with respect to Broken Windows as it is with respect to the Tragedy of the Commons. Broken Windows and the Tragedy of the Commons are both fine analogies, but the fields that originated them now have superior ways of understanding the phenomena in question.

Maybe it’s time for the engineering community to re-examine Broken Windows as it pertains to technological asset quality and technical debt. At this time, the author is aware only of anecdotal support for the Broken Windows theory of technical debt management. Perhaps the Broken Windows theory will work better in engineering than it did in social science or criminology, but do you want to bet your company on that?

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Eck 2006] J. Eck and E.R. Maguire. “Have Changes in Policing Reduced Violent Crime? An Assessment of the Evidence,” in Blumstein, Alfred, and Joel Wallman, eds. The Crime Drop in America, Revised Edition. Cambridge: Cambridge University Press, 2006, 207-265.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Gladwell 2000] Malcolm Gladwell. The Tipping Point: How Little Things Can Make a Big Difference. New York: Little, Brown and Company, 2000.

Order from Amazon

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Harcourt 1998] Bernard E. Harcourt. “Reflecting on the Subject: A Critique of the Social Influence Conception of Deterrence, the Broken Windows Theory, and Order-Maintenance Policing New York Style,” 97 Michigan Law Review 291, 1998.

Available: here; Retrieved: June 26, 2017

Cited in:

[Hunt 1999] Andrew Hunt and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Reading, Massachusetts: Addison Wesley Longman, 1999.

Order from Amazon

Cited in:

[Kelling 1982] Kelling, George L. and James Q. Wilson. “Broken Windows: The police and neighborhood safety,” The Atlantic, 249(3):29–38, March 1982.

Available: here; Retrieved: June 25, 2017

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Nord 2016] Robert L. Nord. “The Future of Managing Technical Debt”, SEI Blog, August 29, 2016 By Robert Nord In Technical Debt

Explicitly defines technical debt to exclude much: “Defects, new features not yet implemented, and lack of process lie outside the boundary and are not technical debt.”

Cited in:

[Note b] Articles and blog entries questioning the validity of the Broken Windows theory of crime prevention:

[Nuwer 2013] Rachel Nuwer. “Sorry, Malcolm Gladwell: NYC’s Drop in Crime Not Due to Broken Window Theory,” SmartNews blog at smithsonian.com, February 6, 2013.

Available: here; Retrieved: June 25, 2017.

Cited in:

[O’Brien 2015] Daniel O’Brien, Robert J. Sampson, and Christopher Winship. “Ecometrics in the Age of Big Data: Measuring and Assessing ‘Broken Windows’ Using Large-scale Administrative Records.” Sociological Methodology 45: 101-147, 2015.

Available: here; Retrieved: June 25, 2017

Cited in:

[Childress 2016] Sarah Childress. “The Problem with ‘Broken Windows’ Policing,” PBS FrontLine, June 28, 2016.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006a] Bernard E. Harcourt. “Bratton's ‘broken windows’:No matter what you’ve heard, the chief’s policing method wastes precious funds,” Los Angeles Times, April 20, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006b] Bernard E. Harcourt and Jens Ludwig. “Broken Windows: New Evidence From New York City and a Five-City Social Experiment,” University of Chicago Law Review, Vol. 73, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread

Posted on

Zero tolerance and work-to-rule deliveries create an adversarial culture

Last updated on February 1st, 2018 at 07:29 am

Defining technical debt at the level of specificity needed for project objectives is difficult. Confronted with this difficulty, some internal customers of technologists adopt a zero-tolerance approach to technical debt, without specifically defining technical debt. Post-delivery — sometimes much, much, post — when technical debt is discovered or recognized, technologists are held responsible, even in cases when no one could have predicted that a specific artifact would eventually come to be regarded as technical debt. This sets up an adversarial dynamic between technologists and their internal customers.

Delayed or cancelled flights
Trouble at the airport. When airline pilots engage in work-to-rule actions, the immediate result can be large numbers of delayed or cancelled flights. The longer-term result might be beneficial to pilots, the airline, and the public, but only if labor peace can be restored, and the damage to the flying public can be overcome. So it is with work-to-rule deliveries as a way of dealing with zero-tolerance technical debt policies. The organization must overcome the adversarial culture that results from indiscriminate attempts to control technical debt. Technologists do gain some measure of protection by working to rule, but the longer-term benefit of the organization’s learning to manage technical debt arrives only if the adversarial culture can be overcome. Image (cc) Hotelstvedi courtesy Wikimedia.

And that’s when the trouble begins.

Within this adversarial dynamic, technologists try to protect themselves against future recriminations by “working to rule.” They perform only work that is specified by the internal customer. If they find something additional that must be done, they perform that work only if they successfully obtain the customer’s approval. Some customers continue to adhere to a zero-tolerance policy with respect to technical debt, but such a non-specific requirement cannot be met. Because technologists are “working to rule,” they use the ambiguity of the zero-tolerance requirement to assert that they performed all work that was sufficiently specified. This level of performance is analogous to the work-to-rule actions of some employees involved in labor disputes with their employers, and who are literally in compliance with the requirements of the employer, but only literally [LIBCom 2006].

Requiring deliverables to be totally free of technical debt contributes to formation of an adversarial culture, wherein the adversaries are the technologists and their internal customers. Shedding that adversarial culture, once it sets in, can be difficult. Compelling employees, vendors, or contractors to deliver work that’s free of all technical debt is therefore unlikely to succeed. Whether work is performed in-house by employees, or is outsourced, or is performed in-house by contractors, deliverables that meet the minimum possible interpretation of the objectives of the effort are almost certainly burdened with unacceptable levels of technical debt. What can we do to prevent this?

To avoid creating an adversarial culture, we can specify in project objectives some kinds of technical debt that must be removed in toto. To ensure steady progress in technical debt retirement, develop a statement of objectives that includes complete retirement of at least one well-defined class of technical debt, emphasizing debt classes that have the highest anticipated MICs in the near term. Other well-defined classes of technical debt can be addressed on a best-effort basis.

We must accept that any other forms of technical debt that remain at the end of a given project, or any constructions that later come to be recognized as technical debt, are just the “cost of doing business.” We’ll get to them, but unfortunately, not this time.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Eck 2006] J. Eck and E.R. Maguire. “Have Changes in Policing Reduced Violent Crime? An Assessment of the Evidence,” in Blumstein, Alfred, and Joel Wallman, eds. The Crime Drop in America, Revised Edition. Cambridge: Cambridge University Press, 2006, 207-265.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Gladwell 2000] Malcolm Gladwell. The Tipping Point: How Little Things Can Make a Big Difference. New York: Little, Brown and Company, 2000.

Order from Amazon

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Harcourt 1998] Bernard E. Harcourt. “Reflecting on the Subject: A Critique of the Social Influence Conception of Deterrence, the Broken Windows Theory, and Order-Maintenance Policing New York Style,” 97 Michigan Law Review 291, 1998.

Available: here; Retrieved: June 26, 2017

Cited in:

[Hunt 1999] Andrew Hunt and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Reading, Massachusetts: Addison Wesley Longman, 1999.

Order from Amazon

Cited in:

[Kelling 1982] Kelling, George L. and James Q. Wilson. “Broken Windows: The police and neighborhood safety,” The Atlantic, 249(3):29–38, March 1982.

Available: here; Retrieved: June 25, 2017

Cited in:

[LIBCom 2006] “Work-to-rule: a guide.” libcom.org.

Available: here; Retrieved: May 9, 2017.

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Nord 2016] Robert L. Nord. “The Future of Managing Technical Debt”, SEI Blog, August 29, 2016 By Robert Nord In Technical Debt

Explicitly defines technical debt to exclude much: “Defects, new features not yet implemented, and lack of process lie outside the boundary and are not technical debt.”

Cited in:

[Note b] Articles and blog entries questioning the validity of the Broken Windows theory of crime prevention:

[Nuwer 2013] Rachel Nuwer. “Sorry, Malcolm Gladwell: NYC’s Drop in Crime Not Due to Broken Window Theory,” SmartNews blog at smithsonian.com, February 6, 2013.

Available: here; Retrieved: June 25, 2017.

Cited in:

[O’Brien 2015] Daniel O’Brien, Robert J. Sampson, and Christopher Winship. “Ecometrics in the Age of Big Data: Measuring and Assessing ‘Broken Windows’ Using Large-scale Administrative Records.” Sociological Methodology 45: 101-147, 2015.

Available: here; Retrieved: June 25, 2017

Cited in:

[Childress 2016] Sarah Childress. “The Problem with ‘Broken Windows’ Policing,” PBS FrontLine, June 28, 2016.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006a] Bernard E. Harcourt. “Bratton's ‘broken windows’:No matter what you’ve heard, the chief’s policing method wastes precious funds,” Los Angeles Times, April 20, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006b] Bernard E. Harcourt and Jens Ludwig. “Broken Windows: New Evidence From New York City and a Five-City Social Experiment,” University of Chicago Law Review, Vol. 73, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread

Posted on

Technical debt: monochronic and polychronic cultures

Last updated on June 2nd, 2018 at 09:50 pm

In the context of today’s prevalence of global teams, the word culture evokes consideration of the differences between different national cultures. We think of the difference between Singapore and Latvia. Or we might also consider regional differences between Massachusetts and Texas. Or between San Francisco and Los Angeles. These differences are real. They do affect the way the inhabitants of these places deal with technical debt. But these cultural differences are topics for another time.

A Simmental cow and calf
A Simmental cow and calf

For now, let’s consider how a particular cultural attribute of teams or groups in organizations affects how they manage technical debt. Specifically, how we regard time correlates closely with our preferences, biases, and even the way we think about approaching technical debt.

Anthropologist Edward Twitchell Hall, Jr. (1914-2009) developed fundamental concepts for thinking about how we experience space and time. In The Silent Language [Hall 1973], he introduced the concepts of polychronic time and monochronic time. They represent two different ways human cultures relate to time.

People and cultures with a monochronic orientation (M-People) regard time as linear. M-People are most comfortable when they undertake only one task at a time. They tend to sequence tasks one after the other. Their motto is “Work on one and get it done.” When M-People must tackle more than one task in an hour, they divide the hour into blocks. They dedicate each block to one task. But time division isn’t always possible. When M-People work in groups, and when they must do more than one thing simultaneously, they divide their groups. Then each subgroup can focus on one task at a time. This is more or less how conventional (pre-Agile) project teams work.

People and cultures with a polychronic orientation (P-People) regard time differently. They define time more in terms of completed tasks than in terms specified by the hands of a clock. For example, a farmer might define time by what’s happening: it’s time for planting, for harvest, for haying, for canning, or for calving. Finer increments are also defined by what’s happening: it’s time for milking, time for breakfast, or it’s sundown or sunset. More than one thing can be happening at any given time.

In organizational cultures, the differences between monochronic and polychronic orientations are evident at every level of activity. For example, in a meeting of M-People, only one person has the floor at any one time. M-People schedule their agenda items, and after they move to the next agenda item, they don’t return. (Well, they do sometimes return, but they aren’t comfortable when they do) A meeting of P-People might have several people talking at once. They bounce from topic to topic as the discussion calls for it. M-People are very uncomfortable in P-style meetings; P-People are just as uncomfortable in M-style meetings [Brenner 2018].

M-approaches can work just fine when we know in advance what the tasks entail. We can carve up our total work into pieces and address them one at a time, because they either don’t interact much, or because we can control their interactions. And we can partition the work across the available people and groups, each working on one piece of the effort.

And that has worked well for much of the work involved in producing and maintaining today’s complex technological assets.

But M-approaches can get into trouble when we work on something that’s complex or unfamiliar. When we don’t have (and can’t devise) a detailed plan of the work, dividing the work either by time or by task can be difficult to get right. To do this kind of work, we must depend more on interpersonal relationships and close teamwork than on scheduling and task partitioning. M-style approaches can flounder. And that’s where P-style approaches have big advantages.

In polychronic work styles, we’re more focused on accomplishments than schedule, because we don’t really know enough about the work to make a detailed schedule. In some instances, estimating how much work something might take is essentially impossible because we don’t understand the task in enough detail. We must then let events guide us, in the way the seasons guide the farmer. The people doing the work must adapt as they go, which requires strong relationships, tight collaboration, and trust.

And that brings us to technical debt management. We can address some kinds of technical debt with M-style approaches. We understand those debts and we know how working on their retirement will affect other work. But many forms of technical debt are so entangled with the rest of the organization’s efforts that P-style approaches are more likely to be successful.

For example, consider a project that’s aimed at retiring some kinds of technical debt. That might require access to assets that are critical to certain revenue streams. And suppose that ongoing work unrelated to retiring that technical debt might also affect these same assets. Coordinating all these efforts is complicated, even when we’re planning for only the next few weeks. But coordinating these efforts six months or twelve months in advance is probably impossible, because we can’t create schedules reliable enough to support such planning. The schedules of the various interlocking activities aren’t well enough controlled for that, any more than a dairy farmer controls the calving schedule. The cows control the calving schedule.

For many technical debt management activities, M-style approaches are simply unworkable. We cannot answer with useful precision questions like these:
  • How much will it cost—total—to retire the technical debt of that kind?
  • How long will it take?
  • Will that debt retirement interfere with our plans for the rollout of the Marigold products next September?

Conventional project management might not work well for technical debt retirement efforts. Regarding them as “projects,” with defined schedules, defined budgets, and defined staff assignments might be a serious error. Agile approaches, which are inherently more P-style, might work better.

References

[APA 2013] American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Association Publishing, 2013.

Order from Amazon

Cited in:

[Ariely 2010] Dan Ariely. “You are what you measure,” Harvard Business Review 88:6, p. 38, 2010.

Available: here; Retrieved: June 4, 2018

Cited in:

[Bouwers 2010] Eric Bouwers, Joost Visser, and Arie van Deursen. “Getting What You Measure: Four common pitfalls in using software metrics for project management,” ACM Queue 10: 50-56, 2012.

Available: here; Retrieved: June 4, 2018

Cited in:

[Brenner 2018] Richard Brenner. “Polychronic Meetings,” Point Lookout 18:1, January 3, 2018.

Available here; . Forthcoming.

Cited in:

[CIO 2018] CIO. “2018 State of the Cio: CIOs Race Towards Digital Business,” CIO, winter 2018.

Available: here; Retrieved March 30, 2018

Cited in:

[Cooper 1857] James Fenimore Cooper. The Last of the Mohicans, New York: Bantam Classics, 1982.

Order from Amazon

Cited in:

[Doran 1981] George T. Doran. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives”, Management Review, 70:11, pp. 35-36, 1981.

Cited in:

[Eck 2006] J. Eck and E.R. Maguire. “Have Changes in Policing Reduced Violent Crime? An Assessment of the Evidence,” in Blumstein, Alfred, and Joel Wallman, eds. The Crime Drop in America, Revised Edition. Cambridge: Cambridge University Press, 2006, 207-265.

Order from Amazon

Cited in:

[Fowler 2006] Martin Fowler. “CodeSmell,” Martin Fowler (blog), February 9, 2006.

Available: here; Retrieved: June 6, 2018

Cited in:

[Gladwell 2000] Malcolm Gladwell. The Tipping Point: How Little Things Can Make a Big Difference. New York: Little, Brown and Company, 2000.

Order from Amazon

Cited in:

[Hall 1973] Edward T. Hall. The Silent Language. New York: Anchor Books, 1973.

Originally published in 1959. Order from Amazon

Cited in:

[Haque 2018] Md Shariful Haque, Jeff Carver, and Travis Atkison. "Causes, impacts, and detection approaches of code smell: a survey." Proceedings of the ACMSE 2018 Conference. ACM, 2018.

Cited in:

[Harcourt 1998] Bernard E. Harcourt. “Reflecting on the Subject: A Critique of the Social Influence Conception of Deterrence, the Broken Windows Theory, and Order-Maintenance Policing New York Style,” 97 Michigan Law Review 291, 1998.

Available: here; Retrieved: June 26, 2017

Cited in:

[Hunt 1999] Andrew Hunt and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Reading, Massachusetts: Addison Wesley Longman, 1999.

Order from Amazon

Cited in:

[Kelling 1982] Kelling, George L. and James Q. Wilson. “Broken Windows: The police and neighborhood safety,” The Atlantic, 249(3):29–38, March 1982.

Available: here; Retrieved: June 25, 2017

Cited in:

[LIBCom 2006] “Work-to-rule: a guide.” libcom.org.

Available: here; Retrieved: May 9, 2017.

Cited in:

[Leavitt 1958] Harold J. Leavitt and Thomas L. Whisler. “Management in the 1980s,” Harvard Business Review, November-December, 36, 41-48, 1958.

Cited in:

[Li 2015] Z. Li, P. Avgeriou, and P. Liang. “A systematic mapping study on technical debt and its management,” Journal of Systems and Software 101, 193-220, 2015.

Cited in:

[Nord 2016] Robert L. Nord. “The Future of Managing Technical Debt”, SEI Blog, August 29, 2016 By Robert Nord In Technical Debt

Explicitly defines technical debt to exclude much: “Defects, new features not yet implemented, and lack of process lie outside the boundary and are not technical debt.”

Cited in:

[Note b] Articles and blog entries questioning the validity of the Broken Windows theory of crime prevention:

[Nuwer 2013] Rachel Nuwer. “Sorry, Malcolm Gladwell: NYC’s Drop in Crime Not Due to Broken Window Theory,” SmartNews blog at smithsonian.com, February 6, 2013.

Available: here; Retrieved: June 25, 2017.

Cited in:

[O’Brien 2015] Daniel O’Brien, Robert J. Sampson, and Christopher Winship. “Ecometrics in the Age of Big Data: Measuring and Assessing ‘Broken Windows’ Using Large-scale Administrative Records.” Sociological Methodology 45: 101-147, 2015.

Available: here; Retrieved: June 25, 2017

Cited in:

[Childress 2016] Sarah Childress. “The Problem with ‘Broken Windows’ Policing,” PBS FrontLine, June 28, 2016.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006a] Bernard E. Harcourt. “Bratton's ‘broken windows’:No matter what you’ve heard, the chief’s policing method wastes precious funds,” Los Angeles Times, April 20, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

[Harcourt 2006b] Bernard E. Harcourt and Jens Ludwig. “Broken Windows: New Evidence From New York City and a Five-City Social Experiment,” University of Chicago Law Review, Vol. 73, 2006.

Available: here; Retrieved: June 25, 2017

Cited in:

Cited in:

[Plant 2014] Robert Plant. “IT Has Finally Cracked the C-Suite,” Harvard Business Review, July 16, 2014.

Available: here; Retrieved: April 8, 2018

Cited in:

[Ross 2000] Jeanne W. Ross and David F. Feeny. “The Evolving Role of the CIO,” in Framing the Domains of IS Management Research: Glimpsing the Future through the Past, edited by Robert W. Zmud. Pinnaflex, 2000.

Available: here; Retrieved: December 20, 2017.

Cited in:

[Schein 2016] Edgar H. Schein. Organizational Culture and Leadership, Fifth Edition, San Francisco: Jossey-Bass, 2016.

Order from Amazon

Cited in:

[Tempest 2005] “The effect of journal title changes on impact factors,” Learned Publishing 18, 57–62, 2005.

Available: here; Retrieved: April 5, 2018

Cited in:

Other posts in this thread