Last updated on July 13th, 2021 at 10:02 am
Modularity is a standard design approach for complex systems. But because modularity can be be a factor in the accumulation and persistence of technical debt, temporary demodularization can help control technical debt. And longer-term demodularization can reduce the rate of accumulation of technical debt.
Modularization is a standard design approach for complex systems
Since the 1970s, modular design of systems has been de rigueur in both software and hardware. More than that, modularization has demonstrated that it’s an essential feature of maintainable, adaptable, and extensible systems [Parnas 1979] [Sullivan 2001]. And we now understand that modularization is a foundational attribute of loose coupling in systems. Loose coupling enables system designers and maintainers to work in parallel, with independence. Independent parallelism renders systems economical at levels of complexity beyond what is achievable with tighter coupling [Orton 1990].
Eliminating duplication is one reason why modularization reduces maintenance and enhancement costs. Modularization enables system designers to create a single system element that provides needed functionality to many other parts of the system. Because there is then only one copy of the system element that provides that capability, adapting it in response to new needs, or to correct defects, need be done only once.
That’s a big deal. If we provided that capability through multiple system elements, and we needed to adapt it, adaptation would be necessary for each of those elements. Moreover, the multiplicity of elements opens the possibility that we might not perform that adaptation consistently. That could create further problems. Eliminating duplication is a most useful property of modularization.
Modularization provides many other advantages. For example, modularization shortens time-to-market for new capabilities. When extending the system by adding new capability, we sometimes need access to capabilities present in existing modules. We can access those capabilities easily in modularized systems, because those modules are already in a form that makes access easy. We have no need to recreate them for the new capability we’re implementing. They exist, and they’re already tested and ready to go. In this way, modularization shortens time-to-market for new capability.
Modularization has a dark side
It’s a bit of a story to show illustrate the dark side of modularization. Let’s give a name to the modular system element for which we’ve removed all duplicates. Since it’s now unique, I’ll call that modular system element “U.” Any system element that interacts with U is now indirectly coupled to every other system element that interacts with U. And that’s where the trouble comes in.
A scenario that illustrates the problem
When implementing a new capability N, and N needs access to U, we gain the advantages described above. But suppose N needs U to do something a bit differently from what U now does. Sometimes we can extend U to accommodate N without disturbing U’s existing “client base” of system elements that already depend on U. That’s no problem. But suppose what N now needs U to do would disturb U’s client base if we implemented the changes in the way we would do it if we were starting fresh. In that case, we would need to modify all of U’s existing clients. And then we would need to re-test everything. So let’s suppose that we don’t have time or resources to do all that work. We requested the time and resources, but we didn’t receive an approval.
Instead, we found a way to extend U in a less elegant, less maintainable, but still reliable way that doesn’t disturb U’s existing clients, and does meet N’s needs. We do that instead, promising ourselves that we’ll go back someday, when we have approval for the time and resources we need. Then we would “fix” U so that it serves both its existing clients and N in the “correct” way.
A natural question
That’s one form—exactly—of what we call technical debt. In this scenario we’ve illustrated one way in which modularization leads to technical debt formation.
So a natural question arises: would it make sense instead to create a new system element—call it U2—that meets N’s needs, and also meets the needs of U’s existing client base? It would make sense, in many cases, if only they knew about U2 and if only they had the time and resources to make the change. To create such a U2 would be demodularization—that is, a violation of modularity—and that is indeed heresy. It also creates a different technical debt: the obligation to convert U’s clients to become U2 clients someday, and then to delete U. But it might be the right approach.
When would demodularization help?
Under what conditions would demodularization be sensible? Here are three possibilities.
When a new and necessary adaptation is incompatible with existing forms
The scenario above is one situation in which demodularization can help. Demodularization helps when adding new capability, or adapting to a new need, requires a change to a shared module, and that change is incompatible with the existing uses of that module. Demodularization is than a useful technique, provided that we retire with due dispatch the technical debt that results.
When retiring technical debt requires an incompatible adaptation
A second situation arises during technical debt retirement operations. During technical debt retirement, it might be necessary to alter a shared module in a way that would be incompatible with the needs of its existing client base. In that case, the approach used above can be useful. First, create a successor (“U2”) to the original shared module (“U”) in a form that isn’t burdened with the technical debt that’s being retired. Then, at the same time, or over an extended period, convert all the clients of U to use the successor U2. In the meantime, the demodularization comprises a technical debt. When the conversion is complete, the you will have retired the original technical debt. Finally delete the original shared module U, thereby retiring the demodularization.
This approach entails some risk. In the interim period before you retire U, when demodularization is still in place, U and U2 are both in use. If you need changes in U, you might need to replicate them in U2. When that happens, duplication of effort can occur. This approach is useful, though, provided the interim period of demodularization is short compared to the anticipated intervals between incidents that require alterations to both U and U2. There is risk, of course, that the resources committed to finally retiring U might become unavailable after U2 is in place. In that case, the technical debt portfolio will have expanded to no good end. To manage this risk, the artifice of secured technical debt can prove useful.
Partial demodularization helps when adaptations are focused
In some instances, portions of a shared system element—call it “U”—evolve very rapidly, while most of the rest of U remains stable. Technical debt can accumulate rapidly if the element remains unitary—that is, in one piece. However, in some cases we can segregate the rapidly evolving portion of U into a smaller unit—call it “S.” If we provide S as a separate shared system element, those portions of the system that are experiencing rapid evolution can access S separately, without disturbing the system elements that require access only to the stable portions of U.
Such segregation might require a bit of duplication There might be pieces of S that U needs, and which must therefore appear in both U and S. Likewise, there might be pieces of U that S needs, and which must therefore appear in both U and S.
But the segregation might be worthwhile, because changes in S usually require testing S and S’s clients. Testing can be expensive in time and resources. Because test coverage isn’t always 100% (read: test coverage is rarely 100%), changes in S entail some operational risk. Segregating S reduces that risk by protecting U’s clients from changes in S.
Later, when the rapidly evolving S stabilizes, you can re-integrate it into its former residence in U. Until that point, its segregation—and the attendant duplications—might constitute a technical debt.
Last words
Accepting modularization as an inviolable design principle is one cause of unnecessary accumulation of technical debt. It makes retiring legacy technical debt more difficult. Be prepared to violate modularity, but do so judiciously.
References
[Orton 1990] J. Douglas Orton and Karl E. Weick. “Loosely Coupled Systems: A Reconceptualization,” The Academy of Management Review, 15:2, 203-223, 1990.
Available: here; Retrieved: July 11, 2018.
[Parnas 1979] David L. Parnas. “Designing Software for Ease of Extension and Contraction,” IEEE Transactions on Software Engineering, vol. SE-5, no. 2, March 1979, 128-138.
Available: here; Retrieved: July 13, 2017
[Sullivan 2001] Kevin J. Sullivan, William G. Griswold, Yuanfang Cai, and Ben Hallen. “The structure and value of modularity in software design,” in ACM SIGSOFT Software Engineering Notes, 26:5, 99-108, 2001.
Available: here; Retrieved: July 11, 2018.
Other posts in this thread
- Managing technical debt
- Leverage points for technical debt management
- Undercounting nonexistent debt items
- Crowdsourcing debt identification
- Legacy debt incurred intentionally
- Metrics for technical debt management: the basics
- Accounting for technical debt
- Three cognitive biases
- The resilience error and technical debt
- Synergy between the reification error and confirmation bias
- Retiring technical debt can be a wicked problem
- Retiring technical debt can be a super wicked problem
- Degrees of wickedness
- Demodularization can help control technical debt
