A Shared Language to talk about Technical Debt

haeuptle · ‎2023 Nov 17

Everyone has heard of financial crisis where the economy is in trouble and where people and companies are overwhelmed with financial debts. The consequences can be disastrous. Did you know that such crisis happen often in software development? Technical debt is a sneaky plague that can hurt projects, deliveries, people, and sometimes an entire company. This blog is the first in a series of blogs on technical debt on what it is, where it comes from, and how to deal with it. It will shed light on something we too often ignore, but can take control over by being proactive!

The intention of this blog is to define the term and to provide a shared language to talk about technical debt.

What is Technical Debt?

Ward Cunningham: "Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a standstill under the debt load of an unconsolidated implementation."

Technical debt, a term coined by Ward Cunningham, is a concept in software development that reflects the implied cost of additional rework caused by choosing a quick and easy solution now instead of using a better approach that would take longer. It's a metaphor that equates software development to financial debt, illustrating the trade-off between short-term and long-term benefits. When a development team takes shortcuts, such as skipping parts of the coding process to meet deadlines or opting for a less optimal solution to save time, they incur technical debt. While these shortcuts might speed up the development process in the short term, they often lead to more work in the future as the team has to go back and fix the issues that arise from these shortcuts. This is similar to how taking on financial debt can provide immediate funds but requires repayment with interest over time. Technical debt can be categorized into two types: intentional and unintentional. Intentional technical debt is a strategic decision made with the understanding of the future cost. Unintentional technical debt, on the other hand, is accrued due to lack of knowledge or oversight, and is often discovered later when issues arise. It's important to manage technical debt effectively. If left unchecked, it can accumulate 'interest' in the form of decreased productivity, increased complexity, and reduced code quality. This can lead to a slower pace of development, making it harder to implement new features or fix bugs. However, not all technical debt is bad. Sometimes, incurring technical debt can be a strategic decision. For instance, a startup might choose to incur some technical debt to speed up development and get their product to market faster. The key is to manage it effectively and pay it off regularly by refactoring code, improving documentation, and investing in automated testing. Technical debt is an inherent part of software development that needs to be managed strategically. By understanding its implications and making informed decisions, development teams can balance the need for speed with the importance of code quality and long-term maintainability.

Technical Debt - Dilbert by Scott Adams

Types of Technical Debt

There are various forms of technical debt. The following list is not exhaustive, but it covers the most common types of technical debt.

Code Debt: A prevalent form of technical debt, code debt stems from shortcuts or subpar coding practices, resulting in inefficient or disorganized code, which is hard to read, maintain and test. This make all development and maintenance tasks slower and will degrade the "enjoyment" of writing code while demotivating engineers. More details on how to deal with code debt can be found in the blog post Clean Code: Writing maintainable, readable and testable code

Design Debt: Design debt emerges when the system design is compromised, often due to time pressures or inadequate foresight during the design phase, resulting in a system that's challenging to maintain and expand. Due to this kind of debt developers can’t get confidence quickly that their code will not break existing functionality and dependencies.

Testing Debt: Testing debt arises from insufficient testing, including a lack of unit, integration, or system tests. This can lead to unnoticed bugs and software issues. Another important type of testing are flaky tests. More details on how to deal with testing debt tests can be found in the blog posts:

Documentation Debt: Documentation debt occurs when system documentation is incomplete or outdated, hindering new team members' understanding and slowing down system maintenance and expansion.

Infrastructure Debt: Infrastructure debt arises when the supporting infrastructure, such as the operating system, database, development environment, build systems, and CI/CD pipelines, are outdated or inefficient.

Architectural Debt: Architectural debt occurs when the system's architecture fails to meet current or future functional and non-functional requirements, making the system difficult to maintain, adapt, or scale.

Knowledge Debt: Team lacks necessary expertise or the knowledge is not spread among the team, leading to "knowledge silos" and bottlenecks that can slow development and increase risk. This may be due to staffing gaps and turnover or inherited orphaned code/projects.

Dependency Debt: Dependency debt occurs when a system depends on outdated or unsupported third-party libraries or services. The Dependencies are unstable or rapidly changing. Teams are unable to take advantage of new improvements and remain vulnerable to security problems. This also can slow down the onboarding of new hires and cause frustration for current developers who are forced to work with older versions. More details on how to deal with dependency debt can be found in the blog post Keeping Dependencies Up To Date

Migration Debt: Migration is needed or in progress: This may be motivated by the need to scale, license issues, costs, to reduce dependencies, or to avoid deprecated technology. It can also occur when a migration was poorly executed or abandoned: This may have resulted in maintaining two versions.

Unused features / Dead Code Debt Dead and/or abandoned code: Code/features/projects were replaced or superseded but not removed. More features creates more conditions and more edge cases that developers have to design around, which erodes the delivery speed.

Reliability or Performance Debt: These can affect the customer experience as well as the ability to scale the business.

Tool Debt Inefficient tools (both proprietary and third-party) can introduce friction or overhead for developers, slowing delivery speed.

Manual Process Debt When a part of the product delivery isn't automated, it requires more manual time and effort.

Automated deployments Debt Automated deployment workflows enable the ability to deliver features to customers continuously and at will.

Coupling Debt Coupling between modules or services leads to teams potentially blocking each other, slowing down delivery speed. Domain Driven Design is a great methodology to deal with this kind of debt.

Duplication Debt Duplication of code, services, or functionality can lead to wasted effort and increased maintenance costs. On the other hand should duplication be balanced with the need for autonomy and independence of teams. Reduction of duplication can cause dependencies between teams, which can slow down delivery speed. So it is important to consider the trade-offs.

Conclusion

In conclusion, the legacy codebase, while being a significant asset, also presents a formidable challenge due to the inherent technical debt. This debt, if not addressed promptly, can become a substantial obstacle in delivering high-quality solutions and adapting to the ever-evolving market needs. Therefore, it is important to implement measures to manage and reduce this technical debt effectively. In the forthcoming year, I will publish a series of blog posts, each focusing on different aspects of technical debt. These posts will not only shed light on the complexities of the issue but also propose potential strategies, tools, practices, behaviorus, constraints and other approaches to deal with it.

Resources

Defining, Measuring, and Managing Technical Debt

Bottleneck Tech Debt

matt · ‎2023 Nov 17

I'm working on some code now that's incurred a huge amount of technical debt. It just makes the whole process extremely difficult.

Every programmer should read this blog and understand the concepts.

former_member579966 · ‎2023 Nov 17

Great article, one comment and one question:

What I would like to mention supplemental to your list of debts is "product debt", although it is not technical. Meaning, shortcuts or even wrong directions taken on the feature side of the product e.g. due to lack of a clear product vision.

Regarding managing debt strategically, do you have more details or best practices to share about that?

Jelena_Perfiljeva · ‎2023 Nov 17

I've seen much fewer instances of "intentional debt" (and I've created some myself) and I would take those any time over what I'd call "technical debt by negligence". I think "unintentional" is too mild word for it. That would mean someone (even an otherwise well-meaning person) made a woopsie or maybe it was just the technical limitation at the time the code was created.

But you'd be surprised how many developers are still out there who either just don't give a hoot about this subject or, worse, intentionally stick bananas in their ears and go "la-la-la-la" when someone tries to tell them there are better / more efficient ways to do things they're doing.

Sadly, it's frequently those people, and not the ones who're wanting to make change and introduce refactoring, who are given support of IT managers. And that brings me to the root cause: IT managers and the customers of consulting companies largely don't give a hoot about this either. They love to complain afterwards but want to do literally nothing to help developers to avoid technical debt. I hope next blog post on this subject will address this aspect.

I had a presentation at SAP TechEd in 2018 on upskilling and it had a section about technical debt. Feel free to use the image that went with it free for educational purposes, colleagues. 🙂

BenjaminWeishei · ‎2023 Nov 18

As developers, we're committed to solving our business colleagues' problems as swiftly as possible. One might even say we're conditioned to do so since we're often praised for rapid implementations and heroic firefighting efforts.

Lending a quick hand to our business colleagues is indeed an honorable approach at first glance. Unfortunately, in practice, there's often a lack of awareness about software quality and the fundamentals of software design (for instance, how many out of 100 SAP developers can explain the SOLID principles and design patterns?).

This is why many legacy SAP systems are riddled with unintentional debt in the form of spaghetti code, dead code, inconsistent patterns, pseudo object orientation, missing tests, and more...

This is not just an IT problem; it's a business problem! Projects exceed budgets, processes fail to digitalize, key features go undelivered, bugs proliferate, knowledge silos develop, and security incidents occur.

Architects and development leads can step up by helping the business/management to rethink, by continuously monitoring and making technical debt transparent. This supports that decisions are made consciously, considering the long-term costs.

A current topic in the HANA context is the Clean Core initiative. This approach has significantly increased the "mountain of debt" in SAP legacy code in recent years. I am still in search of a standard SAP tool that would allow me to quickly and at any time reflect the progress of cloudification in private cloud/on-premise systems for development teams, product owners, and management as a key performance indicator (KPI).

P.S.: Many teams outside the SAP context also face similar problems

matt · ‎2023 Nov 19

I took over the internal development for my employers own SAP systems. I told the manager that I would take more time than expected over enhancements to existing code, and indeed fixes, due to technical debt. He didn't really understand, but trusted me.

So I refactored as I went along, repairing windows.

Now when changes are needed, they arrive quickly, and with far fewer bugs. So he's happy.

Funny thing. Friday I implemented Additional Data A and B in tx VA43 by following a really helpful blog. Then I noticed a comment at the end that said this way was obsolete and there's a far better way. By then the implementation had been tested, so I said to the manager - forget that, I've done it in an un-performant way, I'll redo in Monday.

Again, no problem. He's seen the benefits of doing it right first time. 🙂

00022111734 · ‎2023 Nov 19

Thanks for insightful blog

huseyindereli · ‎2023 Nov 27

One more type;

Technical Debt Debt - Lacking the idea of technical debt

Good read, thanks!

DinoPhilips · ‎2023 Dec 07

Technical Debt: causes & remediation options ....