How to Measure Technical Debt

So one of the biggest challenges I’ve repeatedly heard about technical debt is that although the delivery team knows it exists, there is no easy way to measure the actual cost of technical debt.  However, there are ways to help bridge the unspecific to the specific.

With technical debt, using a financial analysis of comparing addressing the technical debt now versus addressing the technical debt later (at a pre-determined point) will bring useful data to make informed decisions.

So how accurate is that data?  What level of confidence will leaders place on it?  One method to help bridge that trust gap would be to use technical debt evaluation tool like SonarQube that can give detailed counts of technical debt risks.  This can then be used as a “ground level up” method of determining the full cost of fixing technical debt now and given as detailed evidence that this isn’t simply “developer guesswork”.  From organizing the reported debt, the seven axes of Quality reported by SonarQube can then be mathematically estimated and used as supporting evidence.
The example chart gives a comparison that non-technical employees can understand.  This speaks in clear financial figures and can be measured to determine progress and where key decisions can be made.

No longer does technical debt have to be the software delivery department wrestling with business.  There can be a common ground for discussion, collaboration and agreement on how much effort should be made to resolve any accrued technical debt.

Engagement Ceremony

If there is ever an art to delivering technology, it’s about knowing when and how to prioritize events as they unfold.  Some events are small and may only need an email.  Others, a simple test or instant message may do.  At higher levels of modality are direct phone ​calls with screen sharing, Skype, or even face to face meetings.

A recognition of when there is not enough or too much ceremony for the roles involved.  If a dozen emails go by for something that would take two minutes to discuss, then using email is the wrong modality.  Sometimes, that isn’t clear until the communication unfolds, but immediately change the mode if you see that it is dragging down in the form of communication provided.

Also, too much communication can happen.  If a meeting takes up more time than actual doing the work or logging the work is taking up more time than just doing it, the process should be questioned.

​It’s all about business alacrity and a good project manager, and any business professional, should always look for ways to reduce waste in delivery.

Types of Technical Debt

There are three primary types of Technical Debt:

1. Naïve technical debt
2. Unavoidable technical debt
3. Strategic technical debt

Naïve technical debt is common where a specific best practice is not followed due to being unaware of the best design pattern, unfamiliar with naming convention, documentation guidelines or even coding guidelines such as following abstraction in development and having a method serve one purpose.

This happens more frequently with junior level developers, but even the most senior developers can miss things and as a result, create technical debt.  My recommendation would be to use the guidelines in the books “Clean Code” by Robert Martin (http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882) and “Pragmatic Programmer” by Andrew Hunt and David Thomas (http://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=pragmatic+programmer).  This will include several Extreme programming principles such as code reviews, refactoring, TDD and pair programming to name some.

Unavoidable technical debt will happen due to upgrades in development tools that bring newer and often better ways to maintain custom solutions.  Also, when scope changes are requested during mid development, that will have an immediate cost to it.  What about adding a new feature to an existing web site so it better supports mobile devices?  This too will increase the debt due to rewriting the system to support multiple modalities unless it was designed with that in the first place.

In some cases, like the last example, some unavoidable technical debt is really more naïve technical debt and can be avoided, but the third type of technical debt, strategic, is also avoidable, but instead decided (either intentionally or unintentionally) to be accrued.

For instance, skipping proper quality code practices like code reviews and refactoring in order to make a release by a certain deadline is an example of strategic debt.  Continually placing refactoring tasks at the bottom of a product backlog or nowhere at all is another example of strategic debt.  Strategic debt can be tackled by determining an actual financial cost of the technical debt and presenting to leadership in order to get buy in.  This will be discussed further in future blogs.

Technical Debt Explained

This will be first in a series of blogs.

Technical debt is an interesting term that has been formed in the IT industry in order to have non technical businesses understand the cost of building code in a manner that causes higher maintenance and increased barriers to changing.

Technical debt is intended to have a financial consequence in it’s meaning to recognize it costs businesses tangible money, even though due to it’s complexity, technical debt is difficult to measure and hence difficult for non-technical business leaders to understand.

Technical debt if left unchecked, can make a company that depends on technology drag themselves to a complete standstill and often bring down businesses to liquidation.  Technical debt will increase exponentially over time until it becomes unmaintainable.  It’s happened before, even to the early days of the “killer app” Visicalc that was to be superseded by Lotus 1-2-3 and eventually Microsoft Excel largely due to technical debt.  Only through regular code maintenance (via code reviews, refactoring) can technical debt be controlled over the long run.

​I will cover types of technical debt and what they do in the next blog.

Weatherman and Forecaster

Good project managers (and Scrum Masters) consider themselves like project “weathermen” — they report the current project conditions and forecast the future as accurately as possible.

One important aspect of a weatherman (or weatherwoman) is to escalate quickly and effectively.  The project manager should have a “sixth sense” or intuition when a project is going down a bad path and this isn’t an easy skill to learn.  It truly comes from experience with trial and error.  The key as a Project Manager is to be bold and don’t be afraid to inquire on anything that appears to be going too slow and not clear on progress.  Look at my blog on “The Importance Visibility” for a way to help quickly identify issues.

​However, sometimes only through being the “Larry King” of questioners can a PM dig up all potential issues and address them before they become critical.

Does Empathy Matter in the Workplace?

It’s very common in software engineering to recognize there is a right way to do things and then there is everything else.  Often, that “everything else” is the wrong way to do it.

For instance, using no design pattern or an incorrect design pattern (i.e. a behavior pattern for a concurrency problem) would be “everything else”.

Typically, a team will do things for a very specific and logical reasons.  A Scrum Master should not immediately assume a team member is “doing it wrong”, instead keep an open mind and determine the situation and logic behind what was chosen.  For example, have members of the delivery team hear the reason during a breakout session from a daily scrum.  Then have the team decide if the reasoning is acceptable, keeping empathy and “trying on the other person’s shoes” helps foster trust and transparency.  If they are doing it wrong, at least they can find out why and come to an agreement.

As a servant leader, the Scrum Master should promote and encourage empathy within the team, but at the same time recognize that there is always room for improvement.  There is a balancing act that should be made for picking where to make the most improvements and that can be identified, organized and prioritized during retrospectives.

Keep in mind that not assuming someone is “doing it wrong” without empathy even if they really are doing it wrong.  Give them the benefit of the doubt and show empathy when showing mistakes or oversights to others.  This will build team productivity and as a result, increase velocity and team value.

User Stories Expanded

This is a continuation of my July 16th blog on User Stories.

User stories in the simplest form is documentation for Agile delivery.  However, User stories are not necessarily requirements.  There is a fine line between a user story and a requirement and difficult to often distinguish.

A user story is different than the “must have” requirement in that although it provides the instructions on who, what and why something is being built, a user story should be a transparent conversation between the product owner or user/Business Analyst and the delivery team.

It follows the “Card,Conversation,Confirmation” model brought forward by Kent Beck when Extreme Programming was intorduced into the IT world.  User Stories should focus more on making sure the two groups understand each other on the delivery and less on say following a template or having a well written document.  The conversation should lead to building the right solution (at least the right solution understood at that time) instead of just following instructions in a document like a recipe.​  When you create something new, recipes are meant for documenting what has been made, but not the acting the actual process of getting to that recipe.​

As a final point, the relationship between the Product Owner, the team and Scrum Master must be built on trust and frequent communication in order to form and evolve excellent user stories.  Otherwise, user stories will miss key information and lead to unnecessary rework.

The Importance of Visibility

At one of my former employers, they were implementing a huge engineering project (estimated costs in the billions) and brought me in to lead the IT project oversight for the adoption of software to support the Reliability and Maintenance (R&M) of specialized oil tankers once placed into commission.​

At the onset, the overall R&M Project Manager believed this extra level IT oversight wasn’t necessary.  He said his team of SME’s (Subject Matter Experts) could handle the entire adoption and quality validation of the third party software.  Fortunately, I had the Global Head of IT supporting my position, so my position stayed in place.

Using Kanban based techniques, I was able to make the software testing process visible to the R&M Project Manager and his team.  When first receiving the software, my team of QA specialists quickly discovered and exposed hundreds of product issues that the R&M Project Manager and his team of non-technical experts missed.  These weren’t just small usability issues, but complete show stoppers where the software would block any further progress of a specific workflow or even crash.

After going through an intense resetting of expectations and through continued daily visualization of issues and related tasks with the vendor team, we were able to build out a realistic schedule and deliver to it with a high quality solution, now placed in practice.

The key here is to quickly expose and visualize your work in a manner that everyone can see.  This can lead to agreement rapidly to further expand in delivery.

Building Good User Stories

In developing accurate requirements and derive accurate estimates from client engagements, user stories are increasingly becoming the de facto standard for documenting.

Simple in concept, but difficult to master, user stories have this basic format:

As a <actor/user>, I can/want <what> so that <why>

The “how” is kept out, since that is what the delivery team should figure out.  You shouldn’t need as a client know how the car was built as much as how to properly run and maintain it.

However, the trouble can be with getting the right level of granularity to the user stories so no “big surprises” come out weeks or months after the initial requirements gathering process.  There is an art to recognize this and doesn’t follow any template or “silver bullet”.  There are many ways to help define user stories and going to the appropriate level of detail should include recognizing asking questions even in the details for say highlighting key facts of a product.  It can be done in one form of highlighting but upon further research, there can be multiple ways with a “wish list” or favorite, commonly chosen favorites by others with similiar tastes or similiar products.​

That’s when the devil is in the details and has to be handled through many questions and sufficient planning.

Teams Today and In the Past

Modern day development teams can learn a lot from our history.  For instance, the Three (or Four) Musketeers:  One for all and all for one!

​This brings the scrum model of a unified team into principle that everyone is here together and the team in indivisible, i.e. together in one unit.  Teams should practice “gemba” or swarming when a project runs into an impediment and the project manager (or scrum master, if involved) should be right there assisting the team however is needed.