Which KPI's do you have in software development?

[EnnoWulff]

Hey Community!

What KPI'd do you have in your development process?

which KPI's might be sensible in order to get development "to the next level"?

• Number of unit tests -> increasing
• Code coverage percentage
• ATC-Checks (which checks?)
  • no Prio 1 messages 
  • decreasing number of messages
• Estimated time vs actual effort

abaplint provides the following KPI's:

• Average Complexity
• Lines vs unit tests
• Average method length

To have these key figures you need to upload the progress for all objects in scope into github. 

How to define the scope?

How to retrieve KPI's automatically?

Thanks for your experiences and suggestions!

~Enno

[holm]

Sonograph - That has been a predecessor or sister product of Sonargraph. Seems to be dismissed…

https://www.hello2morrow.com

They had an ABAP plugin to analyse also SAP systems, not sure if this is still available in Sonargraph. 

[holm]

Hej Enno,

Very generic question & nice topic for a long Stammtisch discussion 😀

First, what’s to be measured? Performance of the team vs. SW Quality.

Of course that relates somehow, but increasing SWQ is only a part of the teams, well, rating, and the success of your organisation. You know probably Goodhart’s Law, and thats why I am always sceptical when it comes to team/individual KPIs.

So let’s only look at SWQ for now - tech lead perspective. As an engineering manager I will probably look also from an different angle. And so on.

Currently our approach is simple:

1. AU is always green. If not (not a drama), fix until next day. If there are big refactorings which breaks things for longer, consider feature toggles (anyway a good idea).
2. ATC = 0/0/*.  Same as above, fix asap.

Everything else (Team velocity, burndown rates etc.pp.) is a different story. Here measuring starts to distort reality even faster. Skip this…

Now for the details. A good ATC variant is key - fine tuning the checks is a special kind of art. That’s a long discussion for itself.

But apart from single findings it’s a good radar, indicating future problem zones. If there is much noise in a certain edge of my system, I know where to look closer at. If I do not, we will see more problems later exactly there.

Number of tests and coverage is - from my experience - not key. They are not an end in itself. If you take them as KPIs, the team will start to write hundreds of meaningless tests 😉 But if you know your system well, e.g. 30% coverage - placed around the critical functionality - is better than 80% missing them. So you have to go thru the AU coverage statistics and judge by yourself. Unfortunately (or luckily?) that cannot be expressed as a KPI.

The lint KPIs etc. may be interesting to see whether tidy up works. Looking at those trends is helpful, do we have the right direction? Also to evaluate quality of a system you are new to.

Famous last words: keep it simple, don’t overengineer measurements, it won’t pay off. Its only one aspect. Trust your senior devs, they know the truth 😎

Happy to hear more opinions,

Holm

[EnnoWulff]

Thanks for your valuable comment, @holm !

It's a risk, that key figures are used in the wrong way:

And yes, there are many aspects like you already mentioned (team performance, SW quality).

We are forced to have KPI's due to ISO9001 to make sure that we are getting better. But we want to have KPI's that really help us to improve.

ATC-Checks with zero Prio-1-messages can be one key figure but you cannot tell if you are getting better...

And yes: there might be useless or unnecessary unit tests if you get "karma points" for having many, but this could be controlled by code reviews at the end.

It's also difficult because we are not working on one product or in one system but we have lots of customers. 

[holm]

Hi @EnnoWulff ,

i see… yo, that’s a challenge. What about measuring certain KPIs at start of each customer project and at end and measure the improvements project/customer wise?

And ranges to keep are also an idea when it comes to process definitions.

PS: Dilbert has always the best answers 🤓

[fwe1]

I would like to add another question to that topic 😉 Do you measure/apply any KPI for your Software Architecture (e.g. Fitness Function, Modularity Score, Metrics based on tools like ArchUnit, etc.)?

[holm]

Yes, if your development is not just a report, but something larger that’s absolutely worth a try. We did a full sotography of our product some years ago, and I can only recommend this. Especially to check whether my intended architecture is still that what we actually have. 
In this respect I also recommend https://www.langlebige-softwarearchitekturen.de

[EnnoWulff]

@holm What is a sotography?