On some questions and positioning of the openSAP course "Writing testable code"

[juergenheymann]

Hi everyone

We are currently running the openSAP course “Writing testable code for ABAP” that focusses on unit testing with ABAP with the patterns and tools needed. And we are happy to see that several people are blogging about it on SCN. There were some valid questions raised. Let me clarify what we meant and provide some more background.

Disclaimer: I admit that I am one of the authors of the course. I hope that doesn’t disqualify my opinions but of course, you always should be a little more careful with the ‘sales people’ of a topic ?. However, I was also the lead of the Agile Software Engineering program at SAP where for several years we have trained over 6000 developers and gained a lot of experience from many projects about what works well and what doesn’t.

 

On the positioning

• We are not saying “You have not tested before or not done it right”! But are there not things to learn, some things to do better? This is similar to talking about e.g. ‘Clean Code’ (book by Bob Martin) or Object Orientation or design patterns. They are not the answer to everything, do not solve all problems. But they are a step forward and professional developers should know these things deeply.


• Similarly, of course ABAP Unit is not the next big thing or even anything new at all. We rather say that ABAP provides pretty much the same as all other languages (especially the new ones): reasonably good unit test support. And we are saying that it is worth it (more on that later).


• And of course, we want to show how to use the tool and the show some key patterns that go with it. The patterns and design guidance is what you can find in pretty much any other book / source on the topic.


• The other main reason we did this course is simply customer demand. The problems of real software projects are very often: very little automated tests / coverage, lots of manual regression testing needed (high cost, slow), long cycles of upgrades / transports, too many bugs, too much maintenance effort, …  We have lots of experience internally and even from customer cases that there are ways to be in much better shape.

 

Content questions

So, let’s now talk about some content questions.

• Is Unit Testing valid and important? There should be no doubt here. Enough has been written about that (Kent Beck, Michael Feathers, Gerard Meszaros, Robert C Martin, Martin Fowler, …). It simply a necessary skill and we don’t want to make another argument for something obvious. There are many other prominent companies that have a very strong test automation culture, e.g. Microsoft, Google, facebook, amazon, …


• Is TDD overstated / overhyped? Now that one is more interesting. This topic is certainly controversial. There was a nice public controversial discussion summarized by Martin Fowler in “Is TDD Dead?” We agree with the summary result of that discussion which basically states “TDD helps in some cases but is not useful in all cases”. And of course there is debate about which cases belong where. When you look at it in detail you see that there are actually two aspects: “When does unit testing make sense?” and “When does TDD make sense?”. And we all have to accept that the judgement is partly a matter of personal taste – which is perfectly fine. These are my personal thoughts and trade-offs:
  
  
  
  • A lot depends on doing unit testing and TDD at the right place, i.e. at the right API level. I would not do it for trivial code (e.g. structure mapping) and certainly not for private methods. There should be some behavior complexity that is worth safeguarding with a test. Otherwise ‘go a level higher’.
  
  
  • We never push for ‘100 % TDD’ and certainly not for ‘100% unit level code coverage’.
  
  
  • Practicing TDD seems to be a matter of strong personal taste. When you write the unit test ‘right after’ then you should get most of the benefits as well.
  
  
  • There are other ways to get high quality in special cases. When you have to make changes in critically important legacy code and there is not isolation for testability in the code, I would always go for pair programming rather than spending all the effort to make that little change covered with a unit test.
  
  
  • So, we are not dogmatic about it. BUT … there is a big BUT: I think that the skills and patterns is something that every professional developer must know! It is not valid to reject something outright without own experience.
  
  
  
  


• Is this applicable to real life product code? YES! The examples in the course are necessarily small but we have other internal courses that work in the belly of a large application to teach using the patterns (and some more) in legacy code. Our experience (with hundreds of teams) that you can learn these things best when developing new code / features and later tackle the difficult cases of legacy code restructuring. The basic patterns of adding test isolation to legacy code is included in this course at the end but for more you have to consider e.g. the book by Michael Feathers ‘Working Effectively with Legacy Code’.

 

Our experience at SAP

Some of the good stories where teams integrated these techniques into their work habits:

• One product area (100 developers) reduced their defect rate and maintenance load by 5x.


• Testers found only 30% of the number of bugs that from previous year and much less severity.


• Moving tests from system to unit level caused the same test scope to run > 1000 times faster.


• One team did a downport of their application in 2 hours because they had full coverage of automated tests. They just had to fix 2 bugs and were done. Another parallel team that had no tests worked with 10 people for a whole week.


• A new team member became productive after 1 week. Because the product was highly covered with tests, the new team member could add things with the confidence not to break existing functionality.


• … and more …

Of course there are also problems in that in some cases the effort for unit testing is prohibitive. This is especially the case for legacy code (= 'code without tests' according to the definition of Meszaros). In that case we usually go for component level (several packages) isolation and testing.

 

To sum it up …

• It is totally obvious that without good unit level test automation you cannot do agile development in short cycles. Teaching how to do this in ABAP was the focus of the course. I don’t know of an alternative here and all the famous people seem to not know either … ?


• The question of ‘test-first’ is incremental after truly adopting unit testing with isolation and local testability as a design feature. The difficult part is unit testing, isolation patterns, refactoring etc. especially in the context of legacy code. Once you have mastered that, you can take the step to ‘test-first’ if you want at any time.

Regards

Juergen Heymann