Feedback on a small-scale BW project using Scrum - Part 1

[franois_gyngysi]

Introduction

A few months ago I got slightly frustrated with the way we handled an ECC / BW project for our Credit Control & Accounts Receivable departments. Although we delivered the goods, I felt the process was protracted. Several groups were involved: the end-users from two departments, the business analysts' team (in charge of capturing the business requirements), the ECC team, the ABAP development team and the BW team (I am in charge of the latter).

Basically, there were a lot of documentation and long formal exchanges between the various teams. Even with plenty of meetings, this led to misunderstandings that were only discovered very late in the process. I decided to see if there was a way of improving the process and ended up looking at agile methodologies and more particularly at Scrum.

Manifesto for Agile Software Development

In order to really understand what agile practitioners mean by agile, it is worth reading the Agile Manifesto below (available at http://agilemanifesto.org/😞

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Principles behind the Agile Manifesto

We follow these principles:

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Business people and developers must work together daily throughout the project.
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity--the art of maximizing the amount of work not done--is essential.
• The best architectures, requirements, and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

What is Scrum?

If you want a really quick overview, go to http://www.scrumalliance.org/pages/what_is_scrum. A good Introduction to Scrum is available at http://www.mountaingoatsoftware.com/system/asset/file/58/RedistributableIntroToScrum.ppt (the document is available in various languages at http://www.mountaingoatsoftware.com/presentations/30-an-overview-of-scrum). If you want a very detailed explanation (more than one hundred pages!), please read http://www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf.

I read a lot of stuff on the Internet but still had plenty of questions. Just to name a few: how to plan the overall project? How to manage the user requirements? How to apply Scrum to a SAP BW project? Would Scrum work with a data warehouse project? How to manage a team with part-time people? How to move forward? How to choose a project? Etc.

I found the following three books very useful to answer my questions:

• Cohn, Mike Succeeding with Agile
• Cohn, Mike Agile Estimating and Planning
• Aubry, Claude Scrum (this book is in French)

Moving forward

Convinced that Scrum was worth trying, I put a few PowerPoint slides together (I am a manager after all ;-)) to explain the value of Scrum. On the first slide, I explained agile software development and listed the values and principles of the Agile Manifesto. I then added and repackaged the slides from the Introduction to Scrum presentation listed above (not forgetting due credits!). I also added slides on release planning as I felt this was important.

I exchanged a lot with my fellow IT managers to get feedback and criticisms about my presentation and Scrum itself. This led to very interesting and constructive discussions and forced me to further investigate some points.

Choosing the right project

I also thought that our next SAP BW project was the right candidate to use Scrum. As shown by Claude Aubry in his book Scrum, I prepared the graph below to prove the point. The closer to the centre, the easier it is to apply Scrum (or the more relevant Scrum is).

Here is a quick explanation of each dimension:

• Team Size: the best size for a Scrum team is 7 +/- 2 people (Scrum can scale by having multiple Scrum teams). We were pretty good here because I estimated our team would be made of 5 people.
• Engineering Capability: although our BW team is experienced and pretty good at its job, I considered that we had little experience with software engineering practices like test-driven development or test automation. I actually didn't know how we could implement these practices with SAP BW!
• Geographical Dispersion: I thought it would not be difficult to have our business analyst and end-users in the same office as the BW team. So we were pretty good here.
• Level of change: Scrum is very good at handling changing requirements and shifting priorities. As I didn't expect a lot of changes I scored "low" on this dimension (it doesn't mean that Scrum is not useful if the level of change is low).
• Criticality (in case of failure): although important to our business people, our BW project was not critical (i.e. we are not talking about building software for a nuclear power plant). If the project is critical, you may have to provide a lot more documentation and be a lot more formal and you will therefore benefit less from Scrum.
• Deployment: we only have one instance of SAP BW so this is less risky than making an application available for download to millions of Internet users.
• System Age: the older the system, the more difficult (or impossible) it is to use software engineering practices like test-driven development or test automation.
• Architecture Stability: our BW architecture is stable so we were good on this dimension (no risk).
• Economic Model: we scored low on this dimension because I knew we would only use internal resources on our SAP BW project. On the other hand, it could be more difficult if you have to negotiate a fixed price Scrum contract with a supplier and you've never done that before (there are actually some ways of doing it).
• Governance: I scored complex on this dimension because we typically have steering committees, core team meetings, and so on, to manage our projects and my experience was that each team could be defensive about its prerogatives.

Coming out

I gave the presentation to senior management on the IT and business sides and got very positive response. I found the values and principles of the Agile Manifesto to be of real interest to senior management. Also, everybody felt safe about applying Scrum to the chosen SAP BW project.

I then presented Scrum to the BW team. We got into really interesting and sometimes intense discussions! For instance, what amount of design should we do up front. As Mike Cohn says "the difference on a Scrum project is not that intentional design is thrown out, but that it's done incrementally". Was this going to work on SAP BW project? Another interesting discussion took place regarding user stories (the index cards or Post-It used to capture business requirements). Would that be enough to drive the development?

Finally, I presented Scrum to more stakeholders (managers, business analysts, etc.). Everybody was ready to give it a go. We decided to launch the project. This will be covered in my next blog.

Afterword

Should you have any questions on Agile/Scrum/Lean, do not hesitate to contact the Agile+ community on Google+ at http://plus.google.com/u/0/104017139996657183972/posts. Your questions will be forwared to the community if needed.