Development Madness during Upgrade

[#1]

Hello World,

please hear me out before demanding to drive a stake through my heart.

Because I know very well that latest during step "Preprocessing" of the upgrade process (EHP5ERP6 -> EHP7ERP6) *no* development should take place, except absolute emergencies.

Problem was, when we designed the upgrade strategy for our systems 4-6 weeks ago, I "forgot" to narrow down the definition of "emergency" to our developers to the bare essentials - believing the term "emergency" was self-explanatory 😞

Now they come to me almost twice a day with another "let's push this one one in quickly, too ..."

By now the system is done with Preprocessing and we await the maintenance window for step "Execution" - which unfortunately is still two days in the making.

And guess what...they'd like for me to import one more "emergency transport" into the system.

Its a 'Y' report, so the logic goes, "its in the customer name space, so what could possibly happen?"

My big worry is, that *if* something happens, the Production system will be down!

No SE38, no STMS, no SE80 and worst of all, no Prod system and the whole factory is waiting for this thing to come back online - and yes, ours is a 24x7 operation.

Has anyone got any experience with pushing in transports, within the customer namespace, so late in the upgrade process?

PS: I know this whole procedure is a "no, no", but I am one Basis guy here, being outnumbered 5 to 1 by the developer team 😮

[george_credland · Accepted solution]

As a developer I pushed for a change freeze during our upgrade project to avoid this scenario.

Even if from a technical standpoint you can transport the changes through, it also creates uncertainty where differences arise to know whether its a result of the development changes or the version differences between systems. This in turn would result in additonal time and effort to resolve the nature of anything turned up in QA/Production and possibility that it wouldn't be possible to recreate the situation in DEV which is where the code changes need to be made.

Overlapping DEV and Production builds risks commiting to something that you're then stuck to back out of. As you say the risk to the business is very high. Having to restore from back up/snapshot pre-upgrade the final resort.

From experience custom changes can break when transported between versions if they

reference standard SAP function modules/classes. e.g. An SAP function module in the newer version has an additional mandatory parameter that didn't exist previously. It may not even be obvious of the depedendencies as they could be hidden away in includes or indirect calls rather than part of the main piece of code that's being changed.

Part of the problem is the SAP fragmentation of roles and responsibilities. Systems are so complex that people oftern only see their slice of the whole. Business pushes the developers for late changes or fixes, which in turn pushes the BASIS team.

The project team needs representation from all areas so that informed decisions are made, including mix of business representatives, developers/configurers and BASIS. Primary responsibility with the relevant areas, but decisions about the way forward have to be taken by the project team as a whole so that staff are supported rather than played off against one another.

If you set out clearly what the risks are and people still refuse to listen be sure to follow up via email for the record, at least that way they won't be able to deny knowledge and walk away if things do go wrong.

Hope you find a way forward for the future.

George

[Former Member · Accepted solution]

From experience custom changes can break when transported between versions if they
reference standard SAP function modules/classes. e.g. An SAP function module in the newer version has an additional mandatory parameter that didn't exist previously. It may not even be obvious of the depedendencies as they could be hidden away in includes or indirect calls rather than part of the main piece of code that's being changed.

Before I started out on this "(ad)venture", they had *promised* me (I just got hired for the upgrade), that "we made no changes to SAP originals", "all our changes were made within the customer name range".

Guess what happened next?

On all systems (even PRD), SUM complained about development leftovers in SAP originals.

Inactive changes, open transport and repair requests and so on ...

Turned out, many of those were the result of repairs or interdependencies between Z* objects and SAP originals.

In my experience, those promises of "we didn't change any SAP originals", are next to useless - until proven otherwise.

For now I'm safe, let's see how long it lasts - hope the stuff that already went in the system beforehand won't break anything during Execution or in Postprocessing.

One of our developers came running in today, because he just found out that he needed to change an internal structure for the month end jobs to succeed.

How absurd is that?

6 weeks of preparation and now he finds out???

BTW: Wasn't my idea either to have the month end jobs coincide with the System Upgrade 😮

[george_credland · Accepted solution]

One of our developers came running in today, because he just found out that he needed to change an internal structure for the month end jobs to succeed.
How absurd is that?
6 weeks of preparation and now he finds out???

Could it also be a problem with their connection to the business teams? How much exposure do they get to see and understand the processes, or do they get specific requirements handed down to them?

BTW: Wasn't my idea either to have the month end jobs coincide with the System Upgrade :-O

Probably not their idea either to be fair. This is why the project team needs representation from all areas. There may be business requirements that constrain when things need to happen. e.g. Monthly filing of tax returns in GB payroll.

Have you tried arranging a meeting with key people from the various areas to talk through these concerns? It may be that the developers are also being pushed to make the changes in the same manner that you're being pushed to let them through.

You've identified and problem so now you need to own it and take the first steps to do something about it. Start from the point that everyone is working to get the changes in place as quickly as they possibly can. More understanding and less blame is needed otherwise everyone will dig in on their own terms and you'll get nowhere.

[Former Member · Accepted solution]

The developers sit right next door from me 😉

Everybody was involved from day one, its just that when Wednesday came they no longer cared about what they had agreed to on Monday 😉

There is truly not much any of us can do about this, even the devs only react(ed) to outside pressures.

The crazy think is, even the shop floor here is hooked up to SAP, so if Prod breaks *nothing* works anymore.

But because of everyone being hooked up to it, users all over the place keep making demands about "fix this right now, or else we can't get the job done".

I understand the user's problem - that is why I had suggested a parallel development line at the beginning.

But were are on such a tight budget here, but still everybody's keeping the pressure on to get it all done on time, so somewhere, something had to give.

In our case it was the dual development landscape, which would have saved us from many of these problems,

I don't think the stakeholders "out there" appreciate how much of their data may be lost if this thing breaks.

I mean, we're in more than 5 days of upgrade now, so doing a full blown restore to a point in time *before* isn't really an option any more.

But how do you tell someone who oversees an entire production line and complains "w/o this change I can't continue", that he's just outta luck?

Everybody should have put in his (final) change request weeks ago, but they didn't - and there are people you don't say "no" to w/o consequence.

[Former Member · Accepted solution]

Well, looks like luck is on my side 😉

I can see in the log files of DEV that its going into generating the ABAP loads now.

So it'll be finished by the time the switch for PROD is due.

If anyone then still wants to have any changes made, at least we'll have a finished DEV system to test/fix any problems.

[Former Member · Accepted solution]

I'll mark this thread as "solved" at this point, as I think the worst is behind us now.

Thanks for all the helpful insights, and it does help if I can point out, "others think this is a bad idea, too" 😉