Best Practices for NWDI: Track design for ongoing ...

former_member189311 · ‎2006 Mar 30

Introduction

This paper contains recommendations for track design for NWDI-based development. It describes a track design for ongoing development where the release cycle is very short. Development groups need to develop applications that are shipped or deployed to a production system within a short software life cycle. This means ongoing developments and improvements to shipped software are performed in parallel, and the new changes are shipped in a short time frame, e.g. once a week. In this situation, the release cycle is very short and the shipment or deployment to production systems is done immediately after test and approval. The released software is not a new version or release; it is, more or less, a continuous improvement of existing functionality. This article assumes familiarity with the NWDI, especially how to set up and maintain tracks and runtime systems within the CMS WebUI.

Limitations in a Single Track Landscape

In most cases, a single track is sufficient for this scenario. During development of the next version in DEV, the CONS system is used for urgent bug fixes to the released version that is deployed in the production system. As soon as the new version is imported into the CONS system, there is no longer any chance to fix bugs in the current production version. The reason for this is that the track template does not include workspaces and buildspaces for TEST and PROD systems. If a bug occurs in the production system, the only way to fix it is in the CONS system and to ship it together with the new version to the PROD system.

Maintenance Track Approach

The requirement to fix bugs at any time cannot be fulfilled within a single track because there are time frames where the CONS system is blocked with the new version. But the new version is not ready for deployment to the PROD system. This means you require a second track, which is usually called a maintenance track. The maintenance scenario is described in the weblog Best Practices for Running the NWDI). Usually bugs are fixed within the Maintenance track and there within the DEV development configuration. With a repair connection the fixed bug is automatically progagated after the developer releases his changes to the Development track into the import queue of the DEV system. This feature is designed to avoid double maintenance and to make sure that all fixed bugs are automatically integrated into further developments.

Usually the scenario described here for creating a maintenance track for the shipped release is our recommended scenario. Here, the software component is deployed to the production runtime system after it has been transported through the maintenance track. There is no PROD runtime system in the development track. But in some cases, as in the case described above where the release cycle is very short, the transport through a maintenance track is too much overhead and takes too long. In a maintenance track, where buildspaces and workspaces are created to be able to maintain a release, the software is of course built again in DEV and CONS, and must be assembled again. It is obvious that this is overhead for the scenario described above, where bugs are normally fixed in the CONS system of the development track. One way to speed up the deployment process would be the following scenario, where the PROD runtime system is added to the development track to deploy the new changes immediately after the approval step. The same PROD runtime system is also added to the maintenance track. Only in the rare case where a bug cannot be fixed in the CONS system of the development track, must the version deployed to production be imported into the maintenance track in order to fix the bug. In this case, the fixed version of the software component is assembled in the maintenance track and deployed to the common PROD runtime system. In all other cases, the maintenance track is not used. In general, this scenario is not recommended because it can lead to situations where older versions overwrite new versions during deployment, as described in the following example.

Example:

Version 1.0 is developed in the development track and shipped/deployed to the production runtime system. There is a track connection of type Transport between the development track and the maintenance track. This means Version 1.0 is propagated to the import queue of the DEV system in the maintenance track. In the meantime, the development of Version 1.1 is ongoing in the development track. An error occurs in the PROD system in Version 1.0. This bug can now be fixed in the maintenance track. After the bug is fixed and tested it is transported through the configured PROD system (in the maintenance track) to the PROD runtime system. In this case, everything is fine and the production runtime system now contains Version 1.0 BF1 (Version 1.0 including bug fix 1), which hopefully solved the problem. Now, the new version 1.1 is ready and deployed to the production system (using the PROD system of the development track). Again, an error occurs in the production version 1.1 in the production runtime system. The bug must be fixed in the maintenance track, because V1.2 is already deployed to the TEST system in the development track. Here, a problem can occur when the administrator forgets to import Version 1.1 into the DEV system of the maintenance track. Therefore, it could happen that the developer fixes the bug in an older version which is then deployed to the production runtime system. In our example, Version 1.0 BF2 (Version 1.0 including the bug fix 2) overwrites Version 1.1.

If a runtime system is attached to multiple tracks this may lead to the problem that an older version overwrites a newer version already deployed in the production runtime system. The following deployment sequence is possible: 1) Version 1.0 2) Version 1.0 including bug fix 1 3) Version 1.1 4) Version 1.0 including bug fix 2 Neither the deployment tool SDM, which is triggered using CMS, checks the versions, nor is this done by CMS itself. CMS can only handle the version sequence properly if a runtime system is attached to exactly one system of a track.

Recommended Track Design for Ongoing Development

One way of avoiding that an old bug/version comes back again and again is to make sure that the import queue of the DEV system of the maintenance track is empty. Every time a bug has to be fixed, the entries in the import queue must be imported successfully before the developer starts to fix the bug. This is an organizational and process-driven solution; there is no tool support. A better way to solve this issue (and our recommended solution) is as follows: CMS can check the versions to avoid such a scenario. But this means the track design looks a little different. The following track design describes a way of handling the rapid deployment of new changes and immediate fixes to bugs. But this case is very rare, so there is no general requirement to transport source code through the maintenance track every time new changes are deployed. The proposed solution is based on the maintenance pattern, but the development and maintenance tracks have no configured PROD systems. This means there is no direct deployment into the PROD runtime system. The deployment to the PROD runtime system is performed using a separate PROD track, as shown in the next image.

The PROD track only consists of the runtime system PROD. There are no DTR workspaces and buildspaces configured in this track. How to setup such a track is described in the NWDI documentation. Track connections are used to propagate the approved versions of the software from the development track, as well as from the maintenance track to the import queue of the PROD system of the PROD track. Within the PROD track configuration the software component is added as required software component. The advantage in this scenario is that CMS does not allow the deployment of an older version when a newer version is already deployed. This is possible because CMS can compare versions within one import queue even if they are assembled in different tracks. So, the overwriting issue is solved. Please keep in mind that the standard maintenance scenario is still valid here. Which means, the import queue of the DEV system in the Maintenance track has to be empty before a bug can be fixed. And the back transport is used to integrate the fixed bug. There the entries in the import queue of the DEV system in the Development track has to be imported as well, before a new version is going to be approved. For the example described above in this scenario, the CMS checks the versions before deployment and you not at risk of overwriting newer versions. The following sequence is therefore no longer possible: 1) Version 1.0 2) Version 1.0 including bug fix 1 (OK) 3) Version 1.1 (ok) 4) Version 1.0 including bug fix 2 (is blocked) For the ongoing development scenario, the recommended track design combines the advantages of a single track (fast life cycle) with the advantages of a maintenance track (bug fixes at any time), while reducing the overhead of a typical maintenance landscape.

Former Member · ‎2006 Apr 12

Hi Marion,

First I have to compliment you on the blog, it's just what we need!

We have implemented the proposed development scenario by using a DEV track, Prod track and correction track. We have decided not to define runtime systems in the correction track, because that might result in deployment of older versions on development and test runtime systems.
We have done some tests by deploying a component to production and then make a correction on it.

These are our experiences:

1. On each track we see an approval stage, so on the development track we need to approve after the test stage, and on the productive track we need to approve again before importing to the runtime system.
Also on the correction track we need to approve before going to production.

2. On the production track we see a development stage eventhough we don't have a runtime development system defined (the dev stage seems obsolete)

3. When we assemble in the correction track we cannot choose/change the SP version or Patch version number, therefore it's very hard to determine which correction fix is deployed on the production runtime system (might be a bug!?).

Kind regards,

Pascal Willemsen and Richard Plukker
www.Yellow2B.com

former_member189311 · ‎2006 Apr 13

Hi,
thanks for your feedback.
1. You are right, every track has an Approval tab that you can't omit.
2. With the current track template, there are technicaly reasons why the DEV tab can't be omitted. But the import is very fast because no build and deploy steps are performed.
3. It is a known issue with the current NWDI version that the SP level can only be set in the first track of a landscape. It's planned to solve this problem with one of the next support packages in NW2004s.
Best regards,
Marion

Former Member · ‎2007 Jan 31

I'm a bit late getting to this party, so hopefully there's still people about to answer this question 🙂

We're currently constructing our NWDI landscape and track structure. I'm planning on implementing the track structure that you describe above - it seems to be the best structure.

However, I have a question regarding the PROD track. As the software moves through the landscape with new versions being created, does it make sense to create new 'deployment' production tracks, or should we just reuse the production track? My feeling is that a single production track could be used - when you decide to deploy a new version of your code to the production environment, you would remove the track connection from the previous track version, and create a track connection from the new version to the production version. This way, you could ensure that only the correct version of the software would be deployed to your production server.

I suppose you could remove the runtime system from the PROD track, but the fact that the PROD track is still there could be confusing. You would also have three tracks for each version, as opposed to one production track, and two tracks for each version.

Opinions?

former_member189311 · ‎2007 Jan 31

Usually you have just one production track and would reuse it even if new versions of your applications will be deployed. You need to setup a new track for deployments to production server only, if you maintain multiple production runtime systems that run different versions of your developed applications.

Former Member · ‎2007 Feb 01

As I thought Marion. Thanks for verifying that for me.

Joseph_BERTHE · ‎2007 Nov 14

Hi,

First, your blog was very helpfull, good job 🙂

Secondly, according to your blog, we have to make three tracks for one software component (more or less). That's greate but we have plenty of futur applications and we wonder how to make an usefull naming convention for track ID and track name ?

Kind regards.

Joseph BERTHE (Inte6)

former_member189311 · ‎2007 Nov 22

Hi,
thanks for your feedback. To make it clear, you can have more than one software component within one track. But you need at least two tracks: DEV and MAINTENANCE. For a short life-cycle, which means frequent updates on the PROD system, a three track design is recommended. Unfortunately we have no naming convention paper to which I can point you. But you should use a 3-4 letter appreviation for your track ID and add for example a D, M, P for Development, Maintenance and/or Production to that ID. Furthermore maybe a release or SP level number can be added, in case you need to support several development states in parallel (as we as SAP do).
Best regards,
Marion

Former Member · ‎2007 Dec 08

First of all, excellent weblog.. I am relatively new to NWDI, so it took me 3-4 readings over a period of time to make perfect sense of the blog and now it seems so beautiful.. :)..

Thanks a lot for this..

Former Member · ‎2008 Apr 18

Dear Marion,

Thanks so much for your blog. It was really very informative. I just needed some clarification.
We are on JDI 7 SP12 on HPUX 11.23. We need a small clarification on our NWDI track as we are currently stuck with 2 requirements wrt to NWDI.

Presently, We were having a 4 systems SAP CRM landscape. CRM Dev(CED), CRM IST (CEI), CRM Q (CEQ) & CRM P (CEP).

We have the JDI instance connected to the CRM system and all the 4 systems were configured as the run time systems in the JDI. In the track which we are using,in the Runtime systems tab, the Development runtime system points to the CED instace, the Consolidation runtime system points to the CEI box, the test runtime system points to the CEQ box and the production runtime system points to the CEP box.

Clarification for Requirement1: Now we have added a new tier for the User Acceptance testing called CRU there by coming into 5 system landscape (CED,CEI,CEU,CEQ,CEP). We now need to add this system in the JDI instance in the same track and configure it in the runtime system tab and link it to the CEU box.

We have planned to define a new track with the prod instance defined in the Prod run time system & define the CEQ system in the 1st track's prod run time system and do the track connection between them. We need to do this urgently as we have the new U-tier ready. Is this approach fine?

Clarification for Requirement2: In the present scenario with the 4 system landscape, we have another parellel issue same as per your blog. We have 2 Developers , one from the productive support team for bug fixes and other from the new development team. Once both of them release a request from NWDS. The requests comes in the consolidation queue and then I individually assemble them (in this way the request gets applied to CED & CEI) one by one to the test queue. Now, I have these 2 requests assembled(with 3 components each, SAP-SHRAPP, SAP-SHRWEB, SAP-CUSTCRM in our case-so total 6 components) in the test queue. But when I start importing one component,say, SAP-SHRAPP component, of the first request, the other request logically becomes outdated and all the changes are combined into this one SHRAPP request.Similarly when I start importing the SAP-SHRWEB component of the first request, the other becomes outdated. But here the 2nd developer (new development) does not want his request to go to test system and further, but the 1st developer (to fix a bug) wants to the request to P system ASAP.

We have developers who work for production system for bug fixes and other who work for the project work. We have only one track and are in the process of configuring the 2nd track to add the 5th CRM system.

In order to takle requirement1 & requirement2 as per your blog, we are planning to create a maintenance track (for bug fixes) along with the present dev track (new developement)and then link the test run time system of the dev track to the dev runtime(??) system of the maintenance track & the back track the dev runtime system of the maintanence track to the dev runtime system(??) of the dev track.
In both the tracks we plan to define CRM Dev (CED), CRM I (CEI), CRM U (CEU) as the runtime systems for Dev, Consolidation, & test and then have a separate track for the CRM Q instance (CEQ) & and the CRM Prod instance(CEP) and do a track connection between them. Is this approach fine or please let us know the correct and the proper approach. what are the pre-cautions we need to ensure here.

Kindly help us here. Thanks again in advance.
Warm Regards
Rajeet.

former_member189311 · ‎2008 Apr 18

Dear Rajeet,
thanks for your feedback. For the use case where you have developer working on a current project and others working on the productive version, you definitely need two tracks. The track design should be like described in the blog. With every track you have the need to add more runtime systems accordingly. That means your idea to reuse the systems CED, CEI and CEU in both tracks does not work. You will overwrite the changes and can not test the fixed version or the new version. One runtime system has to be connected to one specific role (Dev for current development or Dev for Bug fixes). Therfore you need at least one more runtime system, to give the bug fix developer the opportunity to test. For example: Define the DEV track with CRM Dev (CED), CRM I (CEI) and CRM U (CEU). Define the Prod track with the systems CEQ and CEP. And for the Maintenance Track define a Dev Runtime system only. Please do not use one runtime system in serveral tracks in case they are used to test the same software component in different versions. That does not work as you can only run one version at a time.
Best regards,
Marion

Former Member · ‎2008 Apr 23

Dear Marion,

Thanks a lot for your prompt reply.
As per your reponse, we plan to go ahead with the following design to incorporate the new CRM 5th tier environment as well as to ensure that development & transports for both the new release & the bug fixes co-exist. Can you please confirm whether our understanding is correct or not:

We are planning to create 3 tracks in total.
There would be a 2 master tracks, Development track & a maintenance track.

The Dev track would have the CRM Dev (CED), CRM IST (CEI), CRM UAT (CEU) defined in the Dev, Cons & the Test Runtime systems respectively.

The Maintenance Track would have just one run time system. That is the CRM Q (CEQ) system defined in the Dev Runtime System.

A Track connection (type repair) would be made from the test system of the Dev Track to the Dev system of the Maintenance track.

Similarly a back track connection would be made from the Dev runtime system of the maintenance track to the Dev runtime system of the Dev Track.

Then we would create a 3rd Track with only the CRM Prod (CEP) instance defined in the Prod runtime system. This track is connected to the Maintanence Track and a track connection of type transport is created between these 2 tracks. No track connection is made between the Dev Track and Production Track.

With this approach, all the bug fixes, can be fixed with the Maintanence track & parallely the new development can happen in the Dev Track.

We also believe that this would solve our issue during upgrade of our CRM landscape as we can use the Maintanence track for issues and parallelly the other systems in the landscape would get upgraded.

Thanks in advance for your inputs.
Warm Regards
Rajeet

Former Member · ‎2008 Apr 30

Hi Marion,

Hope you are doing well.
Can you please be so kind to help us out in the below issue.
Thanks a lot in advance.

Warm Regards
Rajeet

former_member189311 · ‎2008 May 09

Hi Rajeet,
it is not completely right. You have two possibilities as described in the blog. The first one - which is the standard maintenance scenario with two tracks. One for development (CED, CEI, CEU) and one for maintenance (CEQ and CEP). Here you have the possibility to fix bugs in the development configuration of your CEQ system and test is there. Here you have a track connection of type transport from the DEV track to the maintenance track and a track connection of type repair from the MAINT track to the DEV track (see first picture in the blog).
Second option is the last picture in the blog. Three tracks: one DEV track with (CED, CEI, CEU), one MAINT track (CEQ) and one pure production track PROD (no development configurations, no DTR workspace, no CBS buildspace) (CEP). A track connection of type transport from DEV to PROD and from DEV to MAINT and from MAINT to PROD. One transport connection of type repair from MAINT to DEV. I would recommend to have the CEU system as TEST system in the PROD track instead. There you can test all changes (also the bug fix changes) before they are imported to CEP.
Best regards,
Marion

Former Member · ‎2008 May 14

Hi Marion,

Thanks so much for your reply. We shall adapt the settings as per your recommendation on the existing landscape

Just another help. Right now, we have another requirement, where in apart from the existing CRM 5.0 Systems, we need to add a separate 5 Tier Landscape of CRM 7.0 Systems (FRD, FRI, FRU, FRQ & FRP) in the same NWDI instance.

Do we adapt the same approach for new CRM systems and make 3 separate tracks (One for Dev, one for Maint & one for Prod). So, with the 3 systems, we would be having in total 6 tracks. This means we will have the same web applications available in different versions.

If this is not the right approach, can you please be so kind to suggest the best practice.
Also, is there any inter dependency here (namespace, versioning, etc.)? Is there anything we should be careful of to ensure smooth functioning of NWDI & Java Transports across all the landscape with different web applications catering to different business units?

Thanks a lot for your support.
Good Day.

Warm Regards
Rajeet

Former Member · ‎2008 Jul 14

Hi,

I am a developer working in NWDI. We have everything in place and working fine upto now.

SAP has release a patch for the software component i am using in a track. Basis guys have updated the track and they confirmed the patch is updated successfully.

Now when ever i rebuild my DC, the changes are not effected as per the new patch. Please let me how I will be able to use new patch updates in my DC(In NWDS I can see the old source code only, I did all SYNC's and refresh but didnt worked, even NWDI is restarted)

How I can confirm that the patch is updated to track successfully, from a developer point of view.

I have access to CMS and CBS.

Regards,
Sateesh Chandra

former_member189311 · ‎2008 Jul 14

Hi Sateesh,
the basis team has checked in and imported the new version of the software components into the track? Go to the system state tab in CMS of your track and check which version is actually imported. Furthermore you can have a look at the history details of the development tab.
Remove the DC from the client and remove the development configuration and reimport it into your Developer Studio. Recreate all your projects. The new version should be synched.

Best regards,
Marion

Former Member · ‎2008 Nov 11

Hi Marion,
We have an upgrade of ESS on EP6 from SP 17 to SP 23. As per the JDI cookbook, I have created 2 tracks, 1 for the original SC and 1 for modifying the upgraded SC. Please let me know how I can use Track Connections in this case to update directly into the Quality system. Thanks in Advance.
Raghav

Former Member · ‎2009 Jan 05

Hi,

Your design sounds valid - but:

What if you develop components for different plattforms (i.E. a software component A contains Webdynpro JAVA for J2EE-system and PI-DC's for PI systems that are not on the same platform)

Former Member · ‎2009 Jan 13

Hi Marion,

Thank you very much for this excellent blog. This was indeed helpful in designing our JAVA track!

In reference to previous communication from my colleauge, Rajeet, we decided to have the second option, i.e., CED, CEI, CEU in the Dev track, CEQ in the maint track and the CRP in the 3rd track. But then, we had a conflict between the code in the ABAP systems and the JAVA systems.

Let me explain the same. Our current ABAP track is CED to CEI and finally to a transport group with CEQ, CEU and CEP. Considering this ABAP track and the recommended JAVA track, the ABAP development is carried out on the CED system and the JAVA development also in the CED system. This code (ABAP+JAVA) can be tested on the CED (B2B Webshop)and released to the further system. This approach is fine.

But coming to the bugfix scenario, we face a conflict. The ABAP bugfixes are to be carried out in CED, whereas the JAVA bugfixes are to be carried out in the CEQ system. In this case, the testing is not possible as the ABAP and the JAVA codes are in disparate systems.

Transporting the ABAP changes to CEQ repeatedly for testing purposes is not practical. And if the ABAP bugfixes are carried out in the CEQ system, we will not have the code changes in the CED system (according to our current ABAP transport route).

Could you kindly help us out in this issue? How are the transport routes designed in this scenario where development and bugfixes run parallely? Any pointers would be greatly appreciated.

Thanks & Regards,
Savan D.

Former Member · ‎2009 Jun 09

Hi Marion,

Thank you for your weblogs. They are very informative. I have few questions regarding our landscape here. We have different kinds of developments going on here (Webdynpro development, Visual composer etc). We had NWDI running on 6.40 earlier and now we have completed the upgrade to the NWDI on 7.0. We now need to move all the SCs to the new environment. Before that, I want to get your suggestion in creating tracks for our requirement here. Could you please let me know, if it is required to have separate tracks for different type of developments - meaning one track for Visual composer, one track for Webdynpro development etc?

could you please provide us your recommendations on this?

Thanks.
Sunitha.

Former Member · ‎2009 Dec 28

Hi Marion!
Congratulations for your article. It is very well written!

I'm new to NWDI, and i'm currently lost on the systems keyword. I've seen that i can configure systems on the following places:

-TMS (ABAP or non-ABAP systems).
-SLD (Technical systems, Business systems).
-CMS (Runtime systems).
-Solution Manager (synchronization of SLD or TMS systems).

Now, i can't understand what is the purpose of all these definitions. I've seen how to define transport routes on TMS, but i can't understand how it is related to the tracks transport routes, and where the techincal systems enters in this game.

Can you clear the things to me please?

Thank you

former_member189311 · ‎2010 May 26

Hi Nuno,
sorry for the delay, but I was not in the office the last year.
There is a tight integration of TMS/CTS+ and NWDI. With EHP1 for NW 7.0 the new CM Services are introduced where you define all technical Systems in TMS with transport routes and the Development Configurations (Source Control and Build Support) in CM Services. Here the CMS of NWDI is not needed anymore. Before Enhancement Package 1 for NW 7.0 the integration takes place during the assembly step where the assembled SCA file is attached to a CTS transport request. Here you define technical systems for DEV and CONS in CMS and all other systems of your transport route in TMS. The SLD is needed in all cases to register the development configurations and manage the software component definitions.
Hope that clears the things a little bit.
Best regards,
Marion

Former Member · ‎2010 Nov 08

Hi

We will like to implement this track strategy in our current landscape. We are faced with a situation where-in we only have 3 systems: DEV, TST and PRD.

So, 2 tracks will be created both having runtime systems : DEV & TST. And separate production track: PRD.

We run into following issues:
1. During bug fixing Maintenance SCA overwriting Development SCA.
2. Both bugfixes and development changes cannot be tested simultaneously.

Our releases are not as frequent as mentioned by you. Considering the situation here, what is the best course to implement this track strategy?

Thanks in advance

Regards,
Srikant

Former Member · ‎2011 Jan 18

Hi,

Does the tighter integration of NWDI with CTS+ in NW 7 EHP1 impact the above recommendation for track design?

If NWDI is only used for DTR and CBS, while deployment is done through CTS+, are there any maintenance scenarios e.g bug fix which we can't address?

Regards,

Shehryar

Former Member · ‎2011 Jan 19

Also, if we wanted to use CM Services rather than CMS, are they any limitations in terms of maintaining multiple versions of same development components across Project and Maintenance landscapes?

Thanks

former_member189311 · ‎2011 Jan 19

Hi Shehryar,
for the integration scenario with CMS and CTS+ there is no difference and the described maintenance scenarios are supported.
For the usage of CM Services we will publish a How To Guide with some best practices on how to work with a 5-system landscape etc. Unfortunately I can't promise a date when it will be available.
Best regards,
Marion

Former Member · ‎2011 Jan 20

Hi,

Thanks for a quick response. Much appreciated.

It is mentioned in the blog that since CMS compares versions before deployment, old versions of the software component will not be deployed. But what happens when CMS is not used for deployment. Rather, CTS+ is configured for deployment? And what about Runtime systems? If CTS+ is deploying, do we need to link runtime system e.g. CE Prod to track system PRD?

For a brief background: we are using NWDI (DTR, CBS) for CE based development of master data management processes. We are linking the tracks with CTS+. For each master data object, like customer, master, vendor, etc a separate track is being created. However, there is also a track for common components. So the common components are built first, then copied into all other master data object tracks.

We'll have two-system tracks, DEV and CONS. DEV will not have a runtime system. Initial deployment to DEV will be directly from NWDS. CONS will point to Runtime CE DEV system to test SCA build before moving to QA through CTS+.

At assembly, the SCA will be attached to a CTS+ transport which will deploy the SCAs to downstream systems. Since CTS+ will be used for deployment, I wanted to confirm if the suggested track design is relevant in this case, and how will it address cases like the ones you have elaborated in the blog.

Thanks and regards,

Shehryar