An often-asked question is how to use Multitarget Applications (MTAs) in the context of µServices-based architectures. In this blog, I show that the relation of MTAs to µServices depends on the concrete setup of development and deployment processes as well as how team responsibilities are assigned. There is a related blog which collates µServices and MTAs based on the definitions given by Martin Fowler et al.
For illustration, this blog restricts to the Cloud Foundry environment of SAP Cloud Platform as a target platform, but everything transfers to the Neo environment and SAP HANA XS Advanced (on-premise) as well.
The basic Cloud Foundry deployment unit is a "CF-app". It is "pushed" to the platform, which results in the creation of a container with a suitable environment to run the CF-app artifacts. There is little metadata attached to a CF-app. More metadata would be helpful, however, to evolve and track an app through development and continuous integration/deployment, including quality management and staging processes. As an apparently trivial example, there is no well-defined ID and version of a CF-app. People are encoding all this somehow into the CF-app name, which in addition must carry the burden of encoding blue/green deployment status.
There are three main reasons (listed below), why application owners and developers are looking for appropriate metadata and tools based on this set of metadata. This lack is compensated by building a parallel code world of scripts for orchestrating and configuring CF-app deployments.
These three reasons are:
All these three aspects are covered by a declarative application model (the MTA model) that is persisted in descriptor files. For more information about MTA, see the document The Multitarget Application Model - A guide to understand multi-target applications.
A common misunderstanding is that an MTA would only make sense, if it would contain multiple components that get deployed as one unit. This composition aspect is only covered by item 1.) above. It makes perfect sense to describe single container apps via an MTA model to take advantage of items 2.) and 3.) above. Although the discussion below focus on the composition aspect of MTAs, don't miss to consider MTAs as a means to develop and deploy even single CF-apps.
For the purpose of this document, we distinguish between development, assembly and deployment processes.
We are looking at the processes from development to deployment. As shown in the figure below, we are distinguishing three phases that lead to the creation of one deployable unit. Such a unit will be deployed using either a single cf push or a single cf deploy command, depending on whether MTAs are involved or not:
Figure: Process from development to deployment
The three vertical scenarios in the figure above represent different ways how these three phases are realized:
There is a fourth scenario (not shown), where the last two phases ASSEMBLE+DEPLOY are fused, because there is only one owner or tool which executed both phases.
The discussion above shows that it is required to distinguish the unit of development from the unit of deployment:
For DEV+ASSEMBLE+DEPLOY, these units do not coincide. The least controversial interpretation then is probably the following: If each unit of development corresponds to one CF-app, then this unit/app should be called µService. However, there is an additional constraint, defined by the release/product architect. Although each µService could in principle evolve independently from the others, it was decided to co-deploy a group of services. This can represent the case ‘one MTA – many µServices’, if the deployment service is able to implement a selective restart strategy for parts of an MTA.
For cloud scenarios, we do not recommend to implement such a process, because it increases the likelihood to create inconsistencies and it detaches developers from deployment artifacts. The DEV+ASSEMBLE+DEPLOY process is typically applied for on-premise delivery of big packages of software.
With this blog, I hope you better understand how we see the relation of MTAs to µServices and how it depends on the concrete setup of development and deployment processes as well as how responsibilities are assigned. In addition, we discussed a set of patterns that shall serve as a guidance, if and how to introduce MTAs in a way to best match your application development and deployment processes.
For illustration, this blog restricts to the Cloud Foundry environment of SAP Cloud Platform as a target platform, but everything transfers to the Neo environment and SAP HANA XS Advanced (on-premise) as well.
Brief Recap of Multitarget Applications (MTAs)
The basic Cloud Foundry deployment unit is a "CF-app". It is "pushed" to the platform, which results in the creation of a container with a suitable environment to run the CF-app artifacts. There is little metadata attached to a CF-app. More metadata would be helpful, however, to evolve and track an app through development and continuous integration/deployment, including quality management and staging processes. As an apparently trivial example, there is no well-defined ID and version of a CF-app. People are encoding all this somehow into the CF-app name, which in addition must carry the burden of encoding blue/green deployment status.
There are three main reasons (listed below), why application owners and developers are looking for appropriate metadata and tools based on this set of metadata. This lack is compensated by building a parallel code world of scripts for orchestrating and configuring CF-app deployments.
These three reasons are:
- Applications are composed of multiple parts:The adequate level at which an application is "managed" (i.e., developed, tested, deployed, updated, operated, monitored, etc.) is not necessarily the level of CF-apps. Although each CF-app might correspond to a micro-service, it is still of value to have a notion of the complete, distributed application to develop, manage and monitor it as a whole.CF-apps that constitute a larger application might even not qualify for being micro-services. Although pushed to an individual container, they might lack the essential micro-services characteristics of being independent components. This happens, if they don't expose stable, public API contracts. Sometimes, it is just not worth the effort to create such contracts, because the individual CF-apps might not be created as a result of a domain-driven design approach, but simply for enabling independent scaling of application parts.Another common pattern is to break down the application into a frontend part (acting as the initial user request processing stage) and a backend part, each implemented with adequate technologies (e.g. node.js vs. Java). Both parts can be tightly coupled, as new backend features need to be reflected on the frontend. A pure compatible evolution of backend APIs is a big overhead, especially for immature products.
- Applications have dependencies: Applications at best self-describe their runtime and deployment-time dependencies (we are not talking about build-time dependencies here). Such dependencies can be required service instances or APIs from other applications.If an application would come with such a description, tools could verify the existence of required resources, can allocate missing resources, and bind the application to the required resources. Today, there are some aspects of this when pushing via an application manifest (yml). Important aspects are missing, like service instance creation, and it is hard to share one manifest within a team of developers, as each developer will want to maintain individual variants for testing, which can easily break consistency among these variants. Also, native Cloud Foundry has no notion of inter-app dependencies.
- Multiplicity of deployments: For our purpose, let's define "deployment configuration data" as those things that end up in environment variables of a CF-app. It's part of the application design to define which variables are used for which purpose.The reason to introduce configuration at all is to enable multiple, differently behaving deployments of an application without changing application code. Applications aren't global singletons. The usage of DEV-spaces vs. PROD-spaces, multiple data centers, and the increasing use of private clouds for large enterprises all lead to a multiplicity of deployments.For a deployer persona, it should be as easy and automated as possible to create a deployment variant. This becomes simpler, if an application declares its configuration variables, their characteristics (e.g. optionality, default values), and expectations from where to get this value. Imagine an application composed of two micro-services S1 and S2. Why should a human care about the URL required by S2 for addressing S1? Based on adequate metadata, a deploy tool can create "any" route to S1 and inject it into the environment of S2.Another aspect of multiple deployments is that it leads to the requirement of application packaging. Only in some cases, it is possible to deploy "from source", e.g., based on a Git-URL. In general, deployables are the result of build and testing. A distribution archive format is required to move deployables through firewalls. Enterprise policies restrict the distribution of software to the process of deploying signed archives only as part of auditable processes.
All these three aspects are covered by a declarative application model (the MTA model) that is persisted in descriptor files. For more information about MTA, see the document The Multitarget Application Model - A guide to understand multi-target applications.
A common misunderstanding is that an MTA would only make sense, if it would contain multiple components that get deployed as one unit. This composition aspect is only covered by item 1.) above. It makes perfect sense to describe single container apps via an MTA model to take advantage of items 2.) and 3.) above. Although the discussion below focus on the composition aspect of MTAs, don't miss to consider MTAs as a means to develop and deploy even single CF-apps.
DEV-to-DEPLOY Scenarios
For the purpose of this document, we distinguish between development, assembly and deployment processes.
We are looking at the processes from development to deployment. As shown in the figure below, we are distinguishing three phases that lead to the creation of one deployable unit. Such a unit will be deployed using either a single cf push or a single cf deploy command, depending on whether MTAs are involved or not:
- DEV: A team of people having a developer role assigned is working on source code using IDEs, version control and build tools.
- ASSEMBLE: More concretely, this refers to building an MTA archive. Someone with the role of a release/product architect decides what is contained in the archive.
- DEPLOY: Deployment to production. Someone with the role of a space developer will push or deploy artifacts to a Cloud Foundry space.
Figure: Process from development to deployment
The three vertical scenarios in the figure above represent different ways how these three phases are realized:
- DEVOPS scenario: The developer or development team is taking all relevant roles (resp. some are taken by a continuous integration toolset). There can be a continuous pipeline from developer commit to deploy. This scenario can be realized either with or without MTA involvement:
- The case without MTA corresponds to a “native” CF-based development with ymland using cf push command; the process is not interesting for our considerations here, hence we omit it.
- If MTAs are used, the MTA model is represented by the MTA development descriptor, yaml, which is part of development sources. MTA archives are either created interactively by SAP Web IDE, or by the MTA build tool. The granularity of the development project/Git repository matches the granularity of the unit of deployment. This means, all sources in a Git repository are built and packaged into one MTA archive (the unit of deployment).
- DEV+DEPLOY scenario: This is similar to the DEVOPS scenario, but here a process break happens after having built the MTA archive. One possible reason for having the packaging step in the middle is a need for the entire application to be distributed to support multiple deployments. A dedicated person taking the deployer role will fetch the archive and deploy it to the space she is responsible for. Another reason for this process break might be a Q-gate implemented as an acceptance/integration test performed prior to a productive deployment.Also for this scenario, if MTAs are used, the MTA model is represented by the MTA development descriptor, yaml, which is part of development sources. Again, MTA archives are either created interactively by SAP Web IDE, or by the MTA build tool. The granularity of the development project/Git repository again matches the granularity of the unit of deployment.
- DEV+ASSEMBLE+DEPLOY scenario: This scenario is different from the former two, in that it starts with a "native" (no MTA involvement) development phase. This is followed by an MTA assembly and deploy phase with the purpose of creating an archive for distribution scenarios. The point is that the granularity of the development project/Git repository does not match the granularity of the unit of deployment! There can be multiple development projects/Git repositories contributing their output to one MTA archive. It is in the responsibility of a release/product architect to create an MTA descriptor, fetch all required artifacts and to create the MTA archive.
There is a fourth scenario (not shown), where the last two phases ASSEMBLE+DEPLOY are fused, because there is only one owner or tool which executed both phases.
Where is the µService?
The discussion above shows that it is required to distinguish the unit of development from the unit of deployment:
- For DEVOPS and DEV+DEPLOY, these units coincide. Therefore, the relation to µServices can be done unambiguously - this unit is the µService. If "native" development is done (without MTAs), it will end up as one CF-app (or a set of CF-apps that have to be manually wired - which might be ok for the pure DEVOPS scenario without multiple deployments from the same project). If MTAs are used, the whole MTA qualifies to be the µService, because it is related to one team, one development project, and one deployment unit. It can end up in multiple CF-apps, but the deploy service makes sure that these apps are managed as one unit. So, it represents the case ‘one MTA - one µService’.
For DEV+ASSEMBLE+DEPLOY, these units do not coincide. The least controversial interpretation then is probably the following: If each unit of development corresponds to one CF-app, then this unit/app should be called µService. However, there is an additional constraint, defined by the release/product architect. Although each µService could in principle evolve independently from the others, it was decided to co-deploy a group of services. This can represent the case ‘one MTA – many µServices’, if the deployment service is able to implement a selective restart strategy for parts of an MTA.
For cloud scenarios, we do not recommend to implement such a process, because it increases the likelihood to create inconsistencies and it detaches developers from deployment artifacts. The DEV+ASSEMBLE+DEPLOY process is typically applied for on-premise delivery of big packages of software.
With this blog, I hope you better understand how we see the relation of MTAs to µServices and how it depends on the concrete setup of development and deployment processes as well as how responsibilities are assigned. In addition, we discussed a set of patterns that shall serve as a guidance, if and how to introduce MTAs in a way to best match your application development and deployment processes.
- SAP Managed Tags:
