- SAP Managed Tags:
Commitment to quality is one of the core values that SAP as an organization strongly backs and believes in. contemplated for decades, dedication to delivering products scoring high on quality and usability underpins everything we do at SAP.
Easier said than done, there's a fair amount of challenges we come across in the journey to brace the vision of anchoring core business values seamlessly into a quality product. this often requires us to endeavor innovative solutions to ensure products are on par with the standards at all times.
One such attempt towards continually improvising on the process to meet the bars of modern cloud-native software development discipline was the inclusion of the Daily Delivery Process Model into the SAP Cloud Application lifecycle Management.
The SAP Cloud ALM is a brand-new cloud-based application lifecycle management offering. It is perfectly tailored for cloud customers and manages hybrid-cloud solutions. This newly designed offering is built on modern cloud technology and is cost-efficient for customers.
Key features:
SAP Cloud ALM is one of the most advanced micro-service-based SaaS applications on SAP CP Cloud Foundry(and probably also beyond). This includes:
SAP Cloud ALM needs to support the life-cycle of the Intelligent Enterprise Suite (IES). So SAP Cloud ALM has to have the fastest innovation lifecycle of all IES applications (SAP Cloud ALM needs to be ready whenever any other IES service is updated).
That means from a business perspective SAP Cloud ALM needs to be able to deliver daily.
The motivation with Cloud Native mindset was not only to be able to deliver continually but also to be able to deliver a high-quality product.
And the solution was simple, innovating for a product promotion mechanism via several tenants eventually into the production landscape and incorporation of the right tools and frameworks to ease up the process keeping quality in check at all times. One such framework is "Deploy With Confidence".
Deploy with Confidence (DwC)
An End-To-End SaaS Framework covering Project Processes, Development Best Practices and providing Tools that will Boost your project quality and efficiency
Modern cloud-native microservice-based applications are inherently distributed, introducing immense challenges. Deploy With Confidence (DwC) helps dramatically increase developer productivity, ensure high quality while delivering daily.
DwC incorporates tools such as JaaS, Piper, Cloud SDK, webdriver.io, sauce labs, local test environment support, mutual TLS, and many more to automate daily deliveries of multi-microservice SaaS applications. Focus is SAP Cloud Platform Cloud Foundry and in a second step also Kubernetes. Incorporated concepts include immutable feature toggle configurations, dynamic routing, true-zero downtime for every individual user session to ensure a robust lifecycle without disruption.
Feature Toggling: immutable feature toggle configurations to develop new features with ease and deliver them to the customers with high quality
The central qualifier of the solution for delivery is the End-To-End automation tests which therefore has the highest priority and used as the underlying baseline for the release decision. The E2E test covers the business-critical* path, depending on what is defined concerning bringing the most value to a customer.
Our Motivation: Successful E2E test runs are crucial for our daily delivery. The earlier failures in the E2E test runs are detected, the earlier they can be address and fixed.
Test Harness
During daily delivery, there might come a situation where the E2E script for a functional scenario fails due to the unavailability of test data.
For example, in the case of integration monitoring scripts, it requires data to be present from systems like Cloud Platform Integration, Cloud for Customer, etc. But there may be no data available for that particular day or the last couple of weeks, so this situation creates an ideal scenario where we require a mechanism which might require us to be able to create and push the required data into integration monitoring service, to facilitate the execution of our script. Similarly, Alert & Event Management scripts require alerts to work on, and it may happen that they don’t have alerts in the system which might result in the script failure.
Being able to push this data would ensure the E2E running green across tenants and failure at least because of the lack of test data. this would make the E2E tests a lot more reliable which is a must as they are one of the most important and a central qualifier of the solution for delivery.
So, the automated test data injection acts as a key to enhancing the reliability of E2E Tests.
Following are the capabilities of Test Data Injection Mechanism currently being followed:
If E2E Scripts Fail?
Since E2E Tests validate the scenarios for any given feature to work as expected which also means any failure could mean a possible bug/issue in that particular feature. this quickly demands one round of manual testing to validate the bug. in such events:
Includes the following tools and methodologies
Absolutely Yes!
Engineering a product that's simple to consume, instantaneously available, and always up-to-date requires us to go above and beyond. it demands innovation around the entire lifecycle of the development process. Be it planning, developing, testing, or delivering.
And SAP Cloud ALM is undoubtedly one of the most advanced micro-service-based SaaS solutions out there to have successfully incorporated all the pieces to building a high-quality solution with a smooth daily delivery process in place.
Contributors:
mdafsar.hussain, Anurag Singh
Easier said than done, there's a fair amount of challenges we come across in the journey to brace the vision of anchoring core business values seamlessly into a quality product. this often requires us to endeavor innovative solutions to ensure products are on par with the standards at all times.
One such attempt towards continually improvising on the process to meet the bars of modern cloud-native software development discipline was the inclusion of the Daily Delivery Process Model into the SAP Cloud Application lifecycle Management.
SAP Cloud ALM
The SAP Cloud ALM is a brand-new cloud-based application lifecycle management offering. It is perfectly tailored for cloud customers and manages hybrid-cloud solutions. This newly designed offering is built on modern cloud technology and is cost-efficient for customers.
Key features:
- Simple to consume, instantaneously available, and always up-to-date
- Support for the entire lifecycle of cloud solutions from SAP, from build to run
- Integration with the implementation experience of cloud products from SAP
- Built for continuous delivery of cloud solutions from SAP
- Integration and business process monitoring included
- Usage rights included with SAP Enterprise Support, cloud editions
Innovation lifecycle
SAP Cloud ALM is one of the most advanced micro-service-based SaaS applications on SAP CP Cloud Foundry(and probably also beyond). This includes:
- the cloud-native level of our services,
- the size/complexity/amount of services
- the amount of needed scalability (≥ 50.000 tenants), number of Engineers (≈ 200), number of development locations (> 6).
SAP Cloud ALM needs to support the life-cycle of the Intelligent Enterprise Suite (IES). So SAP Cloud ALM has to have the fastest innovation lifecycle of all IES applications (SAP Cloud ALM needs to be ready whenever any other IES service is updated).
That means from a business perspective SAP Cloud ALM needs to be able to deliver daily.
How SAP Cloud ALM Keeps Quality Amidst Daily Delivery in Check?
The motivation with Cloud Native mindset was not only to be able to deliver continually but also to be able to deliver a high-quality product.
And the solution was simple, innovating for a product promotion mechanism via several tenants eventually into the production landscape and incorporation of the right tools and frameworks to ease up the process keeping quality in check at all times. One such framework is "Deploy With Confidence".
Deploy with Confidence (DwC)
Develop, Deploy, Deliver Cloud Applications Daily and with Confidence
An End-To-End SaaS Framework covering Project Processes, Development Best Practices and providing Tools that will Boost your project quality and efficiency
Modern cloud-native microservice-based applications are inherently distributed, introducing immense challenges. Deploy With Confidence (DwC) helps dramatically increase developer productivity, ensure high quality while delivering daily.
DwC incorporates tools such as JaaS, Piper, Cloud SDK, webdriver.io, sauce labs, local test environment support, mutual TLS, and many more to automate daily deliveries of multi-microservice SaaS applications. Focus is SAP Cloud Platform Cloud Foundry and in a second step also Kubernetes. Incorporated concepts include immutable feature toggle configurations, dynamic routing, true-zero downtime for every individual user session to ensure a robust lifecycle without disruption.
Feature Toggling: immutable feature toggle configurations to develop new features with ease and deliver them to the customers with high quality
End-To-End (E2E) Automation Testing
The central qualifier of the solution for delivery is the End-To-End automation tests which therefore has the highest priority and used as the underlying baseline for the release decision. The E2E test covers the business-critical* path, depending on what is defined concerning bringing the most value to a customer.
Our Motivation: Successful E2E test runs are crucial for our daily delivery. The earlier failures in the E2E test runs are detected, the earlier they can be address and fixed.
Tools & Frameworks for E2E:
- WebdriverIO is used as an automation tool. written in NodeJS, It's Front-end Engineer friendly, feature-rich, and streamlines the functional test writing process.
- Cucumber framework for writing Behaviour-Driven Development(BDD) Tests.
Cucumber reads executable specifications written in a plain text called Feature files and validates that the software does what those specifications say. - Sauce Labs is used as an automation platform. It provides all kinds of operating systems and browsers to run the tests on (e.g. Windows 10, Chrome 76, 1024x768). the Sauce Labs virtual machine cloud (desktop web and em/simulators) and can update the job metadata ('name', 'passed', 'tags', 'public', 'build', 'custom-data') and runs Sauce Connect if desired.
- Amalthea tool used for DwC for accessing all stages and their tenants, triggering E2E tests across tenants and performing Daily Delivery.
Test Harness
Test Data Availability Availablity For E2E
During daily delivery, there might come a situation where the E2E script for a functional scenario fails due to the unavailability of test data.
For example, in the case of integration monitoring scripts, it requires data to be present from systems like Cloud Platform Integration, Cloud for Customer, etc. But there may be no data available for that particular day or the last couple of weeks, so this situation creates an ideal scenario where we require a mechanism which might require us to be able to create and push the required data into integration monitoring service, to facilitate the execution of our script. Similarly, Alert & Event Management scripts require alerts to work on, and it may happen that they don’t have alerts in the system which might result in the script failure.
Being able to push this data would ensure the E2E running green across tenants and failure at least because of the lack of test data. this would make the E2E tests a lot more reliable which is a must as they are one of the most important and a central qualifier of the solution for delivery.
So, the automated test data injection acts as a key to enhancing the reliability of E2E Tests.
Following are the capabilities of Test Data Injection Mechanism currently being followed:
- It can create test data.
- It can push test data to a system.
- We can configure data parameters before we push the data.
- Data generation could be set up for multiple test tenants.
If E2E Scripts Fail?
Since E2E Tests validate the scenarios for any given feature to work as expected which also means any failure could mean a possible bug/issue in that particular feature. this quickly demands one round of manual testing to validate the bug. in such events:
- The Daily Delivery is stopped
- The failed E2E functionality is tested manually and the DD Team is updated about the gravity of failure and if DD for the application/feature can continue.
- The teams fix the error with the highest priority otherwise the Daily Patch Delivery for the day cannot go through.
SAP SAP Cloud ALM Quality Process In A Nutshell
Test Strategy
- As SAP Cloud ALM Build aligns to develop behavior-driven our main goal is to test features.
- The feature is defined as a set of scenarios in a feature file mapped to a user story in JIRA.
- The decision which method to use depends on the size and scope of a feature, e.g. how can one achieve the best requirement coverage and stability for the given user-story.
- We are aiming at replacing the manual testing efforts with a 100% automated quality checks.
Includes the following tools and methodologies
- E2E UI Automation Tests: using Webdriver.io & tools mentioned above.
- Unit Tests: JUnit, QUnit tests to be run automatically.
- Integration Test: Vertical integration test (UI+Backend+Database+Mocked horizontal layer)
- Code Coverage Checks: JaCoCo (part of the pipeline)
- Frontend test Code Coverage ≧ 70%
- Backend test Code Coverage ≧ 70%
- API Testing: REST Assured using TestNG framework
- Static Code Checks performed at the end of each takt. Quality engineers review the results and take action when applicable.
Delivery Process
- Feature Delivery
- A Feature will create value for the customer and should be delivered in 2-4 weeks.
- Define your Feature as easy and small as possible w/o dependencies
- Daily Patch Delivery
- Ensures all the E2E tests are green confirming the application works as expected.
- After that, we start the promotion process to get the current state of application available in the required tenants
Is It Worth?
Absolutely Yes!
Engineering a product that's simple to consume, instantaneously available, and always up-to-date requires us to go above and beyond. it demands innovation around the entire lifecycle of the development process. Be it planning, developing, testing, or delivering.
And SAP Cloud ALM is undoubtedly one of the most advanced micro-service-based SaaS solutions out there to have successfully incorporated all the pieces to building a high-quality solution with a smooth daily delivery process in place.
Contributors:
mdafsar.hussain, Anurag Singh
1 Comment
