Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
Cloud Foundry Summits are the premier events where the Cloud Foundry community gets together to talk about current developments in the Cloud Foundry ecosystem. Topics range from internal developments to visions of the cloud native future. The event occurs two times every year, once in Europe and once in North America. Cloud Foundry Summit Europe 2018 took place at October 10-11 in Basel, Switzerland. The slogan was "Running at scale", which is one of the major promises of Cloud Foundry. It was present in multiple talks throughout the conference, focusing on operating huge installations of Cloud Foundry.
We ourselves are contributing to the project on a daily basis in the BOSH Europe Team at SAP. BOSH is one of the core projects of Cloud Foundry, managing the lifecycle of many deployments in the ecosystem, like the Cloud Foundry Platform itself. To us as contributors to the Cloud Foundry project, Cloud Foundry Summit is also a great chance to meet other people in the community. It is very useful for us to see what other teams are focusing on - in BOSH as well as the whole community. Also, it's a big opportunity to meet all people we primarily know as Github or Slack nicks and discuss presented topics as well as what challenges they face similarly to us and how they can be overcome.
At Cloud Foundry Summit Europe 2018, more then 900 attendees joined to see what's new, connect with customers and visit all kinds of talks. The talks were organized in the 7 different tracks "Cloud Foundry in the Enterprise", "Core Project Updates", "Experiments + Extensions", "Operations + Security", "Containers + Serverless", and "Wild Card". In addition to the tracks, there were key notes and lightning talks with both technical topics, showing live demos of current projects as well as non-techniqual topics like a great talk on diversity in tech communities by Steve Greenberg.
All talks have been recorded so you can view them online. Following are some of the talks we visited and found interesting as BOSH contributors.
Kube Your Enthusiasm — Bringing the CF Push Experience to Kubernetes Operators (Project Eirini)
In a "standard" Cloud Foundry deployment, your application gets deployed in a VM running a diego cell. This talk by Julian Friedman (@doctor_julz) and Julian Skupnjak(@herrjulz) - both from IBM - is all about running Cloud Foundry applications on Kubernetes. To do so, project Eirini is working on using Kubernetes as a containier scheduler as a replacement for diego. At the end of their talk they show a live demo of what they promise during the session: A cf push to Cloud Foundry, resulting in app containers launched in Kubernetes.
What is the benefit of this? Making Cloud Foundry operations easier by using Kubernetes, while still providing the developer experience of Cloud Foundry.
BPM + Cloud Foundry One Year On
Every BOSH release author, who wants to use BOSH to deploy and manage processes on a deployment VM, has to write code repetitively (e.g. for PID file management, log redirection, process signal handling, process execution and termination, ...). Most of those scripts have simply been copied from other BOSH releases.
The new BOSH Process Manager (bpm) removes a lot of this repetitive boilerplate code.
This talk by James Myers and Aram Price explains how to use bpm, the current status of the project and the plans for the future of bpm.
Bpm allows a declarative process configuration via a bpm.yml file.
By running the configured processes in containers, bpm adds isolation on filesystem and process level. Since BOSH jobs don't see other jobs on the same VM, bpm provides security by default. They can only write to ephemeral and persistent disk, if configured. Standard out and standard error is redirected to log files. Bpm even allows configurable resource limitation for memory, open files and number of processes.
Furthermore, it has a consistent operator interface for BOSH jobs (e.g. 'bpm start/stop', 'bpm logs' to show jobs standard out/error, 'bpm shell' to run a shell inside the job container...).
BOSH uses Monit on Linux VMs to manage and monitor BOSH job processes. Bpm is a layer between Monit and BOSH jobs. It is planned to phase out responsibilities from Monit and replace it with bpm. This will allow to unify the BOSH process management across Linux and Windows operating systems.
It is even conceivable that bpm could be used to run container images natively on BOSH stemcells in the future.
BOSH Package Vendoring and Colocated Errands
Package vendoring provides easy-to-consume, cross-OS and versioned packages for BOSH Releases.
Co-located errands let you run short-running workloads (errands) co-located with other jobs on BOSH deployed VMs. In comparison to tradional errands it simplifies running the errand on a whole instance group and reduces resource consumption.
BOSH Links API
BOSH Links is a concept which allows a release author to share properties between BOSH jobs. One BOSH job provides a property via a so-called link in its release and deployment manifest and another job is able to consume it. BOSH Links is particularly helpful for configuring network communication between jobs. Here the BOSH director becomes responsible for the IP management and provides a consistent way to reference network configuration between jobs. This allows a backing service to easily provide its connection options via a BOSH link to any application job – even if the application job is running in a separate BOSH deployment.
However, the existing approach to BOSH Links also faces some challenges:
The provider of a link does not know its consumers which makes every configuration change a shot in the dark.
BOSH Links doesn’t provide means to visualize job and deployment dependencies.
The consumption of links across different directors is not supported.
There is no built-in propagation of link changes to consumer deployments. Every link consumer has to be re-deployed to discover and consume link changes.
To deal with some of those challenges the BOSH team introduced the Links API. It provides additional HTTP endpoints on the director for managing links, link providers and link consumers.
In the talk “BOSH Links: Bringing Foundations Closer” the BOSH developers Bozhidar Lenchov and Gaurab Dey explain how those endpoints can be utilized to deal with some of the existing challenges like making providers aware of its consumers or notifying consumers about changes. They compile the new possibilities of the Links API into three interesting use cases:
Service Brokers
Link Visualization
Understanding the Impact of Link Configuration Changes
Check out the talk if you are interested in the full potential of the new BOSH Links API.
Autoupgrade BOSH Packages
This talk shows how you can update all the dependencies of your BOSH release automatically. This eliminates the need to bump softare releases manually.
The BOSH Europe team at SAP also has a pipeline (see here) which updates Ruby gems and BOSH packages for the BOSH openstack CPI. It creates a pull request such that a human can still review the changes before they are applied.
The update-packages job is triggered whenever there is a new bosh package version (ruby, rubygems, bundler, libyaml) available. For each package a separate dynamic-metalink-resource checks for new versions.
The update-gems job is triggered periodically. It calls bundle update which updates all gems of the Gemfile ignoring the previously installed gems specified in the Gemfile.lock.
See here for a more detailed explanation of that pipeline.
Making Sense of BOSH Links Through Visualization
This talk by Jamil Shamy (Pivotal) and Saman Alvi (Pivotal) presents a BOSH CLI extension that enables the visualization and categorization of inner and cross deployment Links.
Why would you need this functionality? If you have multiple massive deployments, it is difficult to understand the deployed software. The complexity is caused by properties, variables and links which are shared across deployments. The graphical interface of the presented BOSH CLI extension tool aims to provide a holistic overview of all these parameters and allows to explore them through a filter mechanism. As for now, the extension is still under development and improvements of the visualization through graph libraries is planned.
If you want to learn more about the visualization of BOSH Links, check out the recording of the talk.
Leveraging Istio and Envoy to Deliver New Capabilities in CF
Istio is a completely open source service mesh, which can intelligently manage the in/out traffic and the traffic inside a distributed system. Currently, CF has its own implementation for in/out traffic (the so called north-south traffic) with the routing service. Recently, CF has introduced also the container networking which takes care of the traffic inside CF between the application containers (the so called east-west traffic). The talk by Shannon Coen (@shalako) and Aaron Hurley (@AaronSHurley) starts with the motivation speech about “why” it completely makes sense to integrate Istio into CF. The second part of the talk explains “how” the integration will be implemented and contains a roadmap for it.
This talk will help you to better understand Istio and CF routing and all the new features, which will be available in CF after the integration of Istio. Also, it gives a great example of how a platform like CF, which is so widely used in production can be refactored and a central component like CF routing replaced by Istio.
From Outages to SLOs: Focusing on BOSH Performance
In this talk, Marco Voelz (@voelzmo) speaks about why it is important that BOSH, often conceived as a pure livecycle management tool, performs well. He gives an overview of the challenges our team faced in operating Cloud Foundry at a large scale, using BOSH.
From this talk you can see what we did to measure the performance of BOSH and which hotspots we found in our investigation. Also, you can find out what we did improve the performance of BOSH and, more importantly, how we switched to proactively monitoring BOSH's performance.
The list of talks worth watching contains more than the few outlined above. We really want encourage you to take a look at the youtube playlist mentioned above. More information and pictures from Cloud Foundry Summit Europe 2018 can be found at the event page.
For all the content that cannot be broadcasted via the internet - in-person discussions with presenters, meeting people who contribute to the same project, attending talks and the pub quiz during the Unconference, as well as non-technical activies such as the 5K Fun Run, a run across the place of event - make sure to visit the next Cloud Foundry Summit, which will take place in Philadelphia in April 2019.
