Instance Sharing - SAP Cloud Logging

[Hariharan-Gandhi]

Co-Authored by: @wolfgang_theilmann and @Benny-Goerzig 

Centralizing observability data offers significant advantages by providing a unified view across different organizations, spaces, subaccounts, and even regions.

SAP Cloud Logging provides the ability to "share" a single instance across various application deployment locations. This offers two key benefits:

• It simplifies the centralization of observability data into a single or fewer instances.
• It drastically reduces costs compared to Application Logging, where customers must create a separate instance for each space.

This document outlines different contexts, tools, options, and best practices for instance sharing.

Understanding scopes of sharing and tools

Sharing an instance basically involves (re)using an existing instance from one location and making it available to multiple applications, spaces, or other locations, without having to create a duplicate instance.

There are two primary methods of instance sharing:

Built-in / native sharing

In this approach, the runtime environment (such as Cloud Foundry) or platform (such as BTP) provides APIs to facilitate sharing.

• Advantages:
• Credentials are transparently transferred to the destination.
• The system manages authorization, determining who can share or modify the parent instance.
• The lifecycles of the parent and the shared instances are closely aligned.
• Disadvantages:
• This method limits shareability to a narrower location scope, such as within a single subaccount.

Documentation: CF docs on sharing-instance

Sharing by reusing credentials

Cloud Logging service offers global endpoints, making its APIs accessible from anywhere. To enable access from other locations, you can create a service key from an existing instance and share it with your applications. Cloud Foundry provides the "User-Provided Service Instances" feature to support this approach.

• Advantages:
• Universal applicability across regions, subaccounts, and environments.
• Disadvantages:
• Maintenance considerations, including credential transport and rotation, can require additional effort.

Documentation: User-Provided service instances

Locations of instance sharing

In BTP, instance sharing can happen between different source and destination locations. For this guide, we’ll zoom in on the Cloud Foundry runtime and walk through all the possible ways you can share an instance within it.

Cloud Foundry (CF) runtime

Within the Cloud Foundry environment, you can share instances in several ways.

Same CF API endpoints:

CF spaces in the same org

• Example: Space-A ⇆ Space-B within Org-X

CF spaces in different subaccounts on the same main/extension landscape and region

• Example: Space-A in EU10-004 (SubAccount-X) ⇆ Space-B in EU10-004 (SubAccount-Y)

Different CF API endpoints:

CF spaces across different subaccounts on different hyperscalers

• Example: Space-A in EU10 (SubAccount-X) ⇆ Space-B in EU30 (SubAccount-Y)

CF spaces across different subaccounts on different main/extension landscapes

• Example: Space-A in US10 (SubAccount-X) ⇆ Space-B in US10-002 (SubAccount-Y)

CF spaces across different subaccounts in different regions

• Example: Space-A in US10 (SubAccount-X) ⇆ Space-B in EU10

Sharing using CF CLI

Sharing a Cloud Logging instance using CF API, where possible, is straightforward.

cf share-service SERVICE-INSTANCE -s OTHER-SPACE

Then proceed with binding it to CF apps from the OTHER-SPACE.

Please read the limitations and security considerations from CF docs on sharing-instance

Sharing using User-Provided Service (UPS)

Best Practices

1. Name Service Keys Clearly
• Consider using one service key per remote CF location where the instance will be shared.
• Include the remote CF location identifier in the service key name for easy association with its target destination.
2.  Rotate Service Keys Periodically
• Either based on your organization’s compliance policy or, if you use mTLS, based on the certificate validity, you might have to periodically update the User-Provided Service.
• Rotation involves creating a new service key in the parent instance and updating the UPS in the target locations accordingly.

Sample Script

Below is a sample script demonstrating how to share and update an instance using a User-Provided Service (UPS).

In this example, we create a central SAP Cloud Logging instance in EU10 (referred to as the Source CF location) and make it available in other location - EU30 (referred to as the Destination CF location) by sharing it through a User-Provided Service.

• Create & bind to UPS:  See attachment below create-UPS.txt 📎
• Update UPS: See attachment below update-UPS.txt 📎

Visualizing

Using OpenSearch Dashboards, telemetry data (CF logs & CF Container metrics) from all the bound apps can be viewed centrally from the parent instance.

You can distinguish and filter for different locations based on fields like `organization_name` and `organization_id`

Considerations & Tradeoffs

Planning actual instantiation and deployment of Cloud Logging service instances should consider the following aspects:

• Cost: In most cases, shared instances should be more cost-effective than dedicated ones.
• Legal/operational requirements: Customers may be required to run dedicated instances. However, Cloud Logging provides many security-level features (e.g., index-level security) that might be sufficient to meet these needs.
• Analytics: Putting observability data into shared instances allows for combined analysis. Note that, in contrast to Application Logging, data cannot be analyzed across service instances.
• Ingestion throughput: Remote ingestion (from Cloud Foundry) may result in slightly lower throughput, especially if you connect regions that are physically far apart.

create-UPS.txt

update-UPS.txt