Streamlining Data with SAP CPI Message Mapping : Sorting and Eliminating Incomplete Records

[ShreyashD]

Introduction

In enterprise integration scenarios, managing employee data across systems is a common requirement — but raw data is often messy or incomplete.

In this blog, we’ll walk through a real-world solution in SAP Integration Suite (CPI) where we:
• Filter out invalid employee records using Message Mapping
• Sort valid employee records by EmployeeID using Groovy Script
• Deliver a clean and structured employee dataset to the target system

The goal is to streamline incoming employee data so it’s ready for consumption by downstream applications.

Business Requirement
We needed to:
• ✅ Remove employees missing EmployeeID
• ✅ Preserve only complete records with all required fields
• ✅ Sort the remaining employee records in ascending order by EmployeeID

This reflects a common need in systems like SAP SuccessFactors, SAP HCM, or third-party HR tools where employee records must be clean and ordered.

Note: 
The entire filtering and sorting of employee data could be handled using just a Groovy script. However, here I chose to use Message Mapping for filtering and Groovy only for sorting.
This approach helps demonstrate how standard tools in SAP CPI can be combined effectively — and it’s especially helpful for beginners learning both graphical and scripting options.

⚙️Tools Used
• SAP Integration Suite (CPI)
• Message Mapping – to filter out incomplete data
• Groovy Script – to sort valid data
• No external tools, APIs, or adapters needed

Better Picture

                              Sample input                                                                                    Desired Output
           

Steps to follow

• Step 1: Filter Incomplete Records Using Message Mapping. In CPI, we first create a Message Mapping between the source and target structures (Employees → Employees).
  Goal: remove any <Employee> node where EmployeeID is missing or empty.
  
  Functions Used:
  • removeContext : Flattens or isolates repeated elements from their surrounding hierarchy, treating each entry as a separate context so that they can be processed individually
  • Length: return the number of characters in the given string (used here to check if EmployeeID is empty or not).
  • GreaterThan: compares two numbers and returns true if the first is greater than the second.
  • constant: provides a fixed value to compare against.
  • createIf: creates the corresponding target node only if the connected Boolean condition is true.
  
  
  
  EmployeeID(on source side) is mapped to Employee(on target side) to only pass records having non-empty EmployeeID.
  This is known as conditional node creation.
  Below is the mapping expression used to conditionally create nodes having non-empty EmployeeID.
  
  
  
  We map the rest of the fields directly one to one like shown below.
  
  
  
  
  
• Step 2: Sort Valid Records Using Groovy Script
  
  Important Tip: Why We Use Groovy for Sorting
  🚫Message Mapping in SAP CPI cannot perform sorting operations.
  ✅It is designed for mapping and filtering data, but not for ordering or rearranging nodes.
  
  That’s why sorting must be handled by a Groovy script (or other scripting steps like XSLT) if required.
  If your source system can sort data (e.g., an OData service or database), it’s best to sort at the source when possible.
  But when that’s not feasible, a post-mapping script like the one we used is the recommended approach.
  
  Below is the groovy script used in this.
  
  
  
  iFlow Design
  The iFlow created looks something like this.
  
  
  Final Thoughts
  
  With just two steps we’ve transformed raw employee data into a clean, filtered, and sorted dataset ready for use.
  
  This approach can be applied to:
  • HR integrations
  • Payroll systems
  • Internal reporting systems
  • Master data consolidation flows