Technology Blog Posts by Members
cancel
Showing results for 
Search instead for 
Did you mean: 

Data integration is the heartbeat of any modern data warehouse. In SAP Datasphere, keeping data accurate, timely, and reliable depends largely on how effectively data flows, replication flows, and task chains are automated.

To support a variety of business and technical requirements, SAP Datasphere provides three scheduling approaches that work together:

  1. Frequency-Based
  2. Simple Schedule
  3. Cron Expression

Understanding when to use each method can help organizations simplify operations, improve maintainability, and ensure data is available when the business needs it.

Automated scheduling eliminates manual intervention and ensures data is refreshed consistently. Whether you're loading transactional data every hour, refreshing master data daily, or running complex month-end processes, the right scheduling strategy can:

  • Improve data freshness and availability
  • Reduce operational effort
  • Increase reliability of reporting and analytics
  • Optimize system resource utilization
  • Simplify administration and maintenance

The key is selecting a scheduling approach that matches your business requirements without adding unnecessary complexity.

Frequency-Based Scheduling:

           Frequency-Based Scheduling is the standard method used by most SAP Datasphere users. Instead of creating custom schedules, users simply select predefined execution intervals from the available options.

SAP Datasphere supports:

  • Every 5, 10, 15, 20, or 30 minutes
  • Hourly
  • Daily
  • Weekly
  • Monthly

Benefits

  • Easy to configure No technical expertise is required.

Common Use Cases

  • Hourly Transaction Processing - Load sales orders, inventory transactions, or operational metrics every hour.
  • Daily Master Data Refreshes - Update customer, product, and organizational data during off-peak hours.
  • Weekly Synchronization - Refresh business partner, supplier, or vendor information on a weekly basis.
  • Monthly Data Snapshots - Create period-end financial and operational snapshots for reporting purposes.

Simple Schedule: The Business-Friendly Scheduling Engine

           While Frequency-Based Scheduling defines when jobs run, the Simple Schedule is the SAP Datasphere interface that manages and executes those schedules. Think of it as the user-friendly layer that converts scheduling selections into automated background jobs. Users can configure schedules through a graphical interface without needing to understand technical scheduling syntax.

Key Advantages

  • No Technical Knowledge Required - Business users can create schedules without learning cron expressions or scripting languages.
  • Daylight Saving Time Awareness - Schedules automatically adapt based on the selected regional time zone, helping organizations manage daylight-saving changes.
  • Background Automation - Once configured, jobs execute automatically without ongoing intervention.
  • Easy Maintenance - Schedules are simple to understand and can be easily managed by administrators or business users.

Typical Use Cases

  • Daily persistence jobs at 2:00 AM
  • Weekly replication every Monday and Friday
  • Monthly warehouse refreshes on the first day of the month
  • Standard ETL and ELT workloads
  • For many organizations, Simple Schedule becomes the default scheduling approach because it balances flexibility with ease of use.

Cron Expression Scheduling: Advanced Control and Precision

             Sometimes business requirements go beyond predefined schedules. Perhaps a process must run every 15 minutes during business hours only. Or maybe month-end processing needs to run on the last Sunday of every month. For these advanced scenarios, SAP Datasphere provides Cron Expression Scheduling. Cron expressions allow users to define highly customized execution patterns using a structured scheduling syntax.

Cron scheduling supports:

  • Specific weekdays
  • Multiple executions per day
  • Business-hour-only processing
  • Date range restrictions
  • Month-end scheduling
  • Advanced recurring patterns

Advanced Business Scenarios

  • Business-Hour Processing - Run data loads every 15 minutes during office hours while reducing cloud resource consumption outside business periods.
  • Month-End Refreshes - Execute data warehouse refreshes on the last Sunday of every month.
  • Multiple Daily Executions - Run critical financial updates at: 03:00, 09:00, 15:00
  • This level of flexibility is difficult or impossible to achieve through standard scheduling options alone.

Choosing the Right Scheduling Option :

Business Requirement
Recommended Option
Every few minutes or hoursFrequency-Based Scheduling
Daily, weekly, or monthly loadsSimple Schedule
Business-friendly configurationSimple Schedule
Standard ETLSimple Schedule
Multiple custom run times per dayCron Expression
Complex calendar-based logicCron Expression
Last weekday or last Sunday schedulingCron Expression
Advanced enterprise scheduling needsCron Expression
10 Comments
nilrod
Explorer

How about event based scheduling? This feature was available for process chains concept in SAP BW and I dont see this concept being available in SAP Datasphere - to me, this seems like a very basic feature to have.

yogen_patil
Participant

Hi Nilrod,

While these may be considered basic features, they were previously unavailable for delta flows in DataSphere. With the latest release, these capabilities are now supported. Therefore, I believe this is a good opportunity for us to start leveraging them and take advantage of the enhanced functionality.

Thanks.
appel_solar_dk
Active Participant

@nilrod do you mean events in S/4? or in other systems? It is now possible to schedule via API Managing Tasks via the REST API 

What I'm really missing is a way to schedule depending on plant calendars like you can do in S/4. E.g. load more frequent the first 4 workdays of a month.

@yogen_patil I think that you miss quite a lot by only looking at the technical differences here, they kind of doesn't matter that much. It's much more important to understand the differences between "Frequency-Based Scheduling" and the Scheduled runs in terms of in which context they can be used and which limitations it gives.

Frequency based is more mature technically e.g. it allows you to stop a replication flow (which will clean up on the source side). If you choose to schedule (simple or cron) then you can't stop a replication flow. The frequency is a number of hours / minutes / days after the last job terminates i.e. there is no risk of overlapping loads.

Scheduled jobs on the other hand can be started in task chains and you can thereby start related jobs when the replication has finished, which is not possible when using frequency based schedules. On the negative side you can start replication flows that are all ready running which will cause your chain to fail, i.e. you will have to consider more error handling.

With kind regards

Kristian

nilrod
Explorer

Hi @appel_solar_dk ,

Thank you for your response. We are currently using the REST API approach to trigger some of our task chains through power automate flow. What I was looking for is, back in the BW days, you could have an event-based trigger for the Start Event in BW process chain...this approach seems to be missing in Datasphere. For example - I would like to trigger a task chain post receiving a particular file or after receiving an email, without having to involve tools like power automate, I am wondering how this can be achieved just within SAP Datasphere. 

 

appel_solar_dk
Active Participant

@nilrod so what you are looking for is actually not another way of scheduling, but rather a way of triggering a job from an event (using the API) The question is where is that event taking place? It is that system that needs to call the API.

albertosimeoni
Participant

@nilrod

from a task chain you have the option to call an API task -> so it should be possible to run another task chain ( this can count as an event  🙂  ).
But it doesn't make any sense as you can put a task chain inside another task chain.
If we talk about events, we can talk about message queues, producer/subscriber, subscription to topic, all concepts of integration suite (IPaaS in general) that are not present in Datasphere / BDC.
And looking forward as it will be "the SAP data platform" this missing may be crucial: for example if the consumer are "AI applications" used as support for operational / transactional systems (that needs fresh/new data) like the new "Joule *****" family of products.
Having some data in BDC / Datasphere that is out of sync with the ERP where the "AI app skills" call some tools and apply changes to the ERP, it seems to me very fragile and prone to errors.

From a technical perspective, in modern data platform, we can say that Datasphere do not support lambda / kappa architectures: the data processing is not in real-time, only the ingestion could be in near real-time with the remote table replication, but we don't have internal events / trigger to start the internal processing when the data lands into Datasphere.
In Datasphere the features of spark streaming processing (real time analytics) is not available, even if the platform used by HDLF spaces (spark + delta tables) technically do support it.

nilrod
Explorer

What I was talking about is an event-based scheduling of the task chain. I understand that you could have an API being after a certain event takes place to trigger the task chain but who is triggering the API? For example - In one of our case, once the file is received in an SFTP location, I usually use a power automate flow (Microsoft tool) in order to call the Rest API to trigger the Datasphere task chain. Back in the SAP BW days, we used to have "Event-based Scheduling" as one of the scheduling option which will allow the process chain to be triggered when certain event took place - I just needed to configure those event in SAP BW system for e.g. file received in application directory etc. I am just trying to understand if there is a native way in SAP datasphere itself to look for if file/event took place rather than involving external processes such as power automate etc.

albertosimeoni
Participant

-> write to an external database in a semaphore table. then with a remote table point to this, expose the remote table with a consumable view.
write a stored procedure in an opensql schema to check to this table.

put the stored procedure in "polling" with a task chain. 

if the event "there is a new row in the remote table" occurr -> the stored procedure fire an "ok" -> so the task chain in polling can call another task chain that "does the work".

So we need 1,2,3,...5 objects just for 1 trigger and we do not have conditionals in task chains (so if you need to load multiple files / table selectively replicate the objects
=> My opinion, better to stay on an external orchestrator like you did

nilrod
Explorer

Thank you Alberto! That's exactly what I was looking to understand. And agree on your opinion :).

JulianJuraske
Participant

For more complex (event driven) scheduling I would always recommend to involve a BTP.

BTP is the SAP product for event driven flows/automation.

Just create an iFlow that triggers the DSP Task Chain.
( iFlow itself get triggerd by HTTPS Sender Adapter or JMS/AMQP Sender Adapter)
https://community.sap.com/t5/technology-blog-posts-by-sap/triggering-sap-datasphere-bdc-task-chains-...
You can also do it vice versa and call an iFlow from DSP ( REST API Call).
https://community.sap.com/t5/technology-blog-posts-by-sap/start-remote-process-actions-in-btp-abap-v...


Labels in this area