- SAP Managed Tags:
In this fourth post in my CloudFoundryFun series, we will dive a little bit deeper in the Cloud Foundry universe and talk about Tasks. Tasks are deterministic programs that can be triggered manually or also be scheduled.
When we think of apps and services we usually think of processes that, in the ideal case, never terminate. Most of the time, they stay put and wait for incoming requests. Once a request comes along, the app serves the request and then go back into the wait state.
While this is certainly the main purpose of cloud apps, there are many good examples of tasks that need to do something and then terminate. Possible examples could be:
Those tasks might occur only once, like a DB migration, or they can occur in regular time intervals, like data uploading. In either way, cloud platforms need a mechanism to handle tasks in a resource and cost efficient way.
Cloud Foundry comes with its own concept for tasks to solve those problems. But don't worry, it's a concept that you're already very familiar with!
Tasks in Cloud Foundry are apps that haven't been started and don't have any instance running. You can use the no-start flag to push the application without starting it. This ensures the app won't consume your resources and won't cost you anything when it isn't running.
A task is basically an independent Cloud Foundry app. It can make use of the same service bindings as your apps and communicate via messaging services.
You might wonder now: “Are you describing serverless aka FaaS to me?” And I agree, Cloud Foundry Tasks have some things in common with functions. They can be used to solve the same problems. As always, when you compare two different approaches, both have advantages and disadvantages. The FaaS approach has the potential to scale very well and integrates easily with “stuff” outside of Cloud Foundry. CF Tasks on the other hand, are first-class citizens in Cloud Foundry. Thus integrate well with existing CF services and apps. They are developed, deployed and managed like regular apps, which is why their lifecycle management is easier.
I don't want to say one approach is superior over the other. I think it's important to evaluate this for your individual needs.
If you've ever used the Cloud Application Programming Model, you already used CF tasks. You might have noticed that your CAP project contains at least two modules:
What's the purpose of both modules, what happens during deploy and runtime? This is a very good question, so let’s have a look into the Cloud Cockpit after the deployment of the MTA archive:
You can see, the srv module started to serve incoming requests. The db module on the other side isn't running anymore. This is intended, your app did not crash. Let's have a look at the associated logs
So you can see that this module triggered a task to set up the DB schema and to import the sample data. The Cloud Foundry CLI actually tells you that this is a task ([APP/TASK] in the red box), and not a regular application.
Now you understand what tasks are and how they are already used in production scenarios. The next step would be: How can I use them for my own app?
As you know, Cloud Foundry builds on top of open standards instead of reinventing the wheel. You can run CF tasks in the same manner as local commands on your machine. Use the name of your running Cloud Foundry app as <APPNAME> and <TASK COMMAND> can be any terminal command of your choice.
It often makes sense to schedule recurring tasks. Examples could be sending out reports via email or importing new data into your database. For those scenarios, SAP offers the Job Scheduler service in the Cloud Platform. You can use this service to schedule and manage all your tasks in one dashboard. The service dashboard also allows you to see and download the run logs.
You know my posts, especially in this series, always contain a hands-on part. Let's get to the fun part!
Let’s write a small app to provide information about the latest scores of the Bundesliga. For this task, we leverage a free API to fetch the past results. We then save the results to a MongoDB backing service which is bound to our app. Like in most other major sports leagues, the matches of Bundesliga are (almost) always scheduled on the weekend. It wouldn't make sense to fetch the latest results every day or even more frequently. Instead, we want to fetch the latest results once per week and import them in our database.
Let's split our mini app into the following four files:
The persistence.js file takes care of the database connection. It provides two functions to read and write data into our MongoDB.
The index.js file starts a simple express-based server. A request to this server returns all available match data.
The last JS file, fetch.js, calls a public API to fetch the latest results and inserts them into the database.
In a productive scenario, we would want to check for duplicate data. But for the sake of simplicity, we leave this part out here.
The package.json file ties everything together and defines the npm scripts. When we deploy the app, Cloud Foundry will execute npm start to start up the service.
Here, we integrated the task into the same application, as our server. In productive scenarios, you might want to split the server app from the tasks.
In this edition you have learned:
Fetch (football) scores periodically
About Tasks
When we think of apps and services we usually think of processes that, in the ideal case, never terminate. Most of the time, they stay put and wait for incoming requests. Once a request comes along, the app serves the request and then go back into the wait state.
While this is certainly the main purpose of cloud apps, there are many good examples of tasks that need to do something and then terminate. Possible examples could be:
- Migrating a database
- Sending an email
- Running a batch job
- Running a data processing script
- Processing images
- Optimizing a search index
- Uploading data
- Backing up data
Those tasks might occur only once, like a DB migration, or they can occur in regular time intervals, like data uploading. In either way, cloud platforms need a mechanism to handle tasks in a resource and cost efficient way.
Cloud Foundry Tasks
Cloud Foundry comes with its own concept for tasks to solve those problems. But don't worry, it's a concept that you're already very familiar with!
Tasks in Cloud Foundry are apps that haven't been started and don't have any instance running. You can use the no-start flag to push the application without starting it. This ensures the app won't consume your resources and won't cost you anything when it isn't running.
A task is basically an independent Cloud Foundry app. It can make use of the same service bindings as your apps and communicate via messaging services.
You might wonder now: “Are you describing serverless aka FaaS to me?” And I agree, Cloud Foundry Tasks have some things in common with functions. They can be used to solve the same problems. As always, when you compare two different approaches, both have advantages and disadvantages. The FaaS approach has the potential to scale very well and integrates easily with “stuff” outside of Cloud Foundry. CF Tasks on the other hand, are first-class citizens in Cloud Foundry. Thus integrate well with existing CF services and apps. They are developed, deployed and managed like regular apps, which is why their lifecycle management is easier.
I don't want to say one approach is superior over the other. I think it's important to evaluate this for your individual needs.
Practical Example
If you've ever used the Cloud Application Programming Model, you already used CF tasks. You might have noticed that your CAP project contains at least two modules:
- the db module, to define the data schema
- the srv module, to define the service layer
What's the purpose of both modules, what happens during deploy and runtime? This is a very good question, so let’s have a look into the Cloud Cockpit after the deployment of the MTA archive:
You can see, the srv module started to serve incoming requests. The db module on the other side isn't running anymore. This is intended, your app did not crash. Let's have a look at the associated logs
cf logs cap-project-db --recentSo you can see that this module triggered a task to set up the DB schema and to import the sample data. The Cloud Foundry CLI actually tells you that this is a task ([APP/TASK] in the red box), and not a regular application.
Triggering CF Tasks
Now you understand what tasks are and how they are already used in production scenarios. The next step would be: How can I use them for my own app?
As you know, Cloud Foundry builds on top of open standards instead of reinventing the wheel. You can run CF tasks in the same manner as local commands on your machine. Use the name of your running Cloud Foundry app as <APPNAME> and <TASK COMMAND> can be any terminal command of your choice.
# general command
cf run-task <APPNAME> "<TASK COMMAND>"
# prints "test" to the log of the app named CF-Tasks-CFF4
cf run-task CF-Tasks-CFF4 "echo 'test'"
# triggers the npm script "script" of the app named CF-Tasks-CFF4
cf run-task CF-Tasks-CFF4 "npm run script" Schedule your own CF Tasks
It often makes sense to schedule recurring tasks. Examples could be sending out reports via email or importing new data into your database. For those scenarios, SAP offers the Job Scheduler service in the Cloud Platform. You can use this service to schedule and manage all your tasks in one dashboard. The service dashboard also allows you to see and download the run logs.
Hands-on
You know my posts, especially in this series, always contain a hands-on part. Let's get to the fun part!
Let’s write a small app to provide information about the latest scores of the Bundesliga. For this task, we leverage a free API to fetch the past results. We then save the results to a MongoDB backing service which is bound to our app. Like in most other major sports leagues, the matches of Bundesliga are (almost) always scheduled on the weekend. It wouldn't make sense to fetch the latest results every day or even more frequently. Instead, we want to fetch the latest results once per week and import them in our database.
Let's split our mini app into the following four files:
Persistence
The persistence.js file takes care of the database connection. It provides two functions to read and write data into our MongoDB.
const { MongoClient } = require('mongodb');
const appEnv = require('cfenv').getAppEnv();
function connectDB(fnCb) {
const oMongoCredentials = appEnv.getService('myMongo').credentials;
MongoClient.connect(oMongoCredentials.uri, function(err, db) {
if (err) {
throw err;
}
fnCb(db.db(oMongoCredentials.dbname), () => {
db.close();
});
});
}
module.exports = {
fetchAllMatches: function(fnCb) {
connectDB((dbo, fnCloseConnection) => {
dbo.collection('Matches').find({}, function(err, oCollection) {
if (err) {
throw err;
}
oCollection.toArray((err, aMatches) => {
if (err) {
throw err;
}
fnCb(aMatches);
fnCloseConnection();
});
});
});
},
insertMatches: function(aMatches, fnCb) {
connectDB((dbo, fnCloseConnection) => {
dbo.collection('Matches').insertMany(aMatches, (err) => {
if (err) {
throw err;
}
fnCb();
fnCloseConnection();
});
});
}
};
Access saved results
The index.js file starts a simple express-based server. A request to this server returns all available match data.
const cfenv = require('cfenv');
const express = require('express');
const db = require('./persistence');
express()
.get('/', function(req, res) {
db.fetchAllMatches(function(aMatches) {
res.json(aMatches);
});
})
.listen(cfenv.getAppEnv().port);
Fetch latest results
The last JS file, fetch.js, calls a public API to fetch the latest results and inserts them into the database.
const axios = require('axios');
const db = require('./persistence');
axios.get('https://www.openligadb.de/api/getmatchdata/bl1')
.then(response => {
db.insertMatches(response.data, function() {
console.log(`${response.data.length} documents inserted`);
});
});
In a productive scenario, we would want to check for duplicate data. But for the sake of simplicity, we leave this part out here.
Tying it all together
The package.json file ties everything together and defines the npm scripts. When we deploy the app, Cloud Foundry will execute npm start to start up the service.
{
"name": "CF-Tasks-CFF4",
"version": "1.0.0",
"main": "index.js",
"scripts": {
"start": "node index.js",
"fetch": "node fetch.js"
},
"dependencies": {
"axios": "^0.18.0",
"cfenv": "^1.2.2",
"express": "^4.16.4",
"mongodb": "^3.2.3"
}
}
Here, we integrated the task into the same application, as our server. In productive scenarios, you might want to split the server app from the tasks.
Schedule fetching
- Create a service instance of the Job Scheduler.
- Open the dashboard of this service instance
- Click on the Tasks on the left sidebar and Create Task
- Give the task a name of your choice. Make sure to select the deployed application which contains the task. Then, enter the invocation command to trigger the task (here: npm run fetch)
- Click on the newly created task
- Select Schedules from the sidebar and Create Schedule
- Provide a description for the schedule and define its re-occurrence pattern.
Summary
In this edition you have learned:
- What tasks are and how they differ from apps
- How to trigger a task in Cloud Foundry
- How to use SAP Cloud Platform Job Scheduler
- How to schedule a task in Cloud Foundry
About this series
| This was the fourth blog post of my monthly new series #CloudFoundryFun. The name already says all there is to it, this series won’t be about building enterprise apps on Cloud Foundry. I think there are already plenty of great posts about those aspects out there. This series rather thinks outside the box and demonstrates unconventional Cloud Foundry use-cases ?. |
CloudFoundryFun #5 – Play Asteroids powered by React, secured by SAP Cloud Platform
4 Comments
