How to Send and Receive SMS with the SAP Cloud Application Programming Model

[mariusobert]

The SAP Cloud Application Programming Model (CAP) describes itself as a framework of languages, libraries, and tools for building enterprise-grade services and applications. Its primary capabilities are database modeling via Core Data Services and other enterprise-grade features for localization, data privacy, authorization, and messaging. The framework offers a high abstraction for these services to relieve the coders from writing as much boilerplate code as possible. CAP provides the flexibility to include arbitrary JavaScript libraries in its lifecycle for all other service integrations. This post will demonstrate how to connect CAP apps to 3rd-party services via webhooks.

Using a phone client to interact with a CAP application

For this post, we'll exemplarily pick the Twilio web service as an instance for any state-of-the-art 3rd-party web service. With this service, our CAP application will be able to send and receive SMS.

What are we building today?

We won't reinvent the wheel in this post and stick with the well-known hello-world-scenario used in CAP projects: The bookshop. Don't worry if you haven't used this one before. Bookshop examples are small CRUD web applications that use entities from a bookshop such as books, authors, orders, etc. The project that you will build is straightforward and only has a single entity: Books. All records of this entity will be exposed via a read-only REST endpoint. So, it’s more of a RU (read and update) web app, without create and delete operations. Furthermore, the final application will accept HTTP POST requests to listen for incoming orders, reducing the “stock” field of the ordered book. Thanks to the used SMS service, we'll ensure the shop managers get notified when the stock of a given book runs low and provide an easy way to order more books from the suppliers.

What you need

To follow this tutorial, you will need:

• Node.js installed (version 14 or newer)


• ngrok installed


• CDS CLI installed


• SQLite binary installed (Unix users probably already have it)


• An existing Twilio account with an SMS-capable phone number


• Optional: Visual Studio Code with the following extensions installed
  
  
  
  • The language support extension
  
  
  • The REST client extension
  
  
  
  

Start a bookshop-project

The CLI comes with a convenient init command to bootstrap new projects.

cds init bookshop --add samples

cd bookshop

npm install

In the next step, you will invoke a command to start the webserver on your local machine. This command creates all the needed files. This command uses three options:

The in-memory option ensures that the database schema is deployed to the SQLite database, while the to option controls the web protocol used to expose the data services. And the watch option to restart the server automatically when file changes happen. For the last option, you need to add a new development dependency.

npm add -D @sap/cds-dk

cds serve all --watch --in-memory --to rest

Now it's time to test this basic project by sending an HTTP GET request to http://localhost:4004/catalog/Books. I recommend doing this with the REST Client extension for Visual Studio Code and a new file called requests.http:

### Get stock data from the catalog

GET http://localhost:4004/catalog/Books

VS Code will add a clickable "Rend Request" text above that definition. After sending the request, you should see the following response.



Alternatively, you can use curl to send this request as well.

curl http://localhost:4004/catalog/Books

In the next step, you'll add an action that defines a POST endpoint in your web service. For this, you need to add the following line in the catalog definition of srv/cat-services.cds.

using my.bookshop as my from '../db/data-model';



service CatalogService {

   @readonly

   entity Books as projection on my.Books;



   action submitOrder(book : Books:ID, quantity : Integer);

}

And add the implementation of this action in a new file called srv/cat-service.js. As mentioned above, this action will parse the incoming request and use the parameters to reduce the value of the stock field. If the value of that field turns negative, the request needs to fail:

const cds = require("@sap/cds");



class CatalogService extends cds.ApplicationService {

 init() {

   const { Books } = cds.entities("my.bookshop");



   // Reduce stock of ordered books if available stock suffices

   this.on("submitOrder", async (req) => {

     const { book, quantity } = req.data;

     let { stock, title } = await SELECT`stock, title`.from(Books, book);

     const remaining = stock - quantity;

     if (remaining < 0) {

       return req.reject(409, `${quantity} exceeds stock for book #${book}`);

     }

     await UPDATE(Books, book).with({ stock: remaining });

     return { ID: book, stock: remaining };

   });



   return super.init();

 }

}



module.exports = { CatalogService };

Thanks to the watch option, the service restarts automatically once you save the file.

Append the following lines to the requests.http file to define the second request template:

### Get stock data from the catalog

GET http://localhost:4004/catalog/Books



### Submit an order

POST http://localhost:4004/catalog/submitOrder HTTP/1.1

Content-Type: application/json



{

   "book": 1,

   "quantity": 95

}

Same as before, you can utilize curl here to send the request via the terminal.

curl -X POST -d '{"book":1,"quantity":95}'  -H 'Content-Type: application/json' http://localhost:4004/catalog/submitOrder

The first request will return an empty, but successful, response. Additional requests will fail because the stock has been too low.

Send a message when the stock decreases significantly

In this section, you'll add basic SMS capabilities to your project. As mentioned before, we'll use the Twilio API as an example here.

The npm package makes it possible to call the API with a single line of code. Therefore, add the client package as a dependency to the project.

npm add twilio

To connect this project to your Twilio account, you need to include the Account SID and the Auth Token. It's crucial to keep them private and exclude them from the codebase you check into a git repository. So, it makes sense to keep them in the environment variables of the project. In any CAP project, the default-env.json file is the perfect place for these secrets as it's already on the .gitignore list, and all properties are automatically loaded in the environment variables during startup. For this application, replace all placeholder values such as the sender and receiver number of the text messages to the new file default-env.json as well:

{

 "TWILIO_ACCOUNT_SID": "<Replace with Account SID>",

 "TWILIO_AUTH_TOKEN": "<Replace with Auth Token>",

 "TWILIO_SENDER": "<Replace with number in this format: +18600000000>", 

 "TWILIO_RECEIVER": "<Replace with number in this format: +18600000000>"

}

Now that you have prepared the runtime environment, it's time to initialize the client and send a warning message when the stock threshold is reached. Add the highlighted lines to the service implementation srv/cat-service.js:

const cds = require("@sap/cds");

const twilio = require("twilio");



const twilioClient = twilio();



class CatalogService extends cds.ApplicationService {

 init() {

   const { Books } = cds.entities("my.bookshop");



   // Reduce stock of ordered books if available stock suffices

   this.on("submitOrder", async (req) => {

     const { book, quantity } = req.data;

     let { stock, title } = await SELECT`stock, title`.from(Books, book);

     const remaining = stock - quantity;

     if (remaining < 0) {

       return req.reject(409, `${quantity} exceeds stock for book #${book}`);

     }

     await UPDATE(Books, book).with({ stock: remaining });



     if (remaining < 10) {

       twilioClient.messages

         .create({

           body: `A customer just ordered ${quantity}x "${title}" and there are only ${remaining} left in stock.`,

           from: process.env.TWILIO_SENDER,

           to: process.env.TWILIO_RECEIVER,

         })

         .then((message) =>

           console.log(`Message ${message.sid} has been delivered.`)

         )

         .catch((message) => console.error(message));

     }



     return { ID: book, stock: remaining };

   });



   return super.init();

 }

}



module.exports = { CatalogService };

Submit an order by triggering the second HTTP request.



You should now receive the following message on your phone.

Listen to inbound messages for restocking

The previous section established a one-way channel from your project to the mobile of the bookshop managers. This final section will turn it into a two-way communication channel that can read responses sent back by the managers. In the Twilio Console, you can define what happens when you receive a message. There are several options to react to this event. You can specify a static response, handle the request dynamically with a serverless function, or forward the request to a webhook of your application. In our case, the last option makes the most sense. So, we'll use a custom middleware to implement the webhook and deal with the message. As the application currently runs on localhost, you need to open a tunnel to route traffic to your machine from the Twilio data center. For this, you'll use ngrok.

Let's build this webhook step-by-step to understand what has to be done.

To begin, tell the bookshop manager how they can respond to the initial text message. Therefore, change the following line in the service implementation srv/cat-service.js:

twilioClient.messages

         .create({

           body: `A customer just ordered ${quantity}x "${title}" and there are `+

           `only ${remaining} left in stock. Please respond with "Yes" `+

           `if you would like to restock now.`,

           from: process.env.TWILIO_SENDER,

           to: process.env.TWILIO_RECEIVER,

         })

To create a middleware with CAP, you only need to create a file srv/server.js and listen to the bootstrap event before you can initialize the client. Also include thewebhook() middleware to prevent misuse by making sure only servers in Twilio data centers can call this webhook in production.

const cds = require("@sap/cds");

var bodyParser = require("body-parser");

const twilio = require("twilio");



const MessagingResponse = twilio.twiml.MessagingResponse;



cds.on("bootstrap", (app) => {

 const twilioClient = twilio();



 app.use(bodyParser.urlencoded({ extended: true }));



 app.post(

   "/webhook",

   twilio.webhook({ validate: process.env.NODE_ENV === "production" }), // Don't validate in test mode

   async (req, res) => {

     req.res.writeHead(200, { "Content-Type": "text/xml" });

     res.end({ ok: 200 });

   }

 );

});

Implement the middleware to parse the affected book, update the database record, and inform the bookshop manager whether it worked:

 app.post(

   "/webhook",

   twilio.webhook({ validate: process.env.NODE_ENV === "production" }), // Don't validate in test mode

   async (req, res) => {

     req.res.writeHead(200, { "Content-Type": "text/xml" });

     const twiml = new MessagingResponse();



     if (req.body.Body.includes("Yes")) {

       const parsed = await collectBookDetails(req.body.From, req.body.Body);

       if (parsed?.book?.ID && parsed?.book?.stock) {

         const newStock = parsed?.book.stock + parsed.restock;

         await cds.update("Books").where({ ID: parsed?.book.ID }).with({

           stock: newStock,

         });



         twiml.message(

           `Great, your supplier  has been contacted, and tomorrow there will be ${newStock} items in stock.`

         );

       } else {

         twiml.message("Oh no, something went wrong. ");

       }

     } else {

       twiml.message(

         `I'm sorry, I don't understand that reply. Please answer with "Yes" or "Yes, order 60 additional book."`

       );

     }

     res.end(twiml.toString());

   }

 );

You probably already noticed that you called a missing function. Let's change that by adding the collectBookDetails function in srv/server.js that reads contextual data from the last message sent to the bookshop manager. Add the new function as an inner function right after the declaration of the twilioClient to make sure it is in scope.

 const twilioClient = twilio();



 async function collectBookDetails(sender, message) {

   const lastMessages = await twilioClient.messages.list({

     limit: 1,

     from: process.env.TWILIO_SENDER,

     to: sender,

   });

   const lastMessage = lastMessages[0]?.body;



   if (lastMessage) {

     const restockPattern = /\d+/;

     const lastOrderPattern = /(\d+)x/;

     const titlePattern = /"(.*?)"/;



     const restock = message.match(restockPattern)

       ? +message.match(restockPattern)[0]

       : undefined;



     try {

       const lastOrder = +lastMessage.match(lastOrderPattern)[1];

       const title = lastMessage.match(titlePattern)[1];

       const books = await cds.read("Books").where({ title });



       return {

         restock: restock || lastOrder,

         book: books[0],

       };

     } catch (err) {

       //regex didn't find a last order or book title

       return {};

     }

   }

 }



 app.use(bodyParser.urlencoded({ extended: true }));

Before you test this flow end-to-end, run it locally first. Add the following request to the file requests.http, replace the placeholder with your phone number, and trigger the request.

Note that you need to URL-encode the plus sign with %2b

### Test endpoint to restock books

POST http://localhost:4004/webhook HTTP/1.1

Content-Type: application/x-www-form-urlencoded



Body=Yes 400&From=%2b18600000000

You should now see a TwiML (Twilio Markup Language) response. This markup will tell the Twilio servers how to respond to the sender.

<?xml version="1.0" encoding="UTF-8"?>

<Response>

  <Message>Great, your supplier  has been contacted, and tomorrow there will be 405 items in stock.</Message>

</Response>

Use ngrok to open a tunnel from your local port 4004 to the internet.

ngrok http 4004

Go to the Twilio Console and navigate to your phone number. Add the HTTPS URL that the previous step printed with the /webhook suffix to the section “A message comes in”.



Let's give it a shot. Respond with "Yes, please order 100 additional books" to the message you received a few minutes ago. Now query the current stock info via the first HTTP request once more.



 

What's next

You did it. You added a two-way communication channel to a CAP bookshop application and improved the user experience of bookshop managers!

From here, you can extend the application in multiple ways. You can go down the rabbit hole and learn more about the SAP Cloud Application Programming Model to build enterprise applications or add additional communication channels such as Email, WhatsApp, Voice, or Video. Or you can upgrade your login mechanism to protect your web application from bots. For comparison, you can also find the entire source code on GitHub.