How to access non-standard data sources within SAP Analytics Cloud

[liam_mischewski]

Introduction

SAP Analytics Cloud supports a wide variety of data sources natively, however technology is ever-evolving, and as such you may run into a case where the particular endpoint you wish to use is not able to be connected within the standard data connection environment.

This blog post aims to show you one way you can still access such an endpoint and bring it into the SAC environment using Custom Widgets and the Analytic Application capability. This blog post assumes some familiarity with Custom Widgets, if you are brand new to the topic a good series of blogs can be found here: https://blogs.sap.com/2020/01/27/your-first-sap-analytics-cloud-custom-widget-introduction/

Fetch-ing the data

Custom Widgets allow us to build web components that extend SAC and add custom functionally as needed for your use case. By using Custom Widgets and the Fetch API we are able to access any resource that is served by an endpoint.

The Fetch API is a web standard interface for fetching resources. It can be seen as an evolution of the XHR framework that enabled AJAX to drive the rich web applications we now rely on every day. A good resource to learn about this API is the MDN page for it here: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

Let's start with a simple Custom Widget that uses the Fetch API to call the desired endpoint. In this example, we will use a GeoJSON RSS feed that provides a list of major fire incidents for a region. Since this is a frequently updated feed, we'll also add an auto-refresh feature.

 class RSSDataSource extends HTMLElement {

	constructor() {

	    super();

            this.appendChild(tmpl.content.cloneNode(true));



            this._props = {

                JSONUrl: "http://www.rfs.nsw.gov.au/feeds/majorIncidents.json",

                RefreshTime: 300

            };



            //Get refrences to our root element

            this.$div = this.querySelector('div.datasource');



            //Add the handler for our refresh button 

            this.$div.querySelector('#refresh').onclick = (e) => this.refresh();

        }

        

        refresh() {

            if(this._props["JSONUrl"]) {

                this.updateData(this._props["JSONUrl"])

            }

        }



        updateData(url) {

            fetch(url)

                .then(response => response.json())

                .then(data => {

                    this._rawData = data;

                    this.startRefreshCountdown();

                });

        }



        startRefreshCountdown() {

            if(this._refreshTimeout) clearTimeout(this._refreshTimeout);



            this._refreshTimeout = setTimeout(() => this.refresh(), this._props["RefreshTime"] * 1000);



            const end = new Date().getTime() + (this._props["RefreshTime"] * 1000);



            if(this._countdownInterval) clearInterval(this._countdownInterval);



            this._countdownInterval = setInterval(() => {

                let now = new Date().getTime();

                let distance = end - now;



                let timestring = distance > 60000 ? Math.floor(distance / (1000 * 60)) + " Minutes" : Math.floor(distance / (1000)) + " Seconds";



                this.$div.querySelector(".refresh-timer .value").innerText = timestring; 

            }, 1000);

        }



        onCustomWidgetBeforeUpdate(oChangedProperties) {

            let oldUrl = this._props['JSONUrl'];

            let newUrl = oChangedProperties['JSONUrl'];



            if(newUrl != oldUrl) {

                this.updateData(newUrl);

            }



            this._props = { ...this._props, ...oChangedProperties};

	}

}

And the associated template

let tmpl = document.createElement('template');

    tmpl.innerHTML = `

        <style>

            .datasource {

                width: 100%;

                display: flex;

                justify-content: space-between;

                align-items: center;

            }



            .datasource > div {

                display: inline-block;

                vertical-align: middle;

            }



            .data-age, .refresh-timer {

                margin: 0px 12px;

            }



            .data-age > * {

                display: inline-block;

                vertical-align: middle;

            }



            .refresh-timer > * {

                display: inline-block;

                vertical-align: middle;

            }

        </style>

        <div class="datasource">

            <div class="data-age">

                <div class="label">Last Refreshed:</div>

                <div class="value">Loading</div>

            </div>

            <div class="refresh-timer">

                <div class="label">Next Refresh:</div>

                <div class="value">Never</div>

                <button id="refresh">Refresh Now</button>

            </div>

        </div>

    `;

We can now define our custom widget and create the Custom Widget JSON as per the documentation for Custom Widgets. Once we add the widget to SAC, we can see our refresh button along with a countdown till the next set of data comes in.

Simple app with our new widget

Sharing data to other SAC components

While our data exists inside its own component and we could build out the use case within it, it would be very useful to expose the data to other components within our Analytic Application. We can do this by emitting an event whenever we get new data, as well as providing some accessor functions that can process the data as needed.

To start with we update the "updateData" function to emit an event whenever the data changes.

updateData(url) {

    fetch(url)

    .then(response => response.json())

    .then(data => {

        this._rawData = data;



        this.startRefreshCountdown();



        const event = new Event("onDataUpdate");

        this.dispatchEvent(event);

    });

}

For now, we will return the raw data and allow the other component to decide what to do with it. Since we can only return specific data types and objects from a custom widget method, first we'll convert it to a JSON string before returning it. In case we do not have data available yet we will return an empty object. Since we may want to respond to specific data requests differently, we can also now add a wrapper method in order to handle the requests as they come in.

getRawData() {

    return this._rawData || {};

}



getJSONData(type) {

    let data = {};



    if(type == "raw") {

        data = this.getRawData();

    }



    return JSON.stringify(data);

}

 

Of course, we also need to update our Widget JSON with these new methods for SAC to see it.

Once we have done so we can use the "onDataUpdate" in SAC in order to update other components on our page based on the data.

onDataUpdate method. FireTable is another widget which takes the array and outputs the entries as a table

Keep in mind the event system only runs at Runtime, and so in the Design view, you will not see any results in the table. Once we run the application we can see the table populated with the RSS data.

The running application with a table populated with the latest data from the RSS feed.

Conclusion

Using this approach and with the power of the Analytic Designer, you can build out quite complex applications. Below is a built-out application using this method to provide a map view of the GeoJSON from the feed along with filters and some analytics about the ongoing incidents.

Completed build

I hope this blog post has been useful for you and you can use this knowledge in order to continue building applications within SAC. Let me know how you might use this or if you have any questions in the comments below.

Major Incidents RSS Feed CC BY 4.0 © State of New South Wales (NSW Rural Fire Service). For current information go to www.rfs.nsw.gov.au.