Near-Real time processing with Intelligent Service...

arpitgoyal · ‎2018 Mar 26

Naming the article

Before I begin explain the use-case and how to model it, I want to just share the views of the title of the article. It is my opinion that the system can have millions of events flowing and the processing of this use-case can be near-real time or real-time. To be on the safer side, I quoted the former. This is not scheduled, but faster than that for use-cases of "something" to be processed as soon as the state of the system has changed. This is solely my own views of how the application can be used together beautifully.

Business Use-cases

An employee gets hired and his information needs to be sent to an external vendor like IT procurement team or just a welcome message on Slack Team Channel.

Mass employees are hired and the user information has to be synchronized to an external system or the details has to be shared to external vendor.

What will be covered

We would be going though the following steps in this blog:

Model an Intelligent Services flow to define Custom Integration.

Model the Integration Center definition which will based on the New Hire event would extract the details of the user details on to an sFTP folder.

Define a silly[1] rule which gets triggered on Hire of an employee or even when we make any modification to the employee data.

What is not covered/missing

Creating of a new employee. From admin center it is a straight forward process and has to be done post modeling of the flow and the integration.

Connecting of the rule to the employee central succession data model is not part of this document. I have used provisioning to enable it, but I would like to keep the document more customer focus, then partner implementation. As and when I found more details of how to associate the rule to the employee central succession data model, I will update this missing piece of the puzzle.

Prerequisite

Make sure that the Intelligent Service Center permission is available to the user.

In case you don't see the permission settings, the functionality is disabled and should be enabled via Upgrade Center. The discussion for that aspect is not covered by this blog.

And also the Integration Center permissions

Apart from that you would need to have access to the OData APIs (in this case EmpJob entity to be precise)

Modeling of Business Rule

Navigate from Admin Center to Configure Business Rule.

We are picking a Basic Rule, with same Rule Name and Rule ID. The role Type as displayed in the picture above is the Compensation Eligibility. Missing in the above image is the Base Object, which need to be selected as Job Information.

Save this rule and here is the missing piece earlier described of configuring business rule with the triggering of the event. This detail would be updated eventually in the article.

Modeling of the Intelligent Services Center Flow

From the Admin Center, search for the Intelligent Services Center as shown below

On clicking on the link, the application would navigate to the landing page. Based on your system configuration you would see the list of Events enabled and visible.

For our use-case, we would select the Employee Hire event and configure it.

The Intelligent Services Center (ISC) editor displayed how the event would flow in the system and hence this is what we call the flow. It allows two custom activities of External Events and Integration Center. For this article we would be concentrating on Integration Center.

The details about different components of the ISC editor are not covered part of this blog. For any questions related to that, do leave a comment and would be able to add more content or will create a separate blog explaining those aspects.

On selecting of the Integration custom activity, the application would query all integrations which are already modeled for a specific event. As shown below, in my system here there are already seven integrations modeled for the Employee Hire event - just because I was testing that use-case 🙂

It is important to note, to avoid displaying large information, the system filters only integrations relevant for the event type. Another integration created for say Update of Job Requisition would not be visible when selecting Integration Center activity for Employee Hire event.

In case of creation of a new integration, we would display another overlaying dialog to ask the two questions of Destination Type and Format. We are going with SFTP and CSV. Try other combinations and do drop comments as to how they go.

A little discontinuity in the blog here. We will continue talking about the flow and come back about the creation of integration in the next section.

The reason of having six different integration number 1-6, is to show case the functionality of how the Integration activity can be executed. In the below two screenshot, I am showcasing the following behavior of x days/weeks before, x days/weeks after, on effective date and when event was fired.

When Event is Published: This means in this use-case as on the date employee hiring was entered into the system using Add New Employee. Eg: With respect to Hire, this could be a use case where customer wants to immediately send to update the budget-forecast of the year as soon employee is in the system, to adopt the monetary aspect of hiring employees in general.

Note: For events which are not effective dated, this is the only option and can't be changed like Update of Job Requisition.

X days/weeks Before: This means x number of days before theHire date which is the effective date of this employee. For other use-cases, it would be with respect to corresponding effective date. Eg: Say an employee is getting hired on 1st of April and the entry was made by the HR on 25th of Feb, when the candidate excepted to join the organization. To procure the hardware necessary for the employee to do his work, say 15 days before the hardware could be ordered. This avoids any kind of first day challenges.

X days/weeks After: Pretty simple if you understood the last one. This is x days/weeks after the effective date has gone by. A funny use-case came to my mind was to send an event 30 days after the hiring of an employee to order cake or send flower to wish employee for successful first month.

For both the above cases, the unit of measurement is either days or weeks.

On Effective Date: In our use-case, the event is fired on the day employee joins or his/her hire date.

The six integrations are associated with the flow to showcase some unique cases.

Assume that the employee is entered in the system on 20th March and his Hired date is 1st of April. For that case, the ISC-Integration-1 would get executed on 20th March, where as ISC-Integration-6 would be executed on 1st of April.

Assume that the employee is entered in the system on 28th March and his Hire date is 1st of April (so that is 4 days of gap). For that case, the ISC-Integration-2 would get executed on 29th of March, but ISC-Integration-3 would execute immediate. This is very important to understand from a timeline perspective.

For the same combination as #2, ISC-Integration-4 and ISC-Integration-5 would get scheduled for 3rd and 8th April respectively.

Do play around with all these combination to get a grasp on the concepts of these options. Also, see the Result section at the end where some of these runs are displayed with similar timelines (just the dates are different).

Modeling of the Integration Center definition

This section has been kept in brief and doesn't go deep dive into the modeling of an integration concept. This section show cases two aspect, first if the content developer is coming from ISC with the option of Create New Integration. Secondly, if the content developer is starting with Integration Center and then would like to associate the integration with the flow.

Navigate from ISC

We left above at this screenshot and what if we click at the Create New Integration option, something like below.

We land at the wizard in integration center directly, with our entity pre-selected (In our case EmpJob).

User could model the different aspect of integration such as Configure Field, Filter and Destination Settings (see other articles linked at the bottom). If you have modeled Integration before, you would notice the missing Scheduling tab. This is because, these integrations would be triggered based on event and not a fixed schedule.

At the end, you could go to the Review and Run tab to see all the details. Save is an option available at any point of time of your modeling. At the bottom right, you would see Go to ISC option, which navigates back the application to the event in the ISC editor.

It would navigate back to the ISC flow and you could click on the Integration activity once more on the right of the ISC flow editor to create the association.

Begin modeling from Integration Center

Not so common, but would also like to cover this aspect, that it is not always necessary to start over with ISC flow editor. Integration Center by default has an option on the +Create > More Integration Types option in the My Integration page to opt for Intelligent Services based integrations. Below picture shows the details:

On clicking on create, we would see the list of events enable for the system

Once selected, you would end up with the same wizard as what you have navigated from the ISC Editor

Monitoring

For the last 180 days, all monitoring data is visible in the Event Monitoring section of the ISC Editor itself. This is one stop details of all the activities which are triggered. The next two screen show cases how the monitoring looks like.

Initiator has been cleared, but actually it would list the person name who triggered the event (the case would be HR hiring the employee)

Click on the row, would open up dialog listing when exactly each of the activities part of the flow are triggered. Clicking on the buttons highlighted below in red, would actually navigate you to the Execution Manager Dashboard which would give details of each Integration Center details.

Final Result

On hiring an employee is done, based on the integration, the output could be a CSV file or it could also be invocation of a REST endpoint with XML or JSON format. Integration Center do have limited support for SOAP endpoint invocations with XML payload (to be detailed some other day in another blog).

Below I have an example of one of the integration execution in CSV format. Here all different information of the customer 'Rebecca Jones' is extraced whose effective date is 20-Mar-2018.

Sequence Number,Start Date,User ID,Full Name,Business Unit,Company,Cost Center Account

1,2018-03-20T00:00:00Z,rjones1,Rebecca Jones,ACE_HC,ACE_USA,21000

External Events vs Integration Center

Intelligent Services Center can use either external events or integration center as one of the custom activity. Following are the list of differences:

With External Events, the target system can support SOAP ONLY, but with Integration Center there is a list of protocol/communication option with the target system.

The format with External Events is only XML format, but with Integration Center you have the choice of format like CSV, XML, JSON, etc.

Major - Integration Center you could extract all information necessary with help of different formats. With External Events you get only the event details. For our use-case the external events would only send Sequence Number, Start Date and User ID. The target application would have to make a call-back and fetch additional information - making integration challenging. The target system needs to understand the format and manage all calls. Integration Center can understand the target system structure and send all the details without the need of any call back (unless of-course it is complete unrelated entity).

Citation

[1] I have specified it to be silly rule, because I am just trying to get the event to fire. In real world, real application usage, customer needs to define rules based on requirements.