How to perform Initial Loads & Bulk Init Loads in AIFAEM triggered by an ABAP Program

[AlexPfeil]

Introduction

In my previous Blogs about AIFAEM I demonstrated how to create Custom AIFAEM Events and trigger them by Creating or Changing a Business Partner in Transaction BP. When an AEM Broker is introduced into a Landscape, Customers might want to send out all existing Objects for an Event to the Broker without having to do Mass Changes to the Object (which would trigger the individual Events created before). Also due to Performance It might be necessary to send out the Events as Bulk Messages. Both approaches are explained in this Blog

 

Trigger Program

For triggering the Events there is no Out-of-the-Box Feature in AIFAEM, so we will need a Custom Program which can be triggered by a User.

This is how it could be done:

*&---------------------------------------------------------------------*
*& Report ZAIF_BUPA_INIT_LOAD
*&---------------------------------------------------------------------*
*&
*&---------------------------------------------------------------------*
REPORT zaif_bupa_init_load.

DATA: gt_swcont          TYPE TABLE OF swcont,
      gv_businesspartner TYPE bu_partner,
      gt_events          TYPE TABLE OF /aif/event_raw,
      gt_bulk            TYPE TABLE OF zaif_event_raw_table_structure,
      gv_counter         TYPE i,
      gv_objtype         TYPE swo_objtyp VALUE 'BUS1006'.

SELECT-OPTIONS s_bp FOR gv_businesspartner.

PARAMETERS p_blk AS CHECKBOX TYPE abap_boolean.
PARAMETERS p_pksz TYPE i DEFAULT '1000'.

START-OF-SELECTION.

  "Get IDs
  SELECT businesspartner
      FROM a_businesspartner
      INTO TABLE (gt_businesspartners)
      WHERE businesspartner IN @s_bp.

  LOOP AT gt_businesspartners ASSIGNING FIELD-SYMBOL(<gs_businesspartner>).
    APPEND INITIAL LINE TO gt_events ASSIGNING FIELD-SYMBOL(<gs_event>).
    <gs_event>-event = 'INIT'.
    <gs_event>-objkey = CONV swo_typeid( <gs_businesspartner> ).
    <gs_event>-objtype = gv_objtype.
    <gs_event>-rectype = 'ZAIF_EVENT'.
  ENDLOOP.

  "Send to AIF
  IF p_blk IS INITIAL. "Single Events
    TRY.
        CALL METHOD /aif/cl_enabler_xml=>transfer_to_aif_mult
          EXPORTING
            it_any_structure = gt_events
            iv_queue_ns      = 'AEM'
            iv_queue_name    = '001'.

      CATCH cx_root INTO DATA(lx_root).
        MESSAGE lx_root->get_text( ) TYPE 'E'.
    ENDTRY.
  ELSE. "Bulk
    "Build Packages
    APPEND INITIAL LINE TO gt_bulk ASSIGNING FIELD-SYMBOL(<ls_bulk>).
    <ls_bulk>-objtype = gv_objtype.
    LOOP AT gt_events ASSIGNING FIELD-SYMBOL(<ls_event>).
      APPEND INITIAL LINE TO <ls_bulk>-events ASSIGNING FIELD-SYMBOL(<ls_bulk_event>).
      MOVE-CORRESPONDING <ls_event> TO <ls_bulk_event>.
      gv_counter = gv_counter + 1.
      IF gv_counter = p_pksz AND sy-tabix < lines( gt_events ).
        APPEND INITIAL LINE TO gt_bulk ASSIGNING <ls_bulk>.
        <ls_bulk>-objtype = gv_objtype.
        CLEAR gv_counter.
      ENDIF.
    ENDLOOP.

    "Send Bulk to AIF
    TRY.
        CALL METHOD /aif/cl_enabler_xml=>transfer_to_aif_mult
          EXPORTING
            it_any_structure = gt_bulk
            iv_queue_ns      = 'AEM'
            iv_queue_name    = '001'.

      CATCH cx_root INTO lx_root.
        MESSAGE lx_root->get_text( ) TYPE 'E'.
    ENDTRY.
  ENDIF.

This Program already is able to do both Individual Events and Bulk.

We have 1 Select Option:

Business Partner: Selection of Business Partners which need to be triggered. If all BPs need to be triggered, just leave it empty.

Additionally there are two Parameters:

Send as Bulk: Activate if you want to send out multiple BPs in a Bulk Message

Objects per Bulk Message: Specify how many BPs should be in one Bulk Message

Why are we not using the FM /AIF/TRANSFER_TO_AIF_RECEIVER?

We are passing the Messages to AIF via Class Method /aif/cl_enabler_xml=>transfer_to_aif_mult , because this Method enables us to pass multiple messages to AIF at once. This way we can use the Parallelization, where AIF schedules the Messages in Jobs. In /N/AIF/PERS_CGR we are able to specify the Number of Messages per Job for our Config Group AEM / 001

If we, for instance, trigger 10.000 individual Event Messages to AIF, AIF will plan 10 Jobs with 1.000 AIF Messages each. They will be processed in Jobs in the Background parallely. 

 

AIF Setup

Individual Events

For individual Init Load Events we use the existing AIF Interface set up in this Blog.

Bulk Events

Since the Source Structure /AIF/EVENT_RAW of BOR Events can only hold one Identifier for an Event, we will need another Event which can hold a List of Identifiers (in this case Business Partner Numbers). Therefore I created a new Source Structure 

@EndUserText.label : 'Structure with Array of AIF Events'
@AbapCatalog.enhancement.category : #NOT_EXTENSIBLE
define structure zaif_event_raw_table_structure {

  events  : zaif_event_raw_t;
  objtype : swo_objtyp;

}

zaif_event_raw_t is a Table Type for /AIF/EVENT_RAW

The Target Structure of course also needs to be an Array of Business Partners instead of one Business Partner:

@EndUserText.label : 'Event Message'
@AbapCatalog.enhancement.category : #NOT_EXTENSIBLE
define structure zaif_aem_bupa_event_message {

  controller   : /aif/aem_controller;
  data         : zaif_aem_bupa;
  ignore       : abap_boolean;
}

zaif_aem_bupa_tt = Table Type for zaif_aem_bupa (Data Component of Event in this Blog)

Now I can define my Interface:

Do not forget to set up the Interface Engines to XML and maintain the Interface Determination like in AEM / BUPA in my previous Blog.

The Structure Mapping is similar to AEM / BUPA, except that the Topic is different

Also the Function Module to fetch the Data for the BPs is placed in the CONTROLLER-Structure Mapping (then I only need one Structure Mapping) and changed due to different Source and Target Structure + I select the Data for all BPs with one Select Statement per CDS View.

FUNCTION zaif_aem_bupa_get_data_bulk .
*"----------------------------------------------------------------------
*"*"Local Interface:
*"  IMPORTING
*"     REFERENCE(RAW_STRUCT) TYPE  ZAIF_EVENT_RAW_TABLE_STRUCTURE
*"     REFERENCE(RAW_LINE)
*"     REFERENCE(SMAP) TYPE  /AIF/T_SMAP
*"     REFERENCE(INTREC) TYPE  /AIF/T_INTREC
*"     REFERENCE(SENDING_SYSTEM) TYPE  /AIF/AIF_BUSINESS_SYSTEM_KEY
*"       OPTIONAL
*"  TABLES
*"      RETURN_TAB STRUCTURE  BAPIRET2 OPTIONAL
*"  CHANGING
*"     REFERENCE(OUT_STRUCT) TYPE  ZAIF_AEM_BUPA_EVENT_MSG_BULK
*"     REFERENCE(DEST_LINE)
*"     REFERENCE(DEST_TABLE)
*"     REFERENCE(APPEND_FLAG) TYPE  C
*"----------------------------------------------------------------------

  DATA lt_bupas TYPE TABLE OF but0id.

  LOOP AT out_struct-data ASSIGNING FIELD-SYMBOL(<ls_out_data>).
    APPEND INITIAL LINE TO lt_bupas ASSIGNING FIELD-SYMBOL(<ls_bupa>).
    <ls_bupa>-partner = <ls_out_data>-businesspartnertype-businesspartner.
  ENDLOOP.

  IF lt_bupas IS NOT INITIAL.
    SELECT
      FROM a_businesspartner
      FIELDS
        businesspartner,
        businesspartnergrouping,
        firstname,
        lastname,
        businesspartnercategory
      FOR ALL ENTRIES IN _bupas
     WHERE businesspartner = _bupas-partner
      INTO TABLE (lt_bp_data).

    SELECT
      FROM a_businesspartneraddress
      FIELDS
      businesspartner,
      addressid,
      validitystartdate,
      validityenddate,
      streetname,
      cityname,
      housenumber,
      postalcode,
      country
      FOR ALL ENTRIES IN _bupas
      WHERE businesspartner = _bupas-partner
      INTO TABLE (lt_bp_address_data).
  ENDIF.

  IF lt_bp_data IS NOT INITIAL.
    "First clean up Target Structure
    CLEAR out_struct-data.
    "Fill with Data from CDS Views
    LOOP AT lt_bp_data ASSIGNING FIELD-SYMBOL(<ls_bp_data>).
      APPEND INITIAL LINE TO out_struct-data ASSIGNING FIELD-SYMBOL(<ls_data>).
      MOVE-CORRESPONDING <ls_bp_data> TO <ls_data>-businesspartnertype.
      LOOP AT lt_bp_address_data ASSIGNING FIELD-SYMBOL(<ls_bp_address_data>) WHERE businesspartner = <ls_bp_data>-businesspartner.
        APPEND INITIAL LINE TO <ls_data>-businesspartnertype-to_bpaddress-bpaddress ASSIGNING FIELD-SYMBOL(<ls_address>).
        MOVE-CORRESPONDING <ls_bp_address_data> TO <ls_address>.
      ENDLOOP.
    ENDLOOP.
  ELSE.
    CALL FUNCTION '/AIF/UTIL_ADD_MSG'
      EXPORTING
        msgty      = 'E'
        msgid      = '/AIF/MES'
        msgno      = '000'
        msgv1      = 'No Business partner found'
      TABLES
        return_tab = return_tab.

  ENDIF.

ENDFUNCTION.

The assigned Action of course also is the same

 

Test

Individual Events

Result in /N/AIF/ERR

Bulk Events

Result in AIF:

What if I build Packages with up to 5 Events per Package?

Result in AIF:

Results in BTP Integration Suite:

First Message has 5 Business Partners:

Second Message 2:

Filtering

In one of my previous Blogs I have introduced you to Filtering in AIFAEM. In a Bulk interface the approach must be different, because we cannot decide to send or not to send a Message based on the BP Group, because there are multiple Groups inside of the message. So we need to filter out the BPs inside of AIF before sending the Events out to AEM. Still we can use the same Value Mapping. 

For this I create a Field Mapping with Sub-Table (indirect Mapping) 

This means that for every EVENTS-Record in the Source Side a DATA-Record is created on the Target Side, when I create a Structure Mapping for them. This is what i did:

For the Filtering I will need 2 fields:

1) Filter Criteria: Business Partner Grouping. To be able to filter on it, I´ll need to fetch it first. That is why i created a Field Mapping for it:

 Updating Media

With the Value Mapping BUPA_GET_GROUP I can fetch the Data from the Database:

2) Filter Flag: Since we are working inside of the DATA-Object, I don´t want to add a "IGNORE"-Field. Instead I will use the Field FIRSTNAME. The actual Value from the BP will overwrite the IGNORE-Information later in the Funtion Module, which select the actual Data, anyway.

After clicking the F4-Help for Field Name 1 we can switch to the Target Structure to use the Target Field BUSINESSPARTNERGROUPING (Button "Switch Structure")

This means, that if the Grouping is not maintained in the Value Mapping, it will write "X" into FIRSTNAME.

Then in the same Structure Mapping I add a Check that ensures that the Row is only mapped if FIRSTNAME is empty

The actual Value for FIRSTNAME is mapped in the Function Module here

The EVENTS->DATA Structure Mapping is an indirect mapping. AIF processes mappings recursively, so the lower levels before the higher levels. This means that the actual values of FIRSTNAME are mapped at the end. Here one example:

Lets assume we have these 5 Business partners selected by the Program. Without the Filter the Mapping Result is like this:

In my value mapping i only allow 0001:

I transform the Data and see this as result: Only the 0001 Group is mapped.

That´s it! Happy Eventing 😉