Question Re: Generic Extractor based on Function Module in Technology Q&A

Generic Extractor based on Function Module

Loed — Thu, 17 Sep 2015 09:59:01 GMT

Hi guyz,

I still didn't get the solution in my problem below..

As an alternative, I'm planning to do a full load based on system date using the FM /POSDW/SEARCH_TLOG..I saw several documents but all of them are telling me to use the FM RSAX_BIW_GET_DATA_SIMPLE or RSAX_BIW_GET_DATA..

Some documents:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33?overridelayout=t…

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/30f1a423-02ae-2e10-bd85-bde64e61fe7b?overridelayout=t…

Khoder Elzein's Blog: Create Generic Datasource using Function Module and Timestamps

I just want to ask if using the mentioned FM is the only way to do a generic extractor based on FM? I tried to use directly the FM /POSDW/SEARCH_TLOG and in the extract structure I used the ET_TRANSACTION in the EXPORT part of the FM..

But I got the error below:

I also tried to use the structure /POSDW/TRANSACTION_INT of the table above but also got this error:

I searched for some threads to solve the error above and got this..

Create Datasource with RSO2 &amp; Function Module | SCN

But after copying the FM and creating my own table or structure to be included in the TABLES tab, I still encountered the same errors above..

Any idea how can I use the FM /POSDW/SEARCH_TLOG to create a GENERIC EXTRACTOR?

Thank you..

Loed

Re: Generic Extractor based on Function Module

Former Member — Thu, 17 Sep 2015 12:06:41 GMT

Hi Loed,

The RSAX_* FMs are the right templates to use for creating a Datasource. The reason is that the Datasource expects a certain set of Importing, Exporting and Changing parameters in the FM. You can ensure that only by copying one of the RSAX* FM.

So to meet this requirement, please copy an RSAX* template and call the /POSDW/SEARCH_TLOG in the code

Regards,

Suhas

Re: Generic Extractor based on Function Module

Loed — Fri, 18 Sep 2015 05:44:02 GMT

Hi Suhas,

So to meet this requirement, please copy an RSAX* template and call the /POSDW/SEARCH_TLOG in the code..

Can you guide me in the CALL the /POSDW/SEARCH_TLOG part?

I already copied the RSAX_BIW_GET_DATA_SIMPLE based on this document:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a0f46157-e1c4-2910-27aa-e3f4a9c8df33?overridelayout=t…

But I am stuck in PAGE 9, I already did all steps until ENDLOOP command..How can I use the /POSDW/SEARCH_TLOG in the FM RSAX_BIW_GET_DATA_SIMPLE?

Thank you..

Loed

Re: Generic Extractor based on Function Module

Former Member — Fri, 18 Sep 2015 06:30:29 GMT

Remove the OPEN CURSOR and FETCH statements if you don't see a use for them.

Then, at the place where FETCH was written, write the CALL FUNCTION ... command to call the /POSDW/ FM.

The basic idea in the FM is that the E_T_DATA should be filled up, as that is the data that will go to BW. So where the FETCH was, you write the CALL FUNCTION and then take the results and fill up E_T_DATA.

Re: Generic Extractor based on Function Module

RafkeMagic — Fri, 18 Sep 2015 07:53:58 GMT

hmmm... if you remove the open cursor & fetch, then you're not really using the most important parameters of the function, are you?

there are better ways to do so then (for example: check out this link for more information)

Re: Generic Extractor based on Function Module

Former Member — Fri, 18 Sep 2015 11:52:19 GMT

OK, I wasn't aware that you could write a program for Infoset queries. Thanks for highlighting that, I agree that is a better approach than an FM.

Re: Generic Extractor based on Function Module

Former Member — Fri, 18 Sep 2015 13:21:43 GMT

Hi Loed,

I don't know the FM /POSDW/SEARCH_TLOG. But I can see that it export a internal table.

At first you can put in the FB Z_POSDW_SEARCH_TLOG for the Parameter E_T_DATA the type /POSDW/TRANSACTION_INT. It need a structure type and not table type.

In the DataSource FM you call the FM /POSDW/SEARCH_TLOG and loop the result data to internal table E_T_DATA.

Cursor..fetch is very helpfull vor select of defined amount of data. It is necessary, if you

want to extract only a part of data.

best regards

Peter

Re: Generic Extractor based on Function Module

sander_vanwilligen — Sun, 20 Sep 2015 18:36:32 GMT

Hi Loed,

The main reason for using a Function Module extractor is the scalability. The OPEN CURSOR and FETCH are meant to retrieve and process a data package with a certain package size.

However, it must be used in conjunction with a SELECT statement on a single table or join on multiple tables. It doesn't work directly in conjunction with Function Module /POSDW/SEARCH_TLOG.

I suggest to analyze Function Module /POSDW/SEARCH_TLOG. Maybe you can reuse the logic for the Function Module extractor. If it turns out to be difficult or cumbersome to reuse the logic, then it might be better to look for alternative solutions.

Best regards,

Sander

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 03:26:53 GMT

Hi guys,

I was able to make the FM work using the RSAX_BIW_GET_DATA_SIMPLE..

So I tried to extract data in RSA3 and I got the needed number of data records..

However, after replicating the datasource and extracting the same number of records in the PSA, the request is stuck in YELLOW status..It's always 0 of 965 records instead of 965 of 965 records and with GREEN status..Any idea why? Do I still need to setup something?

Thank you..

Loed

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 03:38:23 GMT

You need to use the "RAISE no_more_data" command in the FM. Usually this is put at the point in the code after the FETCH - so if the fetch results in a sy-subrc <> 0, that means there isn't any more data to be pulled. At this point the FM should raise the no_more_data exception.

The reason is that the datasource will keep calling the FM indefinitely until the no_more_data exception is thrown. RSA3's behaviour is different - it will only call the FM 10 times or until no_more_data, whichever comes earlier.

So enter that command at the appropriate point and the DS will complete execution.

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 06:31:27 GMT

Hi Suhas,

Thanks for your suggestion..I tried to use the "RAISE no_more_data" command but I got ZERO record instead..

FUNCTION zrsax_biw_get_data_simple.

*"----------------------------------------------------------------------

*"*"Local Interface:

*" IMPORTING

*" VALUE(I_REQUNR) TYPE SRSC_S_IF_SIMPLE-REQUNR

*" VALUE(I_DSOURCE) TYPE SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL

*" VALUE(I_MAXSIZE) TYPE SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL

*" VALUE(I_INITFLAG) TYPE SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL

*" VALUE(I_READ_ONLY) TYPE SRSC_S_IF_SIMPLE-READONLY OPTIONAL

*" VALUE(I_REMOTE_CALL) TYPE SBIWA_FLAG DEFAULT SBIWA_C_FLAG_OFF

*" TABLES

*" I_T_SELECT TYPE SRSC_S_IF_SIMPLE-T_SELECT OPTIONAL

*" I_T_FIELDS TYPE SRSC_S_IF_SIMPLE-T_FIELDS OPTIONAL

*" E_T_DATA STRUCTURE YTEST_POSDM OPTIONAL

*" EXCEPTIONS

*" NO_MORE_DATA

*" ERROR_PASSED_TO_MESS_HANDLER

*"----------------------------------------------------------------------

DATA:

test_table TYPE /posdw/tt_transaction_int,

test_wa LIKE LINE OF test_table,

test_result TYPE /posdw/tt_transaction_int.

DATA: temp_result TYPE /posdw/tt_transaction_int.

DATA: wa_e_t_data LIKE LINE OF e_t_data.

DATA: lv_count TYPE i.

* Example: DataSource for table SFLIGHT

TABLES: ytest_posdm.

* Auxiliary Selection criteria structure

DATA: l_s_select TYPE srsc_s_select.

* Maximum number of lines for DB table

STATICS: s_s_if TYPE srsc_s_if_simple,

* counter

s_counter_datapakid LIKE sy-tabix,

* cursor

s_cursor TYPE cursor.

* IMPORT temp_result FROM MEMORY ID 'TAB'.

* Initialization mode (first call by SAPI) or data transfer mode

* (following calls) ?

IF i_initflag = sbiwa_c_flag_on.

************************************************************************

* Initialization: check input parameters

* buffer input parameters

* prepare data selection

************************************************************************

* Check DataSource validity

CASE i_dsource.

WHEN 'YTEST'.

WHEN OTHERS.

IF 1 = 2. MESSAGE e009(r3). ENDIF.

* this is a typical log call. Please write every error message like this

log_write 'E' "message type

'R3' "message class

'009' "message number

i_dsource "message variable 1

' '. "message variable 2

RAISE error_passed_to_mess_handler.

ENDCASE.

APPEND LINES OF i_t_select TO s_s_if-t_select.

* Fill parameter buffer for data extraction calls

s_s_if-requnr = i_requnr.

s_s_if-dsource = i_dsource.

s_s_if-maxsize = i_maxsize.

* Fill field list table for an optimized select statement

* (in case that there is no 1:1 relation between InfoSource fields

* and database table fields this may be far from beeing trivial)

APPEND LINES OF i_t_fields TO s_s_if-t_fields.

ELSE.

************************************************************************

* Data transfer: First Call OPEN CURSOR + FETCH

* Following Calls FETCH only

************************************************************************

IF s_counter_datapakid = 0.

ENDIF.

CALL FUNCTION '/POSDW/SEARCH_TLOG'

* EXPORTING

* I_READTRANSDETAILS = 'X'

* I_RETAILSTOREID =

* I_BUSINESSDAYDATE =

* I_NO_SORTING =

* IS_SELECTION =

* I_USE_TREX = '1'

* I_ONLY_GET_KEY =

IMPORTING

et_transaction = test_result

* CHANGING

* C_SOURCE =

IF test_table[] IS INITIAL.

APPEND LINES OF test_result TO test_table.

ENDIF.

READ TABLE i_t_select WITH KEY fieldnm = 'BUSINESSDAYDATE'.

IF sy-subrc EQ 0.

IF i_t_select-low IS NOT INITIAL AND i_t_select-high IS INITIAL .

LOOP AT test_table INTO test_wa WHERE businessdaydate EQ i_t_select-low.

MOVE-CORRESPONDING test_wa TO wa_e_t_data.

APPEND wa_e_t_data TO e_t_data.

ENDLOOP.

ELSEIF i_t_select-low IS NOT INITIAL AND i_t_select-high IS NOT INITIAL .

LOOP AT test_table INTO test_wa WHERE businessdaydate GE i_t_select-low

AND businessdaydate LE i_t_select-high.

MOVE-CORRESPONDING test_wa TO wa_e_t_data.

APPEND wa_e_t_data TO e_t_data.

ENDLOOP.

ENDIF.

ELSE.

LOOP AT test_table INTO test_wa.

MOVE-CORRESPONDING test_wa TO wa_e_t_data.

APPEND wa_e_t_data TO e_t_data.

ENDLOOP.

ENDIF.

IF sy-subrc = 0.

RAISE no_more_data.

ENDIF.

s_counter_datapakid = s_counter_datapakid + 1.

ENDIF.

ENDFUNCTION.

Screenshots:

Did I use the RAISE no_more_data correctly?

Thank you..

Loed

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 06:41:26 GMT

So the logic seems to be that the POSDW FM is called just once and it gives you all the results required. In other words the POSDW FM (and therefore the extractor FM itself) needs to be called just once.

If that understanding of mine is correct, then we need a bit of a change in the code. You can remove the current no_more_data statement and the IF statement around it as well.

Then, go to the IF statement just before the POSDW FM call and change it to this:


 IF s_counter_datapakid > 0.
      RAISE no_more_data.
ENDIF.

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 07:15:26 GMT

Hi Suhas,

Yes you are right, the POSDW FM should only be called once..

After doing your suggested changes, I tried to re-extract data but request is still stuck in yellow..

Do I still need to do some changes in the code?

Thank you so much..

Loed

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 07:36:59 GMT

Hi Loed,

I think you can try to remove the IF-Statement around RAISE no_more_data.

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 07:40:37 GMT

Hi Peter,

I already tried it but it also gave me ZERO record..

Thank you..

Loed

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 07:55:20 GMT

Hi Loed,

I can imagine, that at the fist call you have to select/send the data, and at the second call of FM you don't send the data but generate RAISE no_more_data.

That is the common way for use of the FM DataSource.

Peter

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 08:02:23 GMT

Hi Peter,

Yes, I think the code is already functioning like it? The selection first of data then RAISE no_more_data afterwards? I just don't know why request is always stuck in YELLOW..

Is the RAISE no_more_data the only way to make the request auto-GREEN?

Thank you..

Loed

Re: Generic Extractor based on Function Module

Loed — Mon, 21 Sep 2015 08:10:58 GMT

Hi Sander,

Thank you for your input..

I will also try to look for alternative or maybe do the suggestion of Raf..

Loed

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 08:34:47 GMT

Hi Loed, the no_more_data is to mark the completion of extraction. Which turns the signal to green.

I would suggest to change the code in one block as the all the data is supplied in one package.

if sy-subrc is INITIAL.

RAISE no_more_data.

s_counter_datapakid = s_counter_datapakid + 1.

ENDIF.

Also please check the SPRO->business wearhouse->automated process->set traffic light

Thanks

Re: Generic Extractor based on Function Module

Former Member — Mon, 21 Sep 2015 09:15:27 GMT

Hi Loed,

Can you post the code as it was after making the changes I suggested? It should work as-is, but please post it so I can see if there's been any mistake from your end or mine.

The Raise no more data is mandatory - without that BW will think that ECC still has still more data to send. That is why the load remains in yellow, as BW is expecting data. Also, if the exception is raised in a particular call of the FM, BW does not take any data for that particular call. So the extraction (filling up e_t_data) must happen in one call and the raise in the next call. If you post your code I can go through and let you know if that's happening.

Regards,

Suhas