Start Routine Issue

[former_member184884]

Hi Mates,

We have are facing one strange issue in our production system we just fed up, i need all your valuable support in resolving this issue, Hope i will succeeded in this.

Scenario:

There is delta DTP which is part of daily process chain which will load the data from DSO(standard) to Cube as usual in the transformation level their is some business logic is written both in the start & end routine but this DTP will approximately 1.5 million records for this it will take

approx 10-14 hrs runtime,so after analysis we have found code which is written in the start routine is not optimized so we have done some minor changes in the code to improve the load performance.

But issue started from here onward we have done changes on development and transported to quality we have ran same DTP which is their in the production it's performance has improved drastically for 1.7 million records it took only 15 mins runtime so we felt very happy and on the same day we have transported code to production but big magic has happened here only.

when that particular DTP has ran in the production through schedule process chain it running for long time like for one data package(50000) to complete it is taking 3-4 hours so to pull all 1.5 million records it is running for 2 days due to this long run it has impacted entire production system means other loads.

So my doubt here is why it is take acting strange in the production, Please give your valuable reviews on my issue.

Below is the piece of code which is there is in the start routine

DATA : s_material  TYPE /bi0/pmaterial ,

          t_material  LIKE STANDARD TABLE OF s_material,

          wa_material LIKE LINE OF t_material,

         s_matunit   TYPE /bi0/pmat_unit,

          t_matunit   LIKE STANDARD TABLE OF s_matunit,

          wa_matunit  LIKE LINE OF t_matunit.

DATA : s_primrate TYPE /bic/azsd_o01900,

       t_primrate LIKE STANDARD TABLE OF s_primrate,

       wa_primrate LIKE LINE OF t_primrate,

       min_calday LIKE sy-datum.

DATA : BEGIN OF r_compcode OCCURS 100,

        sign   TYPE c LENGTH 1,

        option TYPE c LENGTH 2,

        low    TYPE c LENGTH 4,

        high   TYPE c LENGTH 4,

       END OF r_compcode.

data : s_/BI0/PCOMP_CODE  type /BI0/PCOMP_CODE ,

        t_/BI0/PCOMP_CODE  like STANDARD TABLE OF s_/BI0/PCOMP_CODE,

       wa_/BI0/PCOMP_CODE LIKE LINE OF t_/BI0/PCOMP_CODE.

data  wa_r_compcode like LINE OF r_compcode.

CLASS lcl_transform IMPLEMENTATION.

  METHOD start_routine.

    FIELD-SYMBOLS:

      <SOURCE_FIELDS>    TYPE _ty_s_SC_1.

    DATA: MONITOR_REC     TYPE rstmonitor.

min_calday = sy-datum - 30.

    if SOURCE_PACKAGE is NOT INITIAL.

      SELECT *

        INTO TABLE t_material

        FROM /bi0/pmaterial

        FOR ALL ENTRIES IN SOURCE_PACKAGE

       WHERE material eq SOURCE_PACKAGE-material

        and  objvers EQ 'A'.

      SELECT *

         INTO TABLE t_matunit

          FROM /bi0/pmat_unit

          FOR ALL ENTRIES IN SOURCE_PACKAGE

          WHERE material eq SOURCE_PACKAGE-material

          and objvers EQ 'A'.

   endif.

    select *

      into table T_/BI0/PCOMP_CODE

      from /BI0/PCOMP_CODE

     where COUNTRY eq 'PK'

       and OBJVERS eq 'A'.

    loop at t_/BI0/PCOMP_CODE into wa_/BI0/PCOMP_CODE.

      wa_r_compcode-sign   = 'I'.

      wa_r_compcode-option = 'EQ'.

      wa_r_compcode-low    = wa_/BI0/PCOMP_CODE-COMP_CODE.

      wa_r_compcode-high   = ''.

      append wa_r_compcode to r_compcode.

    endloop.

    IF SOURCE_PACKAGE[] IS NOT INITIAL.

  REFRESH: t_primrate.

  DELETE SOURCE_PACKAGE[] WHERE comp_code NOT IN r_compcode.

  SELECT *

        INTO TABLE t_primrate

        FROM /bic/azsd_o01900

        FOR ALL ENTRIES IN SOURCE_PACKAGE

        WHERE salesorg EQ SOURCE_PACKAGE-salesorg

        and comp_code EQ SOURCE_PACKAGE-comp_code

        and division EQ SOURCE_PACKAGE-division

        and distr_chan EQ SOURCE_PACKAGE-distr_chan

        and calday GT min_calday

        and customer  EQ SOURCE_PACKAGE-/bic/zsd_dist

        and material EQ SOURCE_PACKAGE-material.

   ENDIF.

    SORT t_material BY material objvers.

    SORT t_matunit  BY material mat_unit objvers.

    SORT t_primrate BY salesorg   comp_code division

                       distr_chan calday    customer material.

ENDMETHOD.                   

METHOD inverse_start_routine.

ENDMETHOD.      

ENDCLASS.                                

Please free to ask me if you have any doubts regarding my issue.

Regards,

Harish

[anindya_bose]

Hi Haraish

First thing you should do is to check which part in DTP is taking time ...  Open DTP monitor and let us know which part is taking time...

1) start routine

2) Field routine ( Rules)

3) End Routine

4 ) Conversion of Char to SID....  

or Extraction from the DSO is taking time?

Cheers

Anindya

[former_member184884]

Hi Bose,

I've attached the screenshot for your reference do let me know if you need any inputs from my end.

Regards,

Harish

[anshu_lilhori]

Hi Harsih,

One thing is clear after seeing the above screen shot that issue is with the start routine only.

As this process is taking time of 4 hrs and 3 hrs so on you can also check the same.

I would suggest still to get your code optimized with some good Abaper and let them also know that this process is consuming time.

Also you can think of creating secondary indexes on the Dso fields which are not keyfileds and used in where statement.

Check out this document:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a068bb0a-daf5-2e10-f6b8-fc3076070...

Hope that helps.

Regards,

AL

[anindya_bose]

Ok Harish..  Good , now I can see where exactly DTP is spending time.

Overall this code was done in  inefficient manner . Instead of definition your internal table as TYPE of P table of Material , you should only take those fields which you need and only select those instead of using select *.

loop at t_/BI0/PCOMP_CODE into wa_/BI0/PCOMP_CODE.

      wa_r_compcode-sign   = 'I'.

      wa_r_compcode-option = 'EQ'.

      wa_r_compcode-low    = wa_/BI0/PCOMP_CODE-COMP_CODE.

      wa_r_compcode-high   = ''.

      append wa_r_compcode to r_compcode.

    endloop.

    IF SOURCE_PACKAGE[] IS NOT INITIAL.

  REFRESH: t_primrate.

  DELETE SOURCE_PACKAGE[] WHERE comp_code NOT IN r_compcode.

basically you are taking only those comp code for which country is PK ... Deleting anything else from the Source_Package..  right?

Either you should do this in DTP filter , if country selection is available.   Or, you should do this step before executing any other select statement.

So basically if you delete all other data from source_package where Comp Code is not in your range table, then "For All Entries" statement would select much lower data set for every select and would increase the performance.

Please let me know if my understanding is correct.

Cheers

Anindya

[former_member184884]

Hi Bose,

DELETE SOURCE_PACKAGE[] WHERE comp_code NOT IN r_compcode.

this piece of  code is not their earlier but which is added recently by one of my ABAPer colleague he told me it will improve performance so even i am aware of it does not make much sense because here source package is DSO means it is not common DSO, it is country specific DSO so obviously it will have country relevant data only.

Because he is core ABAPer i don't have option to listen his words.

as you are saying there is code needs to be change do let me know what part of code needs to change so that we can discuss in team and proceed accordingly.

Regards,

Harish

[anindya_bose]

Harish

Let me explain you in detail...

Part A:

select *

      into table T_/BI0/PCOMP_CODE

      from /BI0/PCOMP_CODE

     where COUNTRY eq 'PK'

       and OBJVERS eq 'A'.

This piece of code is selecting all company codes where Country is PK.

Part B:

loop at t_/BI0/PCOMP_CODE into wa_/BI0/PCOMP_CODE.

      wa_r_compcode-sign   = 'I'.

      wa_r_compcode-option = 'EQ'.

      wa_r_compcode-low    = wa_/BI0/PCOMP_CODE-COMP_CODE.

      wa_r_compcode-high   = ''.

      append wa_r_compcode to r_compcode.

    endloop.

This piece of code is filling up a range table ; only with those comp codes which has country = PK.

Part C :

DELETE SOURCE_PACKAGE[] WHERE comp_code NOT IN r_compcode.

This part is deleting any source data where comp code is not in range table; that means when Country for Comp Code is not PK.

so basically you are left with only those records which are for comp codes which has country as PK.

I am not sure if this is your business requirement . But if you use this Part A, Part B and Part C before issuing any other SELECT statement , that will decrease the total number of records fetched from database table and routine will finish faster.

Another thing you can look at...

SELECT *

        INTO TABLE t_primrate

        FROM /bic/azsd_o01900

        FOR ALL ENTRIES IN SOURCE_PACKAGE

        WHERE salesorg EQ SOURCE_PACKAGE-salesorg

        and comp_code EQ SOURCE_PACKAGE-comp_code

        and division EQ SOURCE_PACKAGE-division

        and distr_chan EQ SOURCE_PACKAGE-distr_chan

        and calday GT min_calday

        and customer  EQ SOURCE_PACKAGE-/bic/zsd_dist

        and material EQ SOURCE_PACKAGE-material.

   ENDIF.

how many records do you have in DSO zsd_o019.  What are the keys of this DSO? Just change the order of Where  like below...and see if you get some better performance.

SELECT *

        INTO TABLE t_primrate

        FROM /bic/azsd_o01900

        FOR ALL ENTRIES IN SOURCE_PACKAGE

        WHERE comp_code EQ SOURCE_PACKAGE-comp_code

          and calday GT min_calday

         and salesorg EQ SOURCE_PACKAGE-salesorg

        and division EQ SOURCE_PACKAGE-division

        and distr_chan EQ SOURCE_PACKAGE-distr_chan

        and customer  EQ SOURCE_PACKAGE-/bic/zsd_dist

        and material EQ SOURCE_PACKAGE-material.

   ENDIF.

You can also create Secondary Index on Comp Code for this DSO

Cheers

Anindya