Performance problem when creating XML-file

[Former Member]

Hi,

I want to create a XML-file with customer data from an internal table. This internal table has appr. 62000 entries. It takes hours to create the elements of the file.

This is the coding I have used:

loop at it_debtor.

at first.

• Creating a ixml factory

l_ixml = cl_ixml=>create( ).

• Creating the dom object model

l_document = l_ixml->create_document( ).

• Fill root node

l_element_argdeb = l_document->create_simple_element(name = 'argDebtors'

parent = l_document ).

endat.

• Create element 'debtor' as child of 'argdeb'

l_element_debtor = l_document->create_simple_element(

name = 'debtor'

parent = l_element_argdeb ).

• Create elements as child of 'debtor'

l_value = it_debtor-admid.

perform create_element using 'financialAdministrationId'.

l_value = it_debtor-kunnr.

perform create_element using 'financialDebtorId'.

l_value = it_debtor-name1.

.....

21 child elements in total are created

.....

endloop.

• Creating a stream factory

l_streamfactory = l_ixml->create_stream_factory( ).

• Connect internal XML table to stream factory

l_ostream = l_streamfactory->create_ostream_itable( table =

l_xml_table ).

• Rendering the document

l_renderer = l_ixml->create_renderer( ostream = l_ostream

document = l_document ).

l_rc = l_renderer->render( ).

open dataset p_path for output in binary mode.

if sy-subrc eq 0.

loop at l_xml_table into rec.

transfer rec to p_path.

endloop.

close dataset p_path.

if sy-subrc eq 0.

write:/ wa_lines, 'records have been processed'.

endif.

else.

write:/ p_path, 'can not be opened.'.

endif.

form create_element using p_text.

check not l_value is initial.

l_element_dummy = l_document->create_simple_element(

name = p_text

value = l_value

parent = l_element_debtor ).

endform. " create_element

Please can anyone tell me how to improve the performance.

The method create_simple_element takes a long time.

SAP release : 46C

Regard,

Christine

[christian_wohlfahrt]

Hi Christine!

There might be several reasons - but you have to look at the living patient, not only the dead coding.

You did not show any selects, loop at ... where or read table statements -> the usual slow parts are not included (or visible...).

Of course the method-calls can contain slow statements, but you can also have problems because of size (when internal memory gets swaped to disc).

Make a runtime analysis (SE30), have a look in SM50 (details!) for the size, maybe add an info-message for every 1000-customer (-> will be logged in job-log), so you will see if all 1000-packs are executed in same runtime.

With this tools you have to search for the slow parts:

- general out of memory problems, maybe because of to much buffering (visible by slower execution at the end)

- slow methods / selects / other components by SM30 -> have a closer look

- ...?

Maybe a simple solution: make three portions with 20000 entries each and just copy the files into one, if necessary.

Regards,

Christian

[Former Member]

Hi Christian,

The runtime analysis show that the database access is minimal (only 1,3%), ABAP takes 75%.

In sm50 we see that the memory usage is not that high so it would give problems.

I have debugged the job while it was running and noticed that oo-object

l_element_dummy = l_document->create_simple_element(…) takes a long time.

As you have said we decrease the size of the internal table to no more than 10000 entries and create so several XML-files. When more the execution time increases exponential.

Is there no other way to create a XML-file in SAP?

The stranges thing about this is when the program has ended (after a write-statement) it also takes a long time before the execution stops.

Regards,

Christine

[christian_wohlfahrt]

Hi Christine!

I don't know, if there are other tools, just have a look in the forum.

The increasing execution time can have (in general) two reasons:

- a read table is getting slow, because it's accessing a bigger table without key definition (sorted or hashed, not standard key)

- a file scan of the previously added statements is done (for checking the syntax or whatever) for every step.

You might make a closer analyses with the runtime analysis restricted to the method and logging more statements - maybe a simple improvement can be done.

Make the analysis at the end of a 'long runner' - otherwise it's not so easy to see the slow part.

Regards,

Christian

[Former Member]

Hi Christian,

This is what SAP has answered to my question:

"In release 46x there is no special support for

writing xml documents in a streaming fashion. There are only two ways.

The first is to use the DOM interface and then rendering the DOM into a string. This solution is very memory consuming if you want to create large documents. The second way is to use the concatenate statement to

append tags and values as strings directly."

So I changed my program. For each line of the XML-file I fill a field of type string and I transfer it to a dataset. As simple as that!

Many thanks for your help.

Regards,

Christine