XML parser and memory

[eddy_declercq]

Hi,

I need to parse several (rather large) XML files and I'm doing this with the classic:

DATA:     pixml          TYPE REF TO if_ixml,
             pdocument      TYPE REF TO if_ixml_document,
             pstreamfactory TYPE REF TO if_ixml_stream_factory,
             pistream       TYPE REF TO if_ixml_istream.

*-- create the main factory
   pixml = cl_ixml=>create( ).
*-- create the initial document
   pdocument = pixml->create_document( ).
*-- create the stream factory
   pstreamfactory = pixml->create_stream_factory( ).

*-- create an input stream for the table
   pistream = pstreamfactory->create_istream_itable( table = xml_table2
                                                     size  = totalsize ).

*-- create the parser
   pparser = pixml->create_parser( stream_factory = pstreamfactory
                                   istream        = pistream
                                   document       = pdocument ).

....

FREE: pixml, pdocument, pstreamfactory, pistream, pparser, pnode.

Everything works fine for the first file, but with the second file (although a bit smaller) I get a mem_alloc_failed dump. It occurs to me that things keep piling up. This is strange since  I do a free of all objects. There doesn't seem to be a destructor method.

What can I do in order to solve this memory issue?

Eddy

[Former Member]

Can attach a screenshot of the dump? I'd like to see the exact line in your code where the dump happens.

[eddy_declercq]

Hi,

Not a lot to see, just a dump on SAP code:

Runtime errors

MEM_ALLOC_FAILED

Short text

Internally required memory is not available

What happened?

Resource bottleneck

	The current program "CL_IXML=======================CP" had to be terminated
	because
	a capacity limit has been reached.

....

1653	method if_ixml_parser~has_event_subscription
1654	by kernel module abkm_iXMLParser_HasEventSubscription.
1655	endmethod.

1656

1657	method if_ixml_parser~parse_event
1658	by kernel module abkm_iXMLParser_ParseEvent.
1659	endmethod.

1660

1661	method if_ixml_parser~parse
1662	by kernel module abkm_iXMLParser_Parse.
>>>>	endmethod.

1664

1665	method if_ixml_parser~num_errors
1666	by kernel module abkm_iXMLParser_NumErrors.
1667	endmethod.

....

Eddy

[Former Member]

Just trying to zero in on the problem here... Have you tried running only the second xml file independently? has it worked?

[eddy_declercq]

Parsing files one at the time doesn't give any problem. The problem is when one loops on all files in a dir.

There is no rocket science  since it relies on the code from BCCIIXMLT1 (where that code doesn't even free the objects).

[naimesh_patel]

Do you have the method DESTRUCTOR in class CL_IXML? If yes, Try calling this method before freeing the PIXML object.

[eddy_declercq]

Like said in the initial question, there is no destructor method.

Meanwhile I've found a workaround in order to avoid the problem.

[naimesh_patel]

Eddy De Clercq wrote:
Like said in the initial question, there is no destructor method.

What version you are on?

Meanwhile I've found a workaround in order to avoid the problem.

Do you wish to share with us?

[eddy_declercq]

Version of what?

Clearing the XML table  before the actual parsing instead of before reading the file.

[hendrik_brandes]

Hi Eddy,

why do you need the CL_IXML classes?

I have made very good experiences with the XSLT-Transformations ( not the Simple-Transformations ). Especially for filtering data, converting data. And: We have at least processed very huge files.

Kind regards,

Hendrik

[eddy_declercq]

Do you have a link to documentation to that?

[hendrik_brandes]

Sure: http://help.sap.com/saphelp_nw73/helpdata/en/fd/9d734b389211d596a200a0c94260a5/frameset.htm

Another good reference is this book: http://www.amazon.de/ABAP-Fortgeschrittene-Techniken-Tools-PRESS/dp/3836211513/ref=sr_1_1?ie=UTF8&qi... . Sadly, the first volume is not avaiable any more, because the XML-processing has been described very detailed over 2 chapters. (Sorry for the link of the german book).

Within your ABAP-Coding you just have to use the CALL TRANSFORMATION-Statement.

One big advantage of CALL TRANSFORMATION is, that you can pass ABAP-Values to your XSL-program and vice versa. E.g. you do not have to handle with documents at all. If you need to have a internal table as result of your XML-Processing, you can get it.

I think the integrated way for XSL-Transformations in ABAP is very smart. For example look at my blog http://scn.sap.com/community/netweaver-as/blog/2011/12/30/extend-the-sicf-logonscreen-with-xslt -- of course its not the same way, what you want todo, but it should give you a hint, about what you can expect.

Kind regards,

Hendrik

[Former Member]

Hi Eddy,

Are you using xmlparser->parse( ) or set_dom_generating( ) methods? These methods can result in memory issues. Avoid these methods and use only event based parsing for large files.

Regards,Varun

[eddy_declercq]

See my initial msg for the methods used. Examples of alternatives are welcome.

[Former Member]

I saw the initial message already, but it shows only dots '....' after step pixml->create_parser. So it is not clear if Parse( ) is causing the problem. I saw BCCIIXMLT11 is using both Parse() and set_dom_generating( ) . Solution is also in BCCIIXMLT11, but only follow the code when PA_EVENT = 'X' and PA_DOM = ' '.