Service Layer Data Transfer Workbench... Imports just got a lot faster

[JoergAldinger]

As part of a large Business One project we had trouble using SAP's Data Transfer Workbench because of the mere quantity of data involved. Memory management in DTW is not optimal and after several hours of running consumes almost 100% of RAM, slows down extremely and eventually stops responding altogether.

Meet the solution: Since the project was based on SAP Business One version for SAP HANA, we were able to leverage the possibilities of the Service Layer to import data massively and with minimal memory footprint on the machine importing the data.

To illustrate the difference, here are a few key pieces of information:

  SAP B1 DTW CEO SLDTW

Technology used	COM API (DI API)	B1 Service Layer (HTTPS)
Libraries required	.NET Framework, DI API, MS Access DB Engine	.NET Framework only
Memory Consumption on client	Starts at 200MB, grows to several GB after a few hours	110 MB steady, running 4 threads
Import speed	Starts at approx. 3 records per second, eventually slows down to 0	12 records per second, running 4 threads

Here is a screenshot of our first version, running on an uncertified test server with much lower CPU power than a certified server:

Note that we also decided to not use any of the Microsoft ODATA libraries as suggested by the B1 solution architects. We instead built our own communication procedures using plain .NET functions.

Please note this tool was developed for our own in-house use and is currently not available for download or distribution.

Looking forward to your comments!

Thanks,

Joerg.