topic Re: Bringing data through RFC in Application Development and Automation Discussions

Bringing data through RFC

Former Member — Thu, 29 Mar 2007 19:34:58 GMT

Hi ABAP gurus,

I have a RFC function module. It brings back data in an internal table from remote system. Now the problem is the data from the remote ststem is very huge, something like 1,00,000 records. So it takes a lot of time and bandwidth. Sometimes the program has to wait really long hous.

Please give me a solution so that the connection failure can be evaded. Because if we wait for 2-3 hours and there is a failure then all data is lost.

ABAPer,

Sonu.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 19:52:51 GMT

I'm sorry to say that I don't think that there is any solution....RFC depends both on connection and data amount....Maybe what you can do it's just include an extra parameter telling the RFC how many records to bring...For example....From 1 to 1000...One you got the data, you call it again with From 1001 to 2000...Not the best choise....But I can think of something better....

Greetings,

Blag.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:00:11 GMT

Hi,

Since you are dealing with huge volumes of data, I would suggest to use file base interface i.e. FTP file(s). It is not pretty solution like RFC solution.

Regards,

Ferry Lianto

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:02:43 GMT

Why are you bringing in so much of data through RFC? Why not use a file based interface? RFCs are useful only for instant communication for single and simple transactions. For bulk data, you need to employ interfacing techniques.

Also, another point to think about is about the process that is sending so much data. Can something be done so that it doesn't generate so much data or can the external system send individual records instead of sending it in one shot? Of course the overhead in that case is that your RFC will be called 100000 times.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:06:12 GMT

Another <u>unrecommended</u> solution I have seen implemented is to define a Z table in SAP (which will be there in the underlying database as database table and have the external application write data to it directly. But there is a lot of risk in this, that the data will not go through SAP's application layer and hence all validations, conversions etc will not happen. So that data has to be in its purest form to hit the database directly like that. There are some tools that will create a mirror database table of your Z table in another table schema and will automatically keep the two replicas in sync.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:08:09 GMT

Hi Ferry,

File Base Interface is a new concept for me. Are you asking to put the table data in a RAW file and transfer and then decode it back at my end?

Please throw some light on this?

Regards,

Sonu.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:11:32 GMT

Basically have your external system dump the data into a file in an agreed upon format, have that FTPed to SAP application server, you upload the file and process it.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:12:50 GMT

Hi Srinivas,

What do you mean here? I did not get the picture. Please tell me your solution, because its something new for me too. Probably it would help me out here.

Thanks,

Sonu.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:15:13 GMT

Hi,

You could ask the sender system to download the internal table as text file(s) and send via FTP to your system. Once the file(s) arrived, you will then upload the file to SAP and perform validation and so on.

Regards,

Ferry Lianto

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:15:56 GMT

Hi Srinivas,

Is it a one step thing, i mean my program to fetch data is running in background. So is it possible during the life time of the program. Can you guide me through this step?

Thanks,

Sonu.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:22:25 GMT

Okay one step at a time, what is your RFC doing? Is it logging into an external system's tables and getting the data? Is it calling an external API which in turn gets the data for you?

What Ferry and myself are suggesting is that the external system where the data resides should extract the data and put it on a file. That file needs to be FTPed to your application server. You can write a normal report program that uploads the file into internal table and processes it. Look at OPEN DATASET, CLOSE DATASET and READ DATASET commands.

You can then schedule this program to run periodically to pull the file and process.

Re: Bringing data through RFC

Former Member — Thu, 29 Mar 2007 20:28:02 GMT

Hi Sonu,

In addition to Srinivas suggestion, please check this link for sample codes on how to deal with file in SAP both for PC and application server.

http://www.sapdevelopment.co.uk/file/file_updown.htm

Regards,

Ferry Lianto