Technology Blogs by Members
Explore a vibrant mix of technical expertise, industry insights, and tech buzz in member blogs covering SAP products, technology, and events. Get in the mix!
cancel
Showing results for 
Search instead for 
Did you mean: 
0 Kudos
My requirement was to download the data of a big table that contains several million entries to a CSV file. Knowing the function module RFC_READ_TABLE lead me to believe that it would be an easy task. I'm familiar with Perl and the nice cpan module sapnwrfc.

But retrieving millions of data with one RFC call is not a good idea as it will consume all your RAM (in my case 8 GB RAM).

So I searched for a feasible way to retrieve the data. I found what I was looking for - of course on SCN. larsfasel proposed a solution for Python for the same requirement in an archived discussion.

Under the motto of "talk is cheap - we want some action!" here my working code:
# Meaning of ROWSKIPS and ROWCOUNT as parameters of function module RFC_READ_TABLE:
#
# For example, ROWSKIPS = 0, ROWCOUNT = 500 fetches first 500 records,
# then ROWSKIPS = 501, ROWCOUNT = 500 gets next 500 records, and so on.
# If left at 0, then no chunking is implemented. The maximum value to either of these fields is 999999.
my $RecordsCounter = 1;
my $Iteration = 0;
my $FetchSize = 1000;
my $RowSkips = 0;
my $RowCount = 1000;

# Open RFC connection
my $conn = SAPNW::Rfc->rfc_connect;

# Reference to function module call
my $rd = $conn->function_lookup("RFC_READ_TABLE");

# Reference to later function module call
my $rc;

# Loop to get data out of table in several chunks
while ($RecordsCounter > 0){

# Calculate the already retrieved rows that need to be skipped
$RowSkips = $Iteration * $FetchSize;

# Reference to function module call
$rc = $rd->create_function_call;

# Table where data needs to be extracted
$rc->QUERY_TABLE("/PLMB/AUTH_OBSID");

# Delimeter between columns
$rc->DELIMITER("@");

# Columns to be retrieved
$rc->FIELDS([ {'FIELDNAME' => 'OBJECT_ID'}, {'FIELDNAME' => 'SID'} ]);

# SELECT criteria
$rc->OPTIONS([{'TEXT' => 'OBJ_TYPE = \'PLM_DIR\''}]);

# Define number of data to be retrieved
$rc->ROWCOUNT($RowCount);

# Define number of rows to be skipped that have been retrieved in the previous fetch
$rc->ROWSKIPS($RowSkips);

# Function call
$rc->invoke;

$Iteration++;

# Data retrieved
if(defined $rc->DATA->[0]){

print "Fetch $Iteration\n";

foreach my $TableLine ( @{ $rc->DATA } ) {
print "$TableLine->{WA}\n";
}

}

# No more data to retrieve
else{

# Leave loop
$RecordsCounter = 0;
}

}

# Disconnect RFC connection
$conn->disconnect;

 
2 Comments
Labels in this area