on 08-14-2015 8:39 PM
Hi all,
I am trying to implement the SAP Note below,
Question : In the Step 3 , it was suggested to add a condition in while loop $counter < $list_size.But where was the $counter value defined ??
After adding the below logic, The job is not running into the loop. Please advice.
Note 1825911:How to use wait_for_file function in a data services job to access bunch of files in a folder and process it one file at a time?
Resolution
1. Create a batch job in data services designer and declare the following global variables:
$file_list - varchar(1000), $list_size - int, $counter - int, $file_name - varchar(100), $file_list_temp - varchar(1000)
$file_list and $file_list_temp hold comma separated list of absolute paths to the files so the size should be big enough to hold the list. $file_name holds an absolute path to single file.
2. Create a startup script as follows:
wait_for_file('C:\Program Files (x86)\SAP BusinessObjects\Data Services\dsdata\*.txt', 1, 1, -1,$file_list, $list_size,',');
print('Number of file found = ' || $list_size);
print('Comma separated file list = ' || $file_list);
$file_list_temp = $file_list;
The script shows an example of where to look for the files. You can change it according to your system. In this case it looks for all txt files in the folder: C:\Program Files (x86)\SAP BusinessObjects\Data Services\dsdata. If the script runs properly, it will print number of files found and the comma separated list of files with their absolute path. You can tweak polling inteval and other options as desired.
3. Add a while loop after the script in the batch job. the while loop condition is: while $counter < $list_size
4. Add a script in the while loop as follows. It extracts a single file each time from $file_list using simple string functions. It prints the current file in process and remaining file list at the end.
$counter = $counter + 1;
print('While Loop Counter = ' || $counter );
#print(' comma index = ' || index($file_list , ',', 1));
if($counter = $list_size)
$file_name= $file_list_temp;
else
$file_name = substr($file_list_temp , 1, index($file_list, ',',1)-1);
$file_list_temp = substr($file_list_temp, index($file_list,',',1)+1, length($file_list_temp)-index($file_list,',',1));
print('Current file in process is ' || $file_name);
print(' Remaining file list is = '|| $file_list_temp);
5. Now you can add a data flow, use the current file for processing. You can also use $file_name to crate a flat file format to be used in data flow.
Probably not an issue for you now but I was using the same script. The issue is that you need to set the $counter variable to 0 before you step into the while loop
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I got bit confused with the solution, Please explain the solution again
I have to process 3 + files.
Do we need to the add script ( highlighted) with in the while loop.
$file_list = exec('cmd','dir/b C:\Program Files (x86)\SAP BusinessObjects\Data Services\dsdata\MYNAME.*.txt');
$ctr = 1;
$file = word($file_list, $ctr);
while $file is not null
***process the file***
$ctr = $ctr + 1;
$file = word($file_list, $ctr);
Can someone explain me the logic on the step 4 ( in step 3 it is mentioned as $counter < $list_size) in the SAP Note : 1825911
I tried implementing this SAP Note and seems it is not entering into While loop.
I am bit confused with this logic
$counter = $counter + 1;
print('While Loop Counter = ' || $counter );
#print(' comma index = ' || index($file_list , ',', 1));
4. Add a script in the while loop as follows. It extracts a single file each time from $file_list using simple string functions. It prints the current file in process and remaining file list at the end.
$counter = $counter + 1;
print('While Loop Counter = ' || $counter );
#print(' comma index = ' || index($file_list , ',', 1));
if($counter = $list_size)
$file_name= $file_list_temp;
else
$file_name = substr($file_list_temp , 1, index($file_list, ',',1)-1);
$file_list_temp = substr($file_list_temp, index($file_list,',',1)+1, length($file_list_temp)-index($file_list,',',1));
print('Current file in process is ' || $file_name);
print(' Remaining file list is = '|| $file_list_temp);
| User | Count |
|---|---|
| 3 | |
| 3 | |
| 2 | |
| 1 | |
| 1 | |
| 1 | |
| 1 | |
| 1 | |
| 1 | |
| 1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.