How to get file list in a application server in background mode without performance issue??

[chuangmateo]

Hi all:

I know there are many topic about this but I just cannot find a solution without performance issue. In my case, everyday we received thousand .csv file from vendor and use a background job to get these file name from server and then use open dataset to get data from every single file, it works fine but has performance issue to get file list.

We use FM RZL_READ_DIR_LOCAL to get file list, but it need more than 30 minutes to get the file list !!?? I also try FM EPS_GET_DIRECTORY_LISTING has the same performance issue. And yesterday I try method cl_gui_frontend_services=>directory_list_files , this method only spent less than 10 seconds to get the file list but unfortunately this method cannot use in background mode.

Since method cl_gui_frontend_services=>directory_list_files only use 10 seconds but other 2 FM use 30 minutes, it must has something wrong!!?? Just wondering if someone know anyway I can get file list in background mode without performance issue, thanks.

Matt

[matt]

How many files do you have in the folder? Do you ever remove any, or move them to an archive folder?

[chuangmateo]

Hi Matthew:

Thanks your responds,

1. As I mentioned "everyday we received thousand .csv file", about 2,000 files daily.

2. Sure, we remove files from the folder at the end of background job.

Thanks.

[DoanManhQuynh]

sorry im little confused cause i think cl_gui_frontend_services=>directory_list_files can only get file from presentation server ( i tried with my client but it unable to get list of file from application server).

if you run RZL_READ_DIR_LOCAL from SE37, is it also take 30 mins+ too?

[matt]

Badly fragmented disks? Full disks? Problems with ACL? You should ask your Basis team.

[chuangmateo]

Hi Matthew:

Thanks again, but as I mentioned, use cl_gui_frontend_services=>directory_list_files in foreground to read the same directory as

RZL_READ_DIR_LOCAL did it just needs less than 10 seconds.

Matt

[matt]

Bit of a mystery then. Perhaps you can and/or basis can run a trace and figure out what's going on.

[chuangmateo]

Hi Quynh:

Yes, run RZL_READ_DIR_LOCAL from SE37 directly take 30+ minutes too.

In foreground, I tried use both RZL_READ_DIR_LOCAL and cl_gui_frontend_services=>directory_list_files to read the same folder and then use the same program which use open dataset command to get data from .csv file, both works fine except the performance difference.

Matt

[RaymondGiuseppi]

• Have you access to a FTP server allowed to browse this directory, if yes you could use FTP command to get directory content?
• Else did you (or basis) rise an incident/message to OSS for performance (even if the FM is not released)

[chuangmateo]

Hi Raymond:

Thank you, when I posted this question here I also raise an incident too, I'll see how SAP response. And yes, FTP is another way but only if I get no good answer from SAP, since there is FM/BAPI can get the file list then I'll try not use FTP (because internal flow/basis/following maintainance....), thanks.

Matt

[Tomas_Buryanek]

Another way could be list files with "ls" command.

[chuangmateo]

Hi Tomas:

Thanks, I might try this way if I cannot get good answer from SAP.

Matt

[chuangmateo]

Hi Tomas:

Thanks again, SAP suggest use command (SM49) and there are existed command LIST_DB2DUMP. I tried it in SM49 and runs around 3 minutes for about 73,000 files. Even is not as good as method cl_gui_frontend_services=>directory_list_files but is far good than we are now (30+ minutes).

Matt