DS/SAP - DATAFLOW - This is less than(or equal to) the virtual memory <1609564160> bytes available for caches.

[wagner_goncalves]

Hello,

my project is displaying the following error, when you generate a TXT record with 3 million.

I have a project that when executato DS goes to a Z table in SAP selects every record in this table (+ 3 million), and on DS it generates a TXT file separated by semicolons (;). But the DS is not getting serar this file with more than 3 million record, the DS displays the following error:

(12.2) 02-21-14 10:06:27 (2996:2952) PRINTFN: INFO - Definicao da $G_FF_Diretorio_OUT = C:\DS_Neogrid\Upload\QAS

(12.2) 02-21-14 10:06:27 (2996:2952) PRINTFN: INFO - Definicao da $G_FF_Diretorio_OUT = C:\DS_Neogrid\UpLoad\QAS

(12.2) 02-21-14 10:06:27 (5928:3040) DATAFLOW: Process to execute data flow <DF_TransfArq_035> is started.

(12.2) 02-21-14 10:06:27 (5928:3040) DATAFLOW: Data flow <DF_TransfArq_035> is started.

(12.2) 02-21-14 10:06:27 (5928:3040) DATAFLOW: Cache statistics determined that data flow <DF_TransfArq_035> uses <0> caches with a total size of <0> bytes. This is less

  than(or equal to) the virtual memory <1609564160> bytes available for caches. Statistics is switching the cache type to IN

  MEMORY.

(12.2) 02-21-14 10:06:27 (5928:3040) DATAFLOW: Data flow <DF_TransfArq_035> using IN MEMORY Cache.

Can anyone help me...

Thank you.

at.

Wagner

[Former Member · Accepted solution]

Wheres the error? There is no error i see here.

[wagner_goncalves · Accepted solution]

Kumar,

is not a error, it's a message warning, but to reach that point where the txt file is generated, hangs and does not display or another information.

have to increase the virtual memory of the server?

[wagner_goncalves · Accepted solution]

kumar.

to start the process in DS, the icon turns yellow and does not generate the TXT, if I run the same process to record 100 MIL execution is successful.

see the attached print.

[Former Member · Accepted solution]

Enable "Pageable cache" at the time of job run.

Right click dataflow > Enable pageable cache from drop down.

Thanks,

Arun

[wagner_goncalves · Accepted solution]

hello, Arun,

My dataflow is already with Enable pageable.

See the print:

has another setting?

att.

Wagner

[Former Member · Accepted solution]

Hi,

Whats the record count you are dealing with? Could you briefly explain the transformation process? Any lookups, joins etc? Are you able to push down some processes to DB?

Also could you give the degree of parallelism (in the above screen shot) to 5 and try?

Arun

[wagner_goncalves · Accepted solution]

Hi, Arun.

I have lookups and joins only a selection from a table in SAP.

we have the selection in the Z table, and then a Query.

follows the process:

Perform a Z program in SAP, this program saves 3 million record in a Z table, then started the DS via BAPI: BAPI_ISOURCE_DP_REQUEST

when running the DS, he read the Z table 3 million regristo by dataflows, passes through Query and generate a TXT file in a directory.

this works perfect if a table with 100,000 record.

the lock only happens when we run with 3 million records.

[wagner_goncalves · Accepted solution]

Error of the border of error of job.

(12.2) 03-06-14 08:41:54 (W) (1392:6492) SYS-170114: /JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

  Warning: Your system is running low on process virtual memory space. Available virtual memory is <22> megabytes.

(12.2) 03-06-14 08:41:55 (W) (1392:6492) SYS-170114: /JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

  Warning: Your system is running low on process virtual memory space. Available virtual memory is <22> megabytes.

(12.2) 03-06-14 08:41:56 (W) (1392:6492) SYS-170114: /JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

  Warning: Your system is running low on process virtual memory space. Available virtual memory is <22> megabytes.

(12.2) 03-06-14 08:41:57 (W) (1392:6492) SYS-170114: /JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

  Warning: Your system is running low on process virtual memory space. Available virtual memory is <22> megabytes.

(12.2) 03-06-14 08:41:58 (W) (1392:6492) SYS-170114: /JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

  Warning: Your system is running low on process virtual memory space. Available virtual memory is <22> megabytes.

Characteristics of our server.

It will be necessary to increase the virtual memory virtual?

at.

Wagner

[Former Member · Accepted solution]

I see 32 GB RAM, which is more than sufficient but your error shows 22 MB of virtual memory, which is very low. Yes, in that case you have to increase the virtual memory.

[wagner_goncalves · Accepted solution]

Ok, Arun, thanks a lot for the information. I go to request this procedure and I inform the result.

att. to wagner

[wagner_goncalves · Accepted solution]

Arun,

See if you can help me again, increase the virtual memory for:

But still features ERROR: but now with <24>.

error:

Warning: Your system is running low on virtual memory space process. Available virtual memory is <24> megabytes.

  (12.2) 03/10/14 11:03:57 (W) (6576:5836) SYS-170114: / JB_35_Neogrid_DimDemPrimaria/DF_TransfArq_035

you could tell me what is the ideal amount of virtual memory to run this process?

att.

Wagner

[Former Member · Accepted solution]

Hi Wagner,

I see in the screenshot the paging file size is not increased. Why don't you check the option automatically manage paging size for all drives.

Also going by the screen shot its not an error just a warning, you can ignore it as long as job runs.

Arun