Huge data dump and getting error

[Former Member]

Hi, I have webi report tring to pull very large data with 70k materials and fails with below error. The req is really users want a master data dump in excel so different users can do their analysis in SCM or OTC or PTP. when I selected a few material it doesnt fail I know it a very large data to handle but user requirement has this report running weekly and scheduled to sharepoint site.

This is coming off of a single BEx query, User can not slice and dice the data or receive different reports they want all data dumped.

Please provide any inputs as this is a priority task, how can I do this in webi or any other BI tool etc

Thanks in advance,

Kris

[Former Member]

Hi Krishna ,

It shouldn't be a problem to fetch ~5 L rows (5L*50 data cells) . I hope you have configured APS split and assigned needed memory to Webi APS .

as a workaround , in webi add multiple queries based on key dimension (lets assume Material Type) . add one query for each Material Type and in Webi have different tabs for each material types .

[Former Member]

Webi isn't an ETL tool, it's a reporting tool.

The real solution is to establish the reporting requirements and build some Webi reports with drill/input control functionality and similar but that probably isn't what you wanted to hear.

If the users are messing around in Excel and want to pull everything out of MM, isn't it time to consider some properly defined Bex queries rather than one giant data dump?

In the first instance, establish which columns can be chopped because they are duplicates/not needed and see what you are left with and if that's manageable.

[Former Member]

Hi Mark, user says that they need all columns and there are no duplicates. Would you suggest within Webi an advance technique or Business Objects?

Thanks again for your comments

Kris

[Former Member]

Sounds like you're on a complete hospital pass, as they call it in rugby.

So they want 100 columns and how many rows?

Then they want to analyse it?

I've not reported against BW for five years, so probably not the best person to advise you but I'd be looking at what hierarchies there are for drilling, looking at report filters, input controls and things like that. That said, if you can't get the data out with a single Bex query then there's a bigger issue than how to present it.

The key, as I said, is finding out what they want to do with the data - Webi is no ETL tool but it does ad hoc analysis very well.

[Former Member]

Hi,

100 columns, 45000 materials 45000 rows so we are looking at 100*45000 = 4.5M data cells

I discussed if we can use BODS and SQL serv staging database but the answer was a NO. Not sure how to deal with this

Any responses are appreciated

Kris

[Former Member]

Hi Kris,

try splitting your report in many reports by filtering it so each one of the new reports holds a subset of your data.

Then schedule it to the desired format.

Regards,

Rogerio

[Former Member]

Hi Rogerio they need a single report so splitting wouldn't be an option

Thanks,

[Former Member]

Other than asking for more memory for your machine, I'm not sure what they expect. Are they asking you to have 4.5M data cells in memory, plus the associated calculations and manipulations that they'd want to do with them? Sounds like you might be better exploring some sort of SSAS option and put BO over it if you've got that option.

[Former Member]

Thanks for the response Mark, unfortunately we do not have SSAS on the landscape. I am still trying to find out an option (if any at all) in BO

Thanks,

Kris

[Former Member]

Hi,

didf you try scheculing it ?

Regards,

Rogerio

[Former Member]

Unfortunately scheduling fails too Rogerio.

[arijit_das]

This error is coming from BW end. Why do you need to fetch huge dataset in WebI ? What kind of analysis are you trying to do with this big volume of data ?

[Former Member]

Hi, the usage of this is for many teams, many areas so that master data is no longer pulling manually from ECC. from user's point it can not be split up as it lose its value. They do not find receiving 4 reports merging them useful.

Please propose if there are any webi workaround or other solutions.

Thanks,

[former_member45129]

Please check this link

It may be due to temporary Table space. Talk to your Basis Team they can provide suggestion !

- Selva

[Former Member]

Thanks Selva we got BASIS to increase memory to 8G and still it fails. This is actually a Material master data report with plenty attributes of Material and other dimensions aka 'number of columns' almost a 100 which seem to be the issue

Also BEx query/RSRT, Analysis run the query okay however the query didnt have all 100 cols as in Webi (webi has the attributes expanded in Material r Material number dimension apart from other dimensions)

Please provide your inputs

Kris

[former_member45129]

Please split your report based on the key atttribute ,so that you wont miss the data.

In this way load on webi will be less and it wont create error as well

- Selva

[Former Member]

Hi,

Do you mean split the report into sub-reports or have multiple queries and create one report, the actual requirement is that they need everything in a single report.

Can you explain the approach Selva

Thanks,

Kris