Huge data dump and getting error

[Former Member]

Hi, I have webi report tring to pull very large data with 70k materials and fails with below error. The req is really users want a master data dump in excel so different users can do their analysis in SCM or OTC or PTP. when I selected a few material it doesnt fail I know it a very large data to handle but user requirement has this report running weekly and scheduled to sharepoint site.

This is coming off of a single BEx query, User can not slice and dice the data or receive different reports they want all data dumped.

Please provide any inputs as this is a priority task, how can I do this in webi or any other BI tool etc

Thanks in advance,

Kris

[Former Member]

Webi isn't an ETL tool, it's a reporting tool.

The real solution is to establish the reporting requirements and build some Webi reports with drill/input control functionality and similar but that probably isn't what you wanted to hear.

If the users are messing around in Excel and want to pull everything out of MM, isn't it time to consider some properly defined Bex queries rather than one giant data dump?

In the first instance, establish which columns can be chopped because they are duplicates/not needed and see what you are left with and if that's manageable.

[Former Member]

Hi Mark, user says that they need all columns and there are no duplicates. Would you suggest within Webi an advance technique or Business Objects?

Thanks again for your comments

Kris

[Former Member]

Sounds like you're on a complete hospital pass, as they call it in rugby.

So they want 100 columns and how many rows?

Then they want to analyse it?

I've not reported against BW for five years, so probably not the best person to advise you but I'd be looking at what hierarchies there are for drilling, looking at report filters, input controls and things like that. That said, if you can't get the data out with a single Bex query then there's a bigger issue than how to present it.

The key, as I said, is finding out what they want to do with the data - Webi is no ETL tool but it does ad hoc analysis very well.

[Former Member]

Hi,

100 columns, 45000 materials 45000 rows so we are looking at 100*45000 = 4.5M data cells

I discussed if we can use BODS and SQL serv staging database but the answer was a NO. Not sure how to deal with this

Any responses are appreciated

Kris

[Former Member]

Hi Kris,

try splitting your report in many reports by filtering it so each one of the new reports holds a subset of your data.

Then schedule it to the desired format.

Regards,

Rogerio

[Former Member]

Hi Rogerio they need a single report so splitting wouldn't be an option

Thanks,

[Former Member]

Other than asking for more memory for your machine, I'm not sure what they expect. Are they asking you to have 4.5M data cells in memory, plus the associated calculations and manipulations that they'd want to do with them? Sounds like you might be better exploring some sort of SSAS option and put BO over it if you've got that option.

[Former Member]

Thanks for the response Mark, unfortunately we do not have SSAS on the landscape. I am still trying to find out an option (if any at all) in BO

Thanks,

Kris

[Former Member]

Hi,

didf you try scheculing it ?

Regards,

Rogerio

[Former Member]

Unfortunately scheduling fails too Rogerio.