cancel
Showing results for 
Search instead for 
Did you mean: 

My DSO Is getting to large?

former_member205400
Active Participant
0 Kudos

The BI Early Watch report came out and said one of my DSO shold be reduced.

I looked at it and here are some stats for 2010:

201001 3,484,151

201002 3,056,019

201003 3,274,405

201004 3,392,412

201005 18,627,126

201006 18,180,714

201007 20,523,631

201008 30,810,975

201009 59,565,650

201010 50,912,680

201011 80,746,518

201012 89,236,514

Total 2010 -> 381,810,795

You can see that it is pretty much inceasing every month.

So my questions are:

1) so what? ... is it bad that it's getting so large?

2) what are the rest of you doing when you get large DSO's ... sure ly you aren't creating a new DSO for every month when you experience this.

Thanks,

Mike

Accepted Solutions (1)

Accepted Solutions (1)

esjewett
Active Contributor
0 Kudos

Hi Mike,

I look at your numbers and my spidey senses start tingling. Your data volume appears to be growing exponentially. Why is that?

Given your current scenario I would say that you should think about semantic partitioning in the short term (months). But in the longer term (1 year) you should think about a re-architecture. If this growth rate keeps up you are going to be loading well over 200,000,000 records per month by the end of 2011 (probably more like 300 or 400 million, just eye-balling the numbers). If you extrapolate current load and activation times as well as space requirements, can your BW system and business process even handle this many records?

Ethan

former_member205400
Active Participant
0 Kudos

Ethan,

The growth has occurred becasue we have implemented some new products.

semantic partitioning ? How would I implement that?

We currently are not feeding this to a Cube nor do we have any queries on this DSO. So, we are currenlty just accumulating data in case we get future requrements.

Are there any limits to what kinds of sizes I can use in the DSO?

Mike

esjewett
Active Contributor
0 Kudos

Hi Mike,

I can see implementing new products causing the data load to increase in size in relation to the number of new products created. Do you expect new products to continue being created at the same accelerating rate? What sort of data is this?

Regarding semantic partitioning, that is just fancy-talk for exactly what you suggest originally - splitting up the DSO into multiple DSOs based on some semantic key, like time (fiscal period) or product (different ranges of products in different DSOs). In BW 7.3 this becomes a lot easier with the semantically-partitioned object (SPO), but before that it can be quite a pain.

The primary problem with DSOs that are getting as large as the one you have here is logistical. It should continue working, but it may start taking relatively longer and longer to load because index maintenance will get slower as the number of rows in the DSO gets larger. Eventually you will get to the point where the load will run so long that the job is likely to be interrupted intermittently by server issues. If the DSO is a write-optimized type DSO that would probably help alleviate the problem quite a bit as you will not need to activate the data.

You also can't partition DSOs at the database level, so if it gets really enormous you could run into deeper trouble, but you should talk to your DBA about the point at which this will become an issue.

Ethan

Answers (1)

Answers (1)

0 Kudos

Hope its an standard DSO.

For all data system creates SID(for data loading purose).

In BI7 DSO creates intermediate table between new and active log.We have to manually clean up this temporary table to avoid issues(forget the tech name of this intermediate table...better search)

Better create copy of the DSO,move the historical data to the copy DSO.

Check for the index status.

Thanks

TG