2010 Jun 30 8:45 PM
Hi,
sorry for that strange subject.
I'm currently working for a customer with productive systems
that have > 80 productive clients. A few clients (in some cases
only one client) make up the majority of the table entries. E.g. one
or two clients together have 98 % of the table entries. The database
plattforms are either ORACLE, DB2 or MSSQL.
This scenario caused some headaches on some plattforms because
many SQL statements are using "strange" execution plans.
Now I'm wondering if such scenarios are used out there
in other places as well...
Kind regards,
Hermann
Hi,
sorry for that strange subject.
I'm currently working for a customer with productive systems
that have > 80 productive clients. A few clients (in some cases
only one client) make up the majority of the table entries. E.g. one
or two clients together have 98 % of the table entries. The database
plattforms are either ORACLE, DB2 or MSSQL.
This scenario caused some headaches on some plattforms because
many SQL statements are using "strange" execution plans.
Now I'm wondering if such scenarios are used out there
in other places as well...
Kind regards,
Hermann
2010 Jul 05 2:56 PM
Hi Hermann,
I didn't got the point. Sometimes we don't have an equal solution for the same problem when we got different scenarios like different databases.
Are you asking about how to deal with this heterogenous environment?
Regards, Fernando Da Ros
2010 Jul 05 3:04 PM
Hi Fernando,
ok, i was not precise enough.
First i was interested if such a scenario: (many clients, different unequal sizes) are out there on database plattforms like MSSQL or DB2.
Because i had some discussion on SQL tuning in such a scenario with different colleagues i would like to know how
other people tackle SQL tuning on such scenarios. Up to now i think it is not very common, or let's say i dind't found
many people with practical experience in such scenarios.
On the other hand for MSSQL there is a not available which exactly describes the pain we have seen, but for DB2
I'm still looking for some experience.
Kind regards,
Hermann
2010 Jul 05 3:23 PM
Understood.
Unfortunately, I'm just another guy with specific database kwowledge (Oracle).
When developing for SAP I'm try to keep code as ANSI as possible SQL's, with usage of index (the basic) the basic can be automatic enhanced by a good database.
On past when I develop client/server systems basically to Oracle and sometimes to MSSQL I face different internal treatments specially for tables which contains special types (CLOB, BLOB...), of course some problems became from different versions of Oracle.
I hope someother guy on forum have more knowledg in this "multi-database-environment" handling to bring some ligth to this topic.
Regards, Fernando Da Rós
PS: In parallel if you wish split in minor taks, may we should help to understand the MSSQL / DB2 pain
2010 Jul 05 3:34 PM
If I understand Hermann's problem, it's the usual problem with skewed data, but in this case the skewness occurs in the client field. Since this field is used in all SAP queries, it means that the optimizer estimations will always be wrong.
Hermann, do you have problems in the big clients or in the small ones? Or in both?
If mainly in big clients, would somehow forcing the statistics (2 distinct values for field mandt) help?
2010 Jul 05 4:13 PM
Hi Fernando,
Hi Rui,
yes, it is the different treatment in SQL processing in combination with the unequal distribution and the different statistics.
In ORACLE the default is always optimize with parameter markers (:a0, ...) and no data distribution knowledge. For this
scenario it usually is no problem since the client shows >80 distinct keys (is somehow "selective"). In other scenarious
this could lead to ugly problems as well.
In MSSQL the FIRST set of parameter markers is used to build a plan. The data distribution for the biggest 10 values for all columns is known by the optimizer. All other executions continue to use this plan until it is reparsed (with a different set of parameter markers). With this scenario you will get either an efficient plan for a small or a big client (whatever is executed first) and the follow up sql will use the plan which is usually inefficient for the opposite (e.g. big client if first execution was small client) of the first execution.
In DB2 the FIRST set of parameter markers is used to build a plan. The data distribution for the biggest 10 values for the FIRST column is known by the optimizer. All other executions continue to use this plan until it is reparsed (with a different set of parameter markers). With this scenario you will get either an efficient plan for a small or a big client (whatever is executed first) and the follow up sql will use the plan which is usually inefficient for the opposite (e.g. big client if first execution was small client) of the first execution.
Patching statistics is not as easy and allowed as on ORACLE. In MSSQL it is technically possible but strongly forbidden in productive environments. And i don't know how it could be done. In DB2 it might be possible but is usually not done as well.
Hints (e.g. recompile) are no option since we have the issues in almost all SQL statements and in SAP standard programs. Removing the big clients to stand alone systems... yes, this is seriously discussed. As well as options on statistics gathering (not storing the data distribution on the first column in DB2 = would be the same as on ORACLE), or maybe changing the indexes for some tables with bad statements and put the client in the 2nd position = can have unknows side effects on the clustering) or making the DBMS client aware (e.g. on MSSQL with client driver software and database interface settings), or ...
It is deffinatelly an interesting and challenging setup there. I would like to see DB4, DB6 and MAXDB as well on such a scenario which would probably make it even more interesting...
Kind regards,
Hermann
2010 Jul 06 8:47 AM
and an additional challenge is the "estimated" vs. "actual" execution plan thing.
E.g. on MSSQL in ST05 the plan looks ususally fine but the execution times are bad.
This is because ST05 shows the "estimated execution plan" based on the "new" variables
for a statement or based on parameter markers. This is what you would get after a reparse.
The actual execution however may use a different "cached" execution plan based on the
first parameter set. This one could be find in ST04 only.
ON DB2 it is similar, here it depends on where you click to "explain" a statement and you
have to check the flag "cached" which indicates if the execution plan is an estimation or
the actual used one.
2010 Jul 06 3:53 PM
Hermann, if I may I would like to use your thread to explore more this estimated vs actual plan thing (in Oracle).
In Oracle we also have this use-the-FIRST-plan logic, aka bind peeking. Usually bind peeking is regarded as something you don't have to worry about unless you have histograms: since without histograms no detail is known about the distribution, every value will result in the same explain plan. From what I understand, however, that is not entirely true.
In my system I have only one mandt - 121. These are all from ST05:
1) Bind variables:
select * from vbap where mandt = :a1
SELECT STATEMENT ( Estimated Costs = 53.072 , Estimated #Rows = 2.218.599 )
1 TABLE ACCESS FULL VBAP
( Estim. Costs = 53.072 , Estim. #Rows = 2.218.599 )
Estim. CPU-Costs = 14.782.624.065 Estim. IO-Costs = 51.100
2) "Correct" client:
select * from vbap where mandt = '121'
SELECT STATEMENT ( Estimated Costs = 53.072 , Estimated #Rows = 2.218.599 )
1 TABLE ACCESS FULL VBAP
( Estim. Costs = 53.072 , Estim. #Rows = 2.218.599 )
Estim. CPU-Costs = 14.782.624.065 Estim. IO-Costs = 51.100
3) Non-existent client:
select * from vbap where mandt = '000'
SELECT STATEMENT ( Estimated Costs = 1 , Estimated #Rows = 1 )
5 2 TABLE ACCESS BY INDEX ROWID VBAP
( Estim. Costs = 1 , Estim. #Rows = 1 )
Estim. CPU-Costs = 6.851 Estim. IO-Costs = 1
1 INDEX RANGE SCAN VBAP~0
( Estim. Costs = 1 , Estim. #Rows = 1 )
Search Columns: 1
Estim. CPU-Costs = 4.313 Estim. IO-Costs = 1
This happens because oracle keeps 1 bucket even without histograms. In this case, this bucket would be minimum 121, maximum 121.
So my doubt is: if, in oracle, there is a first query with mandt = '000' and then subsequent queries with mandt ='121', all these subsequent queries will use VBAP~0 even if FULL is better, right?
And how does this relate with substitute literals? How I think it works:
1) Without substitute literals, the first query (with 000) will be parsed, and the value 000 will be used to calculate the actual plan (so will use VBAP~0). Subsequent queries (with 121) will use this cached plan (and not a FULL).
2) With substitute literals, each query (with 000 and with 121) will have its own entry in the cache, so they will behave differently.
Comments?
2010 Jul 06 4:59 PM
Hi Rui,
On ORACLE 9i, 10g and 11g it is recommended to switch bind peeking (_OPTIM_PEEK_USER_BINDS) off. See SAP notes 124361,632556, 830576, 1431798. I think OPTIMPEEK_USER_BINDS was introduced in 9i (if i remember right) and it was recommended to switch it off immediatelly for SAP envrionments.
You are right with your comments how it would work if it would be switched on.
You can get inefficient plans if the first query was client '000' and the second one '121'.
That is one off the behaviour we see in MSSQL and DB2.
Since your description of what would happen is right, it is recommended to switch OPTIMPEEK_USER_BINDS off,
because we don't want that to happen.
(By the way even without the histogram bucket we have max and min value for each column statistic value e.g. in ORACLE 9i, so that values out of range may lead to different plans than values in range with substitute literals).
And in ORACLE the indicator whether you look at an estimated or actual used plan is:
Whenever you see something like:
Explain from gv$sql_plan: Address: ... Hash_value: ... Child_number: ... Instance_ID: ...Sql_id: ...
on the execution plan screen it is an "actual used plan" (ST04). Without that information it is an estimated plan (ST05)
Kind regards,
Hermann
2010 Jul 07 10:34 AM
looks like such scenarios seem not be very common or not known here.
I didn't expect that it was common i just thought i ask here to see if anybody
else knows of similar scenarios somewhere. I'll close the question.