cancel
Showing results for 
Search instead for 
Did you mean: 

Rationale not to use Auto model 56 for all products

Former Member
0 Kudos

I could not understand why we should not use Auto model 56 for all Products .

I understand Auto Model 56 - checks for seasonality ,checks for trend , checks for white noise .

Lets say in what case I should use Model35 - seasonal linear regression rather than Model 56 .

Assumption: Performance is not an issue . I can afford the extra performance load on system by Auto 56

Accepted Solutions (0)

Answers (2)

Answers (2)

Former Member
0 Kudos

You should use Auto56 when you have ABSOLUTELY NO CLUE about the pattern of history. i.e. you do not know if has a trend and you do not know if it has any seasonality and you don;t even know if there is some level or is it complete noise!.. EVEN after visual inspection.

Auto56 is NOT recommended for mass processing. Use only in interactive mode to find what model system proposes and why (read the forecast messages tab.. some of it might be greek but you can guess or google). Try various iterations or values while in the forecast screen, until you are happy with the model proposed and then assign that model to the selection. 

It is also not recommended when MAD or MAPE ALONE is not the basis for acceptance of the forecast results.

In normal situations you might like to conclude this is good for sporadic demand but that has another model. Auto 56 is most cases might results in a sporadic model or a constant model and that kind of makes sense!.

Hope this is some answer

Regards

BS

Former Member
0 Kudos

SLR35 might seem like giving low MAPE for MOST of your CVC's but remember that it LOOKS for trend even when there is little or no trend. So you may be misled by low MAPE or MAD whereas in reality a constant model might have generated a more acceptable forecast. You need to read this in conjunction with outlier correction (sigma) factor you are using.

As you see there are no easy straight answers here. There is significant one time effort needed to fine tune the model parameters and you still need to review them. Review frequency can be parameterised, depending on life cycle days the product lived in your supply chain.

Former Member
0 Kudos

You should use Auto56 when you have ABSOLUTELY NO CLUE about the pattern of history. i.e. you do not know if has a trend and you do not know if it has any seasonality and you don;t even know if there is some level or is it complete noise!.. EVEN after visual inspection.

Auto56 is NOT recommended for mass processing. Use only in interactive mode to find what model system proposes and why (read the forecast messages tab.. some of it might be greek but you can guess or google). Try various iterations or values while in the forecast screen, until you are happy with the model proposed and then assign that model to the selection.

Thanks BS for the detailed responses - really appreciate.


I am more than happy to lead the effort of figuring out the underlying patterns. Initially I had trouble understanding messages in interactive run but I understand them in most cases.


Lets say I have "PRODUCT_ROCKET" - which has intermittent demand. Auto Model 2 finds that and applies croston model


Lets say I have "PRODUCT_CHRISTMASTREE" - which has seasonally - Auto Model 2 finds the seasonality and my results show seasonal


In these cases what is the advantage of trying to figue out underlying pattern s-find these sets of skus and assign different models when auto model is giving the same result.


Also any ideas why is auto 2 not recommended in mass processing ?


Thanks fr any further insights

Former Member
0 Kudos

Hi Siva,

Good questions.

Forecast Strategies 50 to 56 in DP do more or less the same thing. In fact 50 and 56 is very similar.

These strategies are applied based on what you THINK and what you KNOW re the trend and/or seasonality in the history.

The table below is not my creation. I added my remarks for clarity sake.

I would say Auto models should NOT be applied in mass because it is very hard to correct the forecast if the strategy has gone stale after sometime. It is good to use them in interactive mode and see what model the strategy proposes, why and what MAPE or MAD and then assign that model manually for background processing. This table is not my IP. I dug into my 2007 disk wheresome notes I made from a expensive class I attended. I cannot vouch for its correctness 100%. Let other experts comment on this.

Strategy #

Forecast Strategy (Sub-methods)

When to Use

When Not to Use

Drawbacks

50

No knowledge of patterns of history

When there is A trend and seasonality that you can decipher

Proposes Constant Model if no pattern is detected

51

When you THINK history has trend and no seasonality

When someone else knows that there is seasonality

Proposes Constant Model if no trend is detected, else a trend model is chosen

52

When you THINK history has Seasonality and no Trend

When someone else knows that there is trend

Proposes Constant Model if no seasonality is detected, else seasonal trend model is used

53

When you THINK history has Seasonality OR a Trend pattern

54

When you THINK there is Trend and when you KNOW that there is seasonal pattern

55

When you THINK there is Seasonality and when you KNOW that there is Trend in Historical data

56

No knowledge of patterns of history

No Recommended in Mass. Not recommended when MAD is not the basis of acceptance of forecast result.

Former Member
0 Kudos

Hi Sagdiyev - Thanks for the extra effort in digging out the table.


Forecast Strategies 50 to 56 in DP do more or less the same thing. In fact 50 and 56 is very similar.

I have some information to add on the above comment.You might already know.50=53 and 56 is slightly different becuase it continously optimizes the alpha beta gamma until it finds the best combination whih results in lowest MAD.

For 56 - you can also make it more thorough by changing the increment size of alpha beta gamma - I think its 0.1 and you can reduce this further . But please keep in mind that this might lead to overfitting .

As you said lets see if any experts have any comments on the table .

Former Member
0 Kudos

Yes, let the real statisticians and quant experts comment on this. I believe there are probably no more than 10 SAP APO DP consultants who understand the real stuff that makes tools like these worthwhile.

When I was not using SAP and I was generally a more useful social creature, I used excel and free java applets for generating high quality statistical forecast on Prof. Hossein Arsham's pages on time series forecasting. I then loaded that forecast to an MRP system that I built using Access and Excel. I wish I could sell it sometime to some takers :slightly_smiling_face:

Keep asking such economically useful questions.

emani_raghavendra
Participant
0 Kudos

Hello All,

Just to note some points -

> Automatic model is used when we dont know the trend of the historical data .

> Automatic model FIRST determines the best forecast model and NEXT ( after the best model was selected ) it optimizes the smoothing factors.

Activities carried out for Automatic model 50 selection :

> First the system checks for periods which do not have history. If this number is more than around 70% of the total no. of periods then in uses CROSTON Method.

> next, system looks for White noise, if test is positive , it uses constant method.

> if it fails in above both tests, then it will test for seasonal or trend

Automodel 56 :

The system calculates the models to be tested using various combinations for alpha, beta, and gamma. The smoothing factors are varied between 0.1 and 0.5 in intervals of 0.1. The system then chooses the model which displays the lowest mean absolute deviation (MAD).

Siva - Auto model is used when you have no idea about history pattern . If you some idea on pattern , yes obviously you can startegy 35 rather than 56. As per my understandings, Auto model startegies are used to test which strategy is system proposing as a best fit. Once you come to know the best fit strategy , you will start using that forecast strategy only. Beacuse, Auto forecast strategy consumes lot of time, and no one would like to spend that time .... may be ...

Regards,

Venkata Emani

satish_waghmare3
Active Contributor
0 Kudos

Hello Siva,

Based on my understanding of Automodel selection 2.

Yes, Automodel selection 2 produces poor/substandard forecast as it relies heavily upon many settings(like Periods per season) done and how correctly those are done. Mostly these configurations are done by consultants with no or minimal knowledge about data and statistical forecast modeling.

Having said above, I have noticed some issue like - Auto Selection 2 often uses Constant model – which provides good MAPE but poor linear future forecast.  I have read Automodel 2  produces different result when run interactively in PBook and when run in the background.

Before you select any forecasting model, you should analyze the (sales) data. More often following option helps : 1. Plotting Data, 2. Summarizing Data

As there are various data patterns like constant(Level), Trend , Seasonal , Seasonal Trend and sporadic history(Randomness), it is important to recognize the data pattern or demand pattern, then apply the suitable forecasting model.


However there is no systematic approach for the identification and selection of an appropriate statistical model, and therefore, the identification process is usually by trial and error(depends on heavily by recognizing the pattern and domain(industry) knowledge)

Here are some useful links.

1. Automatic Model Selection Procedure 2 - Definition/Redefinition of Forecast Models - SAP Library

2. Getting Best Fit to Work in SAP?

3.


Hope this will help.


Thank you

Satish Waghmare

Former Member
0 Kudos

Satish - As always ,thanks for good points and informative answer

"Yes, Automodel selection 2 produces poor/substandard forecast as it relies heavily upon many settings(like Periods per season) "


Periods per season is a super important setting - no matter what model we choose - auto or seasonal model.Even if we know seasonal pattern , even when we choose seasonal model - if we get the periodsper season wrong - then the results are inaccurate




  I have read Automodel 2  produces different result when run interactively in PBook and when run in the background.


I have read in some blog - but this is no more true may be applied to some old version and also it is little tricky. But in my testing - found no such issue



As there are various data patterns like constant(Level), Trend , Seasonal , Seasonal Trend and sporadic history(Randomness), it is important to recognize the data pattern or demand pattern, then apply the suitable forecasting model.


I somehow could not completely understand this - but think this is right statement . Answer to this kind of answers my question .


Lets say I have "PRODUCT_ROCKET" - which has internittemnt demand. Auto Model 2 finds that and applies croston model


Lets say I have "PRODUCT_CHRISTMASTREE" - which has seasonally - Auto Model 2 finds the seasonality and my results show seasonal


In these cases why do I have to run separate models for theseprodcusts which is obviously tough to maintain


I frequently keep referring to sap help in the hope of understanding something new - but always end up unsuccesful

satish_waghmare3
Active Contributor
0 Kudos

Hello Siva,

In the scenario mentioned by you, IF Automodel Selection 2 is able to recognize the pattern (like intermittent and seasonal) and able to apply the respective forecast model to generate the (satisfactory) forecast, THEN we can rely on Automodel Selection 2, which also means we do not need to maintain or run separate models.

Thank you

Satish Waghmare

Former Member
0 Kudos

Thanks Satish - I had no problems with that in my testing , I will post if I have new findings

satish_waghmare3
Active Contributor
0 Kudos

Just to make sure, please do conduct the test running it in Background and interactively.

This will help get the required answer and do post your findings.

Thank you

Satish Waghmare

Former Member
0 Kudos

Yeah -No discrepancies found in background vs interactive . Both tests passed

satish_waghmare3
Active Contributor
0 Kudos

Perfect. As per your findings -    Automodel Selection 2 is able to automatically select (both interactively and background) a (correct) forecasting model for products in the selection by following their history data pattern.

Thank you

Satish Waghmare

Former Member
0 Kudos

Correct - I still think there is some drawback to this which we havent uncovered. Otherwise I see no reason for SAP to provide all these different models separately .

SAP help says not to use Automodel unless you dont know the underlying patterns