topic Re: May Developer Challenge - AI Services - Week 4 in Application Development and Automation Discussions

May Developer Challenge - AI Services - Week 4

noravonthenen — Thu, 30 May 2024 11:40:02 GMT

Congratulations! You made it to week 4 of the AI Developer Challenge! This week we will continue where we left of last week and train our regression model using the dataset we uploaded in week 3 and the Data Attribute Recommendation service (DAR).

Before we start, let me give you a brief intro to machine learning so that you know what kind of model we are going to train this week. Machine Learning is a subclass of AI that focuses on the development of algorithms and models that enable computers to learn from and make predictions or decisions based on data. Traditionally, there are three main types of algorithms: supervised, unsupervised, and reinforcement learning. Furthermore, there are also different types of machine learning models like regression, classification, clustering, and association rules. These models serve various purposes and are used in different types of machine learning problems. Regression models are models that are used to predict continuous values, such as predicting a person's salary based on their years of experience and education or predicting avocado prices based on region, type, and the other values in our dataset. For the regression model we will train this week, we are using a supervised approach, as we provide labeled training data - that is a dataset that includes the avocado prices in the training dataset from past purchases to learn from for future predictions.

There are different supervised regression algorithms that you can use such as Linear Regression, Support Vector Regression, Decision Tree Regression, Gradient Boosting Regression and many more. The Data Attribute Recommendation services uses a neural network for regression that seeks to minimize the mean squared error (MSE). If you want to learn more about the regression capabilities of DAR check out this blog post.

By using the GET List Executables request, you can get all the Executable IDs. We will use the Executable ID for the regression model: “340abda5-d605-452a-bf4c-169249aab362”
Assign the Executable ID (“340abda5-d605-452a-bf4c-169249aab362”) to the trainingExecutableIdRegression variable in your environment.
Use the POST Create Training Configuration Regression request to create a training configuration for DAR. This way DAR knows which algorithm and which dataset and dataset schema to use during training. The datasetSchemaArtifactIdRegression and the datasetArtifactIdRegression should have been assigned from last week. If not you can use the GET List Artifacts request to get the missing values.
Now you need to create a training execution using the configuration from step 3. Therefore, we will use the POST Create Training Execution Regression request. If the variable trainingConfigurationIdRegression was not assigned automatically, you can assign it in the Environment yourself. It is the ID that was returned in step 3.
The current status of your training execution is “UNKNOWN” and the target status is “COMPLETED”. The status might change to “PENDING” and should then change to “RUNNING” as soon as the necessary resources have been allocated and the training started. You can check the status of your training by using the GET Get Execution Regression request.
The output of the training will be a model artifact and the training status “COMPLETED”.
Use the GET Get Metric Details request to check your models performance! You need to change the Variable from trainingExecutionId to trainingExecutionIdRegression as that is the ID that is being populated in step 5.
- MSE (Mean Squared Error) is a common metric for measuring the accuracy of a machine learning model in regression problems. It calculates the average squared difference between the predicted and actual values, which tells us about the average magnitude of error in the model. A smaller MSE is more favorable as it suggests lower error magnitudes.
- MAE (Mean Absolute Error) is another metric used for regression models. It calculates the average absolute difference between the predicted and the actual values, giving us another perspective on the error magnitude of the model. Unlike MSE, MAE isn't sensitive to outliers as it doesn’t square the errors in the calculation. A smaller MAE is again more favorable, indicating less difference between predicted and actual values.
- MAPE (Mean Absolute Percentage Error) measures the average absolute percent difference between the predicted and actual values. It is usually used when you want to understand the error in terms of the relative size of the predictions. It is expressed as a percentage, and smaller MAPE values indicate a better fit of the model to the data.
- A feature contribution score is a measure that assigns the importance of each variable in making a prediction and provides insights into which features are most influential in the model. If a feature has a high positive feature contribution score, it means that the feature significantly contributes to pushing the model's output higher for a given prediction. Conversely, if a feature has a high negative score, it plays a significant role in pushing the model's output lower. Knowing which features are most important can help improve your model by focusing on those variables during feature engineering.
SUBMIT a screenshot of your model’s metrics like this:

Stay tuned for next week to deploy your model and run inferences to see your model in action!

Week 1 challenge

Week 2 challenge

Week 3 challenge

Re: May Developer Challenge - AI Services - Week 4

Hira — Thu, 23 May 2024 10:10:22 GMT

Hi @noravonthenen ,

Hope you doing well.

I followed the steps as define but I am getting below error on step 5. Would appreciate if provide guidance to resolve.

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Thu, 23 May 2024 10:25:46 GMT

Hi @Hira, your model has not been trained yet. How long has the status been unknown now?

Re: May Developer Challenge - AI Services - Week 4

Hira — Thu, 23 May 2024 10:35:06 GMT

Hi @noravonthenen ,

Almost 30 mins now.

Re: May Developer Challenge - AI Services - Week 4

Mikkelj — Thu, 23 May 2024 12:25:25 GMT

I'm facing the same issue and has also been waiting for 1½ hour now.

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Thu, 23 May 2024 12:45:34 GMT

@Hira, @Mikkelj I am looking into it!

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Thu, 23 May 2024 12:58:04 GMT

@Hira Can you please provide the landscape and the subaccount ID?

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Thu, 23 May 2024 13:43:09 GMT

@Mikkelj @Hira So sorry that is entirely on me! I mixed up the versions of the executable ID! We want to use the latest version 3 and the ID for that is: 340abda5-d605-452a-bf4c-169249aab362. Please try to start the training again using that executable ID!

Re: May Developer Challenge - AI Services - Week 4

MioYasutake — Thu, 23 May 2024 20:54:28 GMT

Hi @noravonthenen,

I am experiencing the same issue as @Hira at step 5. In the status details there's an error "Execution 'e8937ee4c1f0daee' could not be found". Can this be causing the issue?

For the environment variable "trainingExecutableIdRegression", "340abda5-d605-452a-bf4c-169249aab362" has been set.

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Fri, 24 May 2024 11:28:58 GMT

Your training is COMPLETED. Can you check again with GET Get Execution Regression? And then continue with step 7?

Re: May Developer Challenge - AI Services - Week 4

Vitaliy-R — Fri, 24 May 2024 18:31:39 GMT

Here are my results:

Please note I had to modify the name of the variable in the last step from `{{trainingExecutionId}}` to `{{trainingExecutionIdRegression}}` to avoid the Error 500, as `trainingExecutionIdRegression` is the name of the variable populated at the end of the step 4:

Going back to the results it was interesting to see the top 3 contributing scores (if I read the JSON properly):

37.7% Type
31.0% Region
17.4% Date (where the price - I just assume - has been changing over time)

Have a good weekend!

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Fri, 24 May 2024 19:03:05 GMT

Thanks for your comment Vitaliy! I tried to specify that in step 7 but thanks for making it clear! I will clarify that more in step 7 🙂

Re: May Developer Challenge - AI Services - Week 4

MioYasutake — Fri, 24 May 2024 20:21:56 GMT

Completed week 3. Thanks @noravonthenen and @Vitaliy-R for your assistance!

Re: May Developer Challenge - AI Services - Week 4

Alpesa1990 — Sat, 25 May 2024 12:36:45 GMT

Hi,

I get the following error.

does it happen to anyone else?

thanks!

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Sat, 25 May 2024 15:11:43 GMT

Your execution is still running so the model is not finished training. Please check again 🙂

Re: May Developer Challenge - AI Services - Week 4

Alpesa1990 — Sat, 25 May 2024 16:26:08 GMT

HI Nora!

The model has been running from yesterday. I didn´t have this error on my last check yesterday and today I have this error and still persist.

Thanks by your attention!

Re: May Developer Challenge - AI Services - Week 4

Vitaliy-R — Sat, 25 May 2024 22:17:26 GMT

Btw, because I was using VS Code with Postman extension, I did not know that the Postman application has this nice "Visualize" tab to have a nicer overview of the data from the response:

Re: May Developer Challenge - AI Services - Week 4

gphadnis2000 — Sun, 26 May 2024 06:38:38 GMT

Getting below error for step 1 please help.

Re: May Developer Challenge - AI Services - Week 4

M-K — Sun, 26 May 2024 18:00:40 GMT

Here's my submission for this week:

Re: May Developer Challenge - AI Services - Week 4

noravonthenen — Mon, 27 May 2024 08:01:27 GMT

have you refreshed your token?