- SAP Managed Tags
Executive Summary
This study compares the prediction accuracy of SAP RPT-1, Strava, and leading AI chat models (Copilot, ChatGPT, Gemini) for a real cycling route. Using historical data from 85 rides, each model predicted time and average speed for the next session. Results show that SAP RPT-1 delivered highly accurate, deterministic forecasts, outperforming general-purpose LLMs and coming close to Strava’s specialized algorithm. These findings highlight SAP RPT-1’s potential for precise forecasting in structured data scenarios. Once the model reaches General Availability, we will extend this analysis to real business datasets for even more impactful insights.
This article was created with the support of Microsoft Copilot, but the results are based on a real-world experience.
Introduction
Predicting performance in cycling is not just a sports challenge - it’s a fascinating analogy for forecasting in business. Just like companies predict production or maintenance needs, cyclists can try to anticipate their next ride’s time and average speed. In this study, we explored how different prediction models perform when estimating these metrics for a real cycling route.
We compared five approaches:
- Strava’s prediction engine, widely trusted by athletes.
- SAP RPT-1, a tabular model designed for structured data.
- Three popular AI chat models: Microsoft Copilot, ChatGPT, and Gemini.
- The actual ride result, serving as the baseline.
The goal was simple: predict the route time and average speed as accurately as possible.
Methodology
The dataset included 85 cycling routes recorded over four months, covering distances, times, speeds, and elevation:
Each model received the same prompt:
“I will give you a CSV file with all the data about my last 4 months with the bike. You should predict my route time and average speed during the route on the next Sunday. Label [PREDICT] identifies which data need to be predicted by you. Analyse and give me your expectations for this Sunday.”
We then compared:
- Predicted time (in seconds)
- Average speed (in km/h)
- Response time (how fast the model delivered its prediction)
Finally, we benchmarked these predictions against the real ride, which covered 53.6 km with 765 m elevation.
Results
The predictions from each model were compared against the actual ride, which covered 53.6 km with 765 m elevation. The key metrics analyzed were time (in seconds) and average speed (in km/h).
The simplified comparison shows clear differences in accuracy:
- Strava delivered the closest prediction to reality, with only minor deviations in time and speed.
- SAP RPT-1 performed impressively, producing deterministic results and staying very close to the real values.
- Gemini was fast and understood units correctly, but its speed prediction deviated more than others.
- Copilot struggled with consistency and unit interpretation, leading to larger errors.
- ChatGPT showed the biggest gap, especially in time prediction, due to misinterpretation of units.
These results underline the importance of structured models like SAP RPT-1 for numeric predictions and reveal the limitations of general-purpose LLMs when handling precise, unit-based data.
Conclusions
The comparison highlights clear patterns:
- Strava remains the most accurate overall, with minimal deviation in both time and speed. Its specialized algorithm and access to large datasets give it a strong advantage.
- SAP RPT-1 stands out as a powerful alternative. It delivered deterministic predictions, handled structured data effectively, and achieved results very close to reality. This shows its potential for numeric forecasting beyond sports—think production planning or predictive maintenance.
- Gemini produced fast responses and understood units correctly, but its speed prediction was less precise.
- Copilot and ChatGPT struggled with numeric interpretation. Copilot showed inconsistent outputs, while ChatGPT had the largest error in time prediction, likely due to misunderstanding units.
The charts you will include (bar chart, dispersion plot, and error chart) reinforce these insights, showing how SAP RPT-1 consistently outperforms general-purpose LLMs in accuracy and stability.
- Bar charts reveal how predicted times and speeds compare to actual values.
- Dispersion plots show the spread of speed predictions across models.
- Error charts make deviations in time and speed crystal clear.
Key takeaway:
SAP RPT-1 has demonstrated a clear advantage over current LLM models by effectively leveraging structured data to deliver more accurate forecasts. Once the model reaches General Availability, we will be able to perform even more precise comparisons using real business datasets, unlocking its full potential for enterprise forecasting scenarios.
Annexes
To provide full transparency and context, this section includes screenshots of the responses generated by each model during the experiment. These captures show the raw outputs based on the same prompt and dataset, highlighting differences in interpretation, numeric handling, and prediction logic.
- SAP RPT-1 output:
- Strava Forecast:
- Microsoft Copilot output:
- Chat GPT output:
- Gemini output:
- Actual ride data from Strava:
Bluesky| User | Count |
|---|---|
| 36 | |
| 25 | |
| 17 | |
| 16 | |
| 16 | |
| 15 | |
| 13 | |
| 13 | |
| 12 | |
| 12 |
