Technology Blog Posts by Members
cancel
Showing results for 
Search instead for 
Did you mean: 
VictorHo
Participant
6,157

 

Starting from November 2025, SAP officially introduced a significant change to its certification journey: an open-book, scenario-based exam format. This new approach is currently in pilot for selected certifications, including the SAP BTP Solution Architect exam.

Details about format here: Reimagining certification in the age of AI

Last week, I had the chance to take this new exam for the certificate “SAP BTP Solution Architect” and I want to share my honest experience, especially for those preparing for it soon.

After reading the new update from SAP (above link), I learn some more details in a blog in SAP community:

How to Prepare for SAP’s New Certification — SAP Community

However, it’s still not very clear for me how it looks like in the actual exam. So I gave it a try. For the “SAP BTP Solution Architect”, it’s a “scenario-based” exam format, where you need to play the role of a solution architect, designing the solution for a use case, then pitch the solution to AI avatar, or record a video and upload for expert review.

Unlike the previous exam format, where you need to setup some software (private browser) to take a prottor-monitored multiple-choice exam, in this new format, you will start directly from SAP learning website. This is the first look at the new exam format.

(To have this view, you must have purchased the SAP Certification Subcription; I will share in another post)

VictorHo_0-1765709167091.png

SAP Certificate landing page, where you can start the exam

After pressing on the “Go to Certification Exam”, the website will open a wizard like dialog to guide you through the process, which is quite user friendly:

VictorHo_1-1765709165225.png

SAP Certification Exam — Scenario Based Assessment — AI Roleplay

For the exam of “Solution Architect”, you will be assigned to a use case, where you play the architect role to propose a solution to project stakeholders. The use case resembles the actual use case in one project.

For example, in my exam, I need to design a soluton for Field Service Maintainance Engineer to efficiently get and update data with S/4 systems, while they’re working at customer site.

After reading the use case, once you’re ready, you can choose 1 out of 2 options to proceed with the pitching part.

For the next step, you will have 2 options to join the xam:

  • AI Roleplay with AI Guidance.
  • Record Video with Human Review.

I tried both, and share my experience as below:

1. A Fresh Concept: Scenario-Based + AI Roleplay

The most exciting part of the new format is the “Roleplay with AI” feature. Instead of answering static questions, you interact with an AI avatar acting as your stakeholder, and you must propose and justify your architecture on SAP BTP.

This idea is brilliant — it mimics real project work and tests your ability to think, design, and explain like a true architect.

However… the reality wasn’t as smooth as expected.

2. Technical Issues: The AI Mode Was Not Stable

Despite an impressive concept, the AI roleplay mode repeatedly failed during my attempts. I tried everything:

  • Company laptop and personal MacBook
  • Corporate network, home Wi-Fi, mobile hotspot
  • Multiple browsers and system setups

Yet the final step always returned the same message:

“An error occurred. Please refresh the page and try again. If the issue persists, contact the support team for assistance.”

VictorHo_2-1765709165112.png

AI Roleplay option — error

The other option with Human review also didn’t work:

VictorHo_3-1765709165222.png

Human review option — error

Since SAP requires exam submission within 24 hours, the pressure was real. I opened a support ticket, but even after a full working day, there was no response.

3. The Backup Plan: Human Review

Fortunately, late in the day, one of my setups finally allowed me to switch to the “Traditional Assessment” option, with human review my recording. This required me to:

  • Record and upload a video explaining my architecture
  • Wait for a manual evaluation by an SAP expert

For this option, things are very “generic”. SAP does not provides any details on the expectation of the video. All you can do is to prepare the best solution, and present it, during the 2 hours period.

After submitting your video, you will get the result immediately with the AI Roleplace approach. But with the human review approach, you will need to wait for a long time to get the result. During this period, if you enter the certification page again, you will get the “In Review” status:

VictorHo_4-1765709164987.png

 

The downside?
 The waiting time for results is now 20 days.
 This delay is a huge gap in the expectation of exam taker in this fast-paced world.

4. Final Thoughts: Great Idea, but Needs Stability

I truly appreciate SAP’s direction. A scenario-based, open-book exam that simulates real project challenges is the right way forward. It evaluates real skills — not memorization.

But the instability of the AI mode currently holds back the full potential of this format. When candidates must fall back to manual review, the process becomes:

  • Slow
  • Costly
  • Frustrating
  • Uncertain

For exam takers, waiting nearly a month for results is a painful experience.

5. Looking Ahead

I hope SAP can quickly stabilize the AI roleplay feature. When it works, it will elevate the certification experience and set a new benchmark for professional exams.

The concept is powerful.
 The execution just needs more refinement.

And once it’s stable, I believe this format will truly redefine how SAP professionals prove their expertise in the future.

10 Comments