How to generate SAPUI5 tests with IA tools. Joule Vs Copilot

[Clement]

Objectives

In this blog, we are testing AI tools to write our SAPUI5 unit tests! Wouldn’t it be wonderful to no longer have to do this tedious task?! Our hopes are high… The goal is to provide a SAPUI5 application controller file and ask an AI to generate unit tests for each of the controller’s functions.

The application used for this test is the demo app “Manage Products” available here: https://sapui5.hana.ondemand.com/sdk/#/demoapps

                     Application used for this blog     

We are using this test to compare the relevance of SAP tools and market tools in this unit test generation exercise. Two techniques will therefore be compared :

• Using SAP tools: SAP Build Code with Joule.
• Using market external tools: VS Code with GitHub Copilot.

SAP Build Code with Joule

Prerequisites :

• Configure SAP Build Code ⇒ Add SAP Build Code to your BTP.
• Import the “Manage Products” sample application.

The Joule Assistant is integrated into SAP Build Code as an extension accessible from the left menu. It's a chat where you can ask Joule for help on your SAPUI5, CAP, and ABAP Cloud projects. Remember to be courteous in your requests, we don't know how far AI will go in the future…

Joule extension of SAP Build Code

Now that the sample application has been imported into SAP Build Code, we can ask the Joule assistant to help us complete some files. It can read the application's various files and is generally quite good at writing code.

Let's try the following prompt to generate our unit tests: /ui5 Write me the unit test files for the controller Home.controller.js.

/ui5 create unit test file for #Home.controller.js write the test for every method in the controller.

 A specific syntax can be used to frame Joule :

• / at the beginning of your prompt allows you to choose its action.

ex : /fiori-gen-spec-app Generate an SAP Fiori elements application.

•  # allows you to reference a file from context.
• The documentation is available here. 

 The magic happens! Joule suggests a QUnit code with some explanatory comments. You're free to accept this suggestion or make adjustments with additional prompts. When you accept, the code suggested by Joule is automatically added to a test file in the SAPUI5 project.

   Response containing the unit tests generated by Joule.

 By pressing “Accept” the file is added to the project.

Check the file naming for me, I had to remove the .test, or specify it in the initial prompt. Without further ado, let's start the unit tests ! npm run unit-tests

Results: 4 out of 6 tests passed. It's not perfect, let's see what mistakes Joule made:

• Lack of mocks for the view or component: Joule calls methods like this.getView() or this.getOwnerComponent() without ensuring that these objects have been mocked in the test environment, resulting in errors like "Cannot read properties of undefined."
• Some function names to be tested needed to be renamed, which is relatively easy to fix.
• Poor test environment management: The lack of sandboxing or cleanup between tests can lead to interference, side effects, or unreliable tests.
• Incomplete test structure: Often, a beforeEach statement to initialize the controller or an afterEach statement to restore stubs is missing, compromising test independence.

Despite these few errors, generating the file still saved me a lot of time, because correcting them is quick and easy.

VS Code with GitHub Copilot

Now, let's try the same exercise in VS Code using GitHub Copilot as the AI ​​tool. Do you think it will do better than Joule ?!

Prerequisites:

• Install Visual Studio Code + the “Remote Access for SAP Business Application Studio” extension ⇒ Access SAP Business Application Studio as a remote from Visual Studio Code.
• Install the “GitHub Copilot” and “GitHub Copilot Chat” extensions.
• Import the “Manage Products” sample application.

These extensions add a chat to the Visual Studio Code interface to communicate with Copilot.

Copilot acts on the currently open file. So we open the controller before giving the prompt to the AI: Write the unit test for this file.

Copilot will put little dots on the files it is going to modify and if you press Keep it will really be included in the project, check what the AI ​​writes, it can be wrong.

• Forgetting to include sinon.js: sinon.js is a JavaScript library used to create stubs, mocks, and spies in unit tests. These tools are essential for isolating the code under test and simulating external behavior. Omitting sinon.js means that Copilot didn't properly configure the test environment, which could have led to errors or incorrect test results.
• Error with the OData model verification test: SAPUI5 applications often use OData models to interact with data services. A unit test verifying the OData model must ensure that data is correctly retrieved and manipulated. Copilot's error in this area suggests that it misinterpreted or miscoded how to test interaction with the OData model. This could include errors in how it simulates responses from the OData service, verifies received data, or tests methods that interact with the model.

Copilot is also less effective with the SapUI5 framework. It makes more errors with models. I haven't tried it for comprehensive tests like OPA5, but having an AI model trained for that purpose, like Joule, seems to be more effective than Copilot, which is too general.

Conclusion

	Pros	Cons
Joule	Directly in BAS the commands to be added before the prompts allow a better result. The integration into BAS.	Copilot's user interface is better. The possibilities seem limited even though we are only at the beginning of Joule. The cost of Joule is higher, it is difficult to estimate an average price, but you can calculate it on SAP discovery center.
Copilot	The interface in VS Code is really well designed to understand what the AI ​​is doing with the two tabs Chat and Edit. Copilot's handling is its big strong point. All the tools related to copilot you can find in VS Code.	The model is a bit too general in JavaScript and makes errors on SapUI5.

Copilot and Joule aren't perfect yet; they currently make a lot of mistakes. But with a little time to get used to it, these AIs can really save you time on less interesting coding phases.

Personally, I prefer Copilot for now. But Joule is worth keeping an eye on, as it's new, unlike Copilot, which has two years of experience, and I have no doubt that with time and SAP engineers, it will become very efficient.

And if you want to do these tests quickly and more flexibly, you can test with ChatGPT and Mistral by copy-pasting the code into their browser alongside.

Feel free to test and share your experiences with these AI assistants, you may get different feedback from me, I'm still new to SAP and these tests are really simple, I'm going to try to get them working on more complex tasks.