Hello SAPCommunity, Thank you again for continuing to journey with me during this series of AI assistant generated code using MCP servers and VS code.
If you got here and didn’t see the previous blogs, or are interested in reading them, here are two other helpful blogs; one for CAP Experiences from prompt to CAP: AI-Powered Development using MCP Servers and another for SAPUI5 Experiences from prompt to UI5: AI-Powered Development using MCP Servers - Part 2
This blog can be read in Spanish here -->Testing OData via AI assistant generated code - Español
For this blog, I wanted to leverage code that was already generated from AI tools. One of the most missed or incomplete features in some projects is to have complete testing scenarios. In my case, I went back to cline and asked to “Create a test for my vendor entity”. Very quickly, code started generating, it created a test folder with a vendor.test.js file. There was also an entry in the package.json file to run the command to run a number of tests as shown in the next image.
I didn’t specify anything about the structure or location of the vendor entity. The AI tool itself knew where to find the entity, the other endpoints of the service, it was able to find the metadata so that it could understand definitions, navigation and such. Below is displayed a partial file that was generated from the request.
Later, I asked to verify my vendor test because, sometimes the generated code is almost correct but not fully – based on previous experience from working on these scenarios. Therefore, the process started validating its own generated code and made some adjustments.
After it was validated, I was able to run the npm command to execute the tests. You can see that some tests passed, and some failed. While looking at the actual code, I noticed that some tests still had some issues (such as, how did it find html if the odata service returns json).
At the conclusion of the execution of these tests, the AI tool itself noticed these issues and it corrected itself. I was asked to agree to save some adjusted code and I said Ok.
After the code was adjusted a couple of times, then, I executed the test command again and this time all the tests were successfully executed and passed. Below are the sample output of the execution and a brief description of each of the tasks completed.
Notice that one of the tests, used actual data from the previously deployed sample csv file and it was used during these tests; again, I didn’t give any specific scenarios to test but the tool did pretty good at it.
The tests run are shown below.
Now, let’s modify something that was already generated. Let’s request a negative test.
The code base was updated as shown below.
After re-running the test, a new test scenario was added and executed.
Well, there you have it. The testing cases that were added, executed, updated, and re-executed. This little handy tool is very helpful for generating different types of tests for any OData service.
Let’s take it a step further. Let’s ask to generate a load test (not ideal in development since we only have 10 records but this could be a starting point for future scenarios).
Before asking to generate this load test scenario, I switched from Act to Plan in the options of cline. Thus far, I have asked to “Act” meaning to generate or execute, but since I am not sure how the stress/load test will be done, I decided to ask “Plan”, meaning, tell me how you will generate this task. I asked, “Generate a load testing scenario. Simulate n number of users. Simulate m number of minutes”. At this point, I want to simulate a load, and I want to simulate number of users for a duration of time in minutes – more of a real case event.
Cline thought for a bit and gave me three different options.
Since I am not familiar with any of the 3 options (I’ve only heard of JMeter but have not implemented it myself), I chose the first option. Feel free to change and/or test all three.
Cline generated a plan and some questions for me prior to running anything. Further, I told it how to run it by answering the questions posted by the tool. Again, it prompted me to save so I did. The expected code generated was there and I also expected an entry in the package.json file to be able to request an execution of the same.
It gave me these 2 options in the package.json file, however, I think I only need the one shown on line 22.
When cline tried running the choco install, it failed.
After a few re-tries from cline and getting a second opinion from chatgpt, I found out that this command failed due to too much data trying to be sent from cline. Apparently, there is a max limit of how much data can be sent. Ok, so going with the chatgpt recommendation on how to install this first, then going back to cline to complete the execution of the load test scenarios.
The suggestion was to close VS code, and re-open it with admin privileges. In my case, I can do this, however, some organizations may have this option locked down for some users.
After reopening VS code, I was able to execution the installation correctly.
Then, I asked cline to verify my load test before I execute it. It verified and generated this.
Then I opened a new terminal (insuring my service was still running) & the load test started
It ran for 1 min. It ramped up the number of users from 0 to10, then ramped down back to 0.
Honestly, not knowing what to expect, I am happy to report the output contains the number of users, execution time, key metrics, total requests, error rate and even the navigation response time. This is totally awesome!
Closing thoughts on this blog:
- I was able to generate a set of tests cases (without specifics)
- I was able to request to modify it and add a negative test (modifying what had been generated)
- I was able to leverage another AI tool to troubleshoot an issue with installation. Proof that one tool will not do it all
- I was able to validate different scenarios and most importantly, we can do all of this without writing a single line of code
What are your most/least favorite test cases?
Were you familiar with the different options for load tests?
What other tests would you recommend other developers to automate? (UI tests will be separate from this odata blog).
Thank you for following up and continuing with this journey.
Sergio Guerrero
#SAPChampion
Bluesky
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.