Test Sequences with Focused Build Test Steps

[TobiasMeinzer]

In my recent blog posts I described Focused Build Test Steps for manual testing with SAP Solution Manager, how you can benefit from special functions like the Testing Mode in a 4-eyes-principle use case or the automated translation of test cases.

In this blog post I will describe how you can use a new feature for Test Sequences which was introduced with Focused Build SP05.

With Test Sequences you can put test cases into a defined order and make sure that testers execute them exactly in this sequence. This is typically helpful when it comes to End-2-End scenarios covering several processes which need to be tested one after the other.



In theory, this functionality gives guidance to your testers and simplifies the test execution in the correct order. However, during practical implementation, you will quickly realize that it is not only the right sequence which is important but also the handover from one tester to the other.

At the interface between two test cases you usually want to transfer test results which are needed for the subsequent execution. For example, in the first test case you create a new Sales Order. The ID of this order is required in the subsequent test case for further processing by a different tester.

In Test Steps Designer you can define result attributes which should be visible to all testers during the execution within a Test Sequence.



In this example, a Sales Order is created in the first step. Based on the settings of the referring result attribute, the tester has to maintain the Sales Order ID during execution of the first step. Since the attribute is also flagged as "Public in Test Sequences", the Sales Order ID will also be visible to all testers in all following test cases within the sequence.

That's one small setting for a test case designer, one giant improvement for all testers during execution.

[mbiebeler]

Hi Tobias,

I have a question regarding the test sequence functionality. When you want to create a test package out of a test sequence you can choose between 2 different options: template mode and reference mode. I fully understand the reference mode, but what is the template mode? I thought I can use it to define a sequence by the test sequence and then to create a test package out of this sequence, but without forcing the sequence to stop, when a test case failes. So I would have a test package with a number of test cases that come in a defined order but without the limitation of stopping the test process when it comes to failes.

But it doesn't work like that. The order in my test executions comes as a random order. So what is the sense of this template mode?

Thanks a lot in advance!

[TobiasMeinzer]

Hi Marga,

thank your for this interesting question. Please refer to this page on SAP Help Portal for further details. There you find the following explanation:

"A test package created from a sequence as template initially has the same selection of test cases, executables, and sequences, but it can be changed after creation and is independent of changes in the test sequence."

In other words: With the template functionality you get a predefined set of test cases for test package creation. But you will lose all other capabilities of test sequences afterwards. The derived test package is basically a simple test package without sequencing functions then.

In case you have doubts regarding this functionality I can only encourage you to create an incident (development request) on component SV-SMG-TWB-PLN.

Best regards
Tobias

[mbiebeler]

Hi Tobias,

thanks for your Feedback.

I will discuss with the customer to open an incident ticket.

Best regards

Marga

[Scherello]

Thank you for this blog.

After spending some time to understand sequences, they are a huge help.

Do you know if we can achieve one challenge: parallel test-sequences within one package?

I have MA1a->MA1b and a parallel test sequence of MA2a->MA2b. Would like to have them run in parallel - and for reporting reasons withing one package.

Putting them in 2 sequence groups creates a 1-1,1-2 and a 2-1,2-2.
But in the package they arrive as 1,1,2,2

do you know if a parallelization is foreseen?

thank you for your thoughts

[TobiasMeinzer]

Hi Marcel,

multiple (or even parallel) sequences within a single test package are not supported.

However, you could join two sequences like the following:

SEQ1: T1 –> T2 –> T3
SEQ2: T4 –> T5 –> T6

into a single new sequence: T1 –> T4 –> T2 –> T5 –> T3 –> T6

But I am not sure if this solves your problem. Maybe you can elaborate a bit further on the requirement why you want to parallelize two sequences?

Best regards
Tobias

[Scherello]

Hi,

thank you for your input.

the requirment shown with a simplified example:

• I have an employee process with 2 Test Cases


• Onboard an employee


• Plan an employee on assignment


• both need to run sequencial 1-2

Now, this process needs to be run in different flavors. For various reasons, I have different test cases (not reusable ones with parameters)

While they need to be run 1-2 per employee, several employees could be tested in parallel

I could achieve this if I create one test package per employee, but this brings other challenges with reporting. so the preferred solution would be to have sequences run in parallel. but I think this will not be possible within one package.

thank you,
Marcel

[S0022323026]

Hi Marcel

You can from SP15, create groups within a Test Sequence. So that a test cases can be executed in parallel with in one Test Sequence Group.

 

Test Case 1

Group 1 - START:

Test Case 2

Test Case 3

Test Case 4

Group 1 - Finish.

Test Case 5.

 

When Test Case 1, has been tested okay and test closed.

The Group 1 will open for testing, meaning that the Test Cases within this group can be executed in paralell.

When all test cases within the group has been tested.

Test Case 5, can now be tested.

 

I am not quite sure if this solves your problem.

Regards Jeppe

[Roszak]

@Tobias Meinzer, if the "STEP NUMBER" is missing in the result attributes, then the result is not visible in next test step? does it need to be maintained to or can the "Step Number" left empty.

 

Actually we have the problem, that our previous result is not taken from test step a to test step b

 

regards

[TobiasMeinzer]

Hello Martin,

does this issue occur within a Test Sequence? Do you mean different test cases with "test step a" and "test step b"?

The step number should not be needed for the feature "Public in Test Sequences". In case the issue still occurs please open an incident for further analysis.

Best regards
Tobias

[roberto_livieri2]

Hi Tobias,

Martin has left our company so I take over 🙂

We have 3 test steps that are within one test sequence

 

Test Sequence 1

• Test Step 1  Create Purchase Order (Mandatory Attribute "Purchase Order Number")


• Test Step 2  Post Goods Receipt with reference to PO


• Test Step 3  Post Invoice

We can't see the PO number in test steps 2 and 3.

Regards

Roberto

[TobiasMeinzer]

Hi Roberto,

please make sure that the flag "Public in Test Sequences" is active in "Test Step 1 Create Purchase Order" and that the Test Plan includes the latest version of this test case from Test Steps Designer/Solution Documentation Branch.

If the issue still persists, please open an Incident for further analysis.

Best regards

Tobias

[roberto_livieri2]

Hi Tobias,

the attribute is public, it is the only/latest version of the test step.

There is an OSS note for that issue but for an older support package. We are on SP11 so the note can't be implemented.

We will raise an incident.

 

Thank you for your feedback 🙂

Cheers

Roberto

[purple-it_rik_vriens]

Hello,

We have configured a test plan, with all kind of automated testcases.
From this we created a test package, with a certain sequence in it, different from the given sequence from SolDoc. Everything looks fine, but when trying to start the Testplan from Test Suite WC, I get an error that I should use testpackages without sequencing in them.

Is this a limitation of ordinary Test Suite, I mean without Focused Build?
Otherwise we will have to install FB to get this running the way we want it to run.

We want to have a Test Scenario, selecting testscripts from everywhere in SolDoc.
Then create test packages from this testplan, with the correct sequence, for instance First Login, action, action, action, Logout. And these Testpackages must be executed automatically.

Without Focuses Build, this is not possible. I get the error, to use packages, without sequencing.

[barbara_bendjama]

Hello,

The feature "Execute Immediately" is not supported for test package in sequence (regardless if you are using Focused Build or not).

The capability to request execution of several test cases at once, was only possible without sequence.

This has been enhanced in the context of Customer Connection Program 2019, enabling "Schedule Execution" for test package in sequence.

There was no request to extend the "Execute Immediately" in the same manner,

If you want to execute a test package in sequence without goinf gor the scheduled execution, then you will have to execute each test case one after the other fomr test cases list in the tester worklist.

The system will not allow you to execute them in a wrong order.

 

I hope this clarify the point.

Best Regards,

Barbara

 

 

[purple-it_rik_vriens]

Hi there,

So, in fact, the "one start button" for automated regression testing from Test Suite, can never be achieved, if there is any sequence in the Test Package?

So, on top of that, with Test Suite, you are not able to perform an automated regression test, that runs every, let's say thursday afternoon, and have the final result sent back to Solution Manager, if you have sequencing.

That being said, with which product can this be achieved. From Test Suite, execute a test package with sequencing, and after a while receive the total result as a traffic light. Tricentis Tosca. Qualibrate, SAP CBTA, Worksoft ??????

Best regards, Rik Vriens.

[barbara_bendjama]

Hi Rik,

Can you explain why Scheduling Test Package in sequence is not relevant for you ?

 

Best Regards,

Barbara

[purple-it_rik_vriens]

No, it is the other way around, it IS extremely important to us.

For our regression tests we have to test a certain flow of test configurations in a predefined sequence. This flow is defined in a Test Package with sequences. Let's say 1-2-3-4-5...25

We want to be able to just press Execute Automatically, and after a while have a look at the final result of the Test Execution, this is NOT possible, because Test Suite does not allow to Execute Automatically with sequencing in a test package. But they need to be executed in a certain way, otherwise at test script 4 or 5 we will have no input data.

But, sitting next to the system, executing test confg 1, waiting for result, executing test config 2, waiting for result etc, is not an option of course. We want to have an Automatic Test Execution without any interferenace from a person in between steps.

We have automated our testscripts with Qualibrate so seperately we can execute automatically, but the bigger goal is to have everything tested in one run.

 

[barbara_bendjama]

In the tester worklist, you can select the test package and choose "Automatic Test" -> "Schedule Execution".

Since Support Package 10, this is also working for a test package in sequence.

In this case, the execution will be done in the expected order and once it is finished you can see the result in the tester worklist.

 

 

 

[MartinRoszak]

Hello,

is there a way to change result attributes en masse? or download test cases and then change the attributes and upload them again?

regards

Martin

[BauerMichael]

Hello,

I am currently creating a test scenario with public Result Attributes used in a Test Sequence.

Unfortunately, during execution I am not able to see the Result Attribute values from my previously executed test cases. (The sequence contains 10 cases, some of them have at least one Result Attribute which is mandatory, mapped to a test step and "Public in Test Sequence") 

I already checked with a colleague about all relevant settings but it seems that there is no error from my side with the configuration from Result Attributes, Test Sequence, Test Case as well as the Test Plan. I already searched for known bugs but can't find anything about this...
Anyone else who has the same issue or a solution?

Thank you and regards,

Michael