topic Re: Order in which test methods are executed in Application Development and Automation Discussions

Order in which test methods are executed

SuhaSaha — Wed, 15 May 2013 10:41:06 GMT

Hello Team,

This is my first hands on experience with ABAP Unit, so please excuse me if the question is a basic one.

I have a class with 1 public instance method (PUB1) which in turn makes call to 3 pvt. instance methods - PVT1, PVT2, PVT3 - in this sequence. But when i execute my local test class (LCT_UNIT_TEST), the sequence PVT1 -> PVT2 -> PVT3 is not maintained

SAP states & i quote -

The order in which the tests are executed is not specified deterministically.

ref.: Executing ABAP Unit Tests - Test and Analysis Tools in ABAP - SAP Library

If i declare the CUT(Class/Component Under Test) instantiation in the SETUP it works okay, but initially i had declared that in CLASS_SETUP and i was getting test failures!

BR,

Suhas

Re: Order in which test methods are executed

adam_krawczyk1 — Wed, 15 May 2013 12:05:10 GMT

Hi Suhas,

CLASS_SETUP is run only once, before all tests are run (once per class).

SETUP is run before every test method, so if you have 3 test cases (3 tests methods marked with FOR TESTING addition) it will be run 3 times, before each. That is why you can see difference between putting CREATE OBJECT CUT into CLASS_SETUP or SETUP. It is recommended to have it in SETUP, as each single test method should have clear and fresh instance to work on (not keeping state from previous test run).

We should not rely on order of test methods (methods with addition FOR TESTING). I observe that order of running is close to alphabetical, but I never rely on the order - each single method test case must be individual, independent of other methods. So it is not recommended to rely on tested object state from one method to another - always work on new clear instance. However "randomized" sequence is assumed only for test methods (with FOR TESTING addition) which are run by Unit Test ABAP framework (CTRL + SHIFT + F10). If you have some method, that inside is running method M1, M2, M3 - this order will be kept, as this is your code implementation that cannot randomly change.

On the other hand, you may be sure that order of test methods and fixtures will be constant as:

CLASS_SETUP

SETUP

TEST_METHOD_1

TEARDOWN

SETUP

TEST_METHOD_2

TEARDOWN

...

CLASS_TEARDOWN

Regards

Adam

Re: Order in which test methods are executed

SuhaSaha — Wed, 15 May 2013 14:32:48 GMT

Hello Adam,

I understand the difference between CLASS_SETUP & SETUP , they are similar to static (CLASS_CONSTRUCTOR) & instance CONSTRUCTOR!

 I observe that order of running is close to alphabetical, but I never rely on the order

Me too! But i was hoping they would be called called in some defined sequence

each single method test case must be individual, independent of other methods

Learning#1 - that each test method is independent and that i where your DAO injection technique is going to be of some help!

It is recommended to have it in SETUP, as each single test method should have clear and fresh instance to work on (not keeping state from previous test run).

Learning#2 - Test methods should be de-coupled & should not be dependent on the previous runs!

I generally develop "singleton"applications (using the CLASS_CONSTRUCTOR), I think you can relate my affinity towards using CLASS_SETUP

BR,

Suhas

Re: Order in which test methods are executed

kakshat — Wed, 15 May 2013 15:03:39 GMT

Hey Suhas,

I am not writing to provide you an answer to your question but to appreciate your posting a question inspite of being a moderator! I know there's no bar on moderators from asking questions but I don't think I have ever seen one posting a question on SCN.

Regards,

Akshat

Re: Order in which test methods are executed

adam_krawczyk1 — Thu, 16 May 2013 06:38:41 GMT

Good points Suhas.

If we only know the framework rules, we can set up tests as we want, depending on case.

Regards

Adam

Re: Order in which test methods are executed

SuhaSaha — Thu, 16 May 2013 07:01:16 GMT

Hello Akshat,

Actually I am humbled by your post There's no age to learning, isn't it?

I don't think I have ever seen one posting a question on SCN

Spend a few more days on SCN & you'll see that us Mods (atleast me) aren't Mr. Know-it-all

BR,

Suhas