Issue on webtests with SAP Commerce 2205P1

[nihil]

We have an issue on our web tests with the newest 2205P1 Upgrade of SAP Commerce when running

ant allwebtests -Dtestclasses.suppress.junit.tenant=true

The tests are randomily failing due to an Mockito mocking error exceptions(see below). Running the same test again via the same ant command is then succeeding without any code change.

So the result of the tests is just random and not deterministic (sometimes they fail, sometimes not)

It might be due to some classpath issue? Was anything changed in the 2205P1 regarding that and tests? Running the same tests with no code change on 2205P0, all tests are always successful.

<code>java.lang.NoClassDefFoundError: Could not initialize class org.mockito.codegen.ReturnService$MockitoMock$1335712202
	at jdk.internal.reflect.GeneratedSerializationConstructorAccessor2.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:48)
	at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:73)
	at org.mockito.internal.creation.instance.ObjenesisInstantiator.newInstance(ObjenesisInstantiator.java:22)
	at org.mockito.internal.creation.bytebuddy.InlineDelegateByteBuddyMockMaker.doCreateMock(InlineDelegateByteBuddyMockMaker.java:372)
	at org.mockito.internal.creation.bytebuddy.InlineDelegateByteBuddyMockMaker.createMock(InlineDelegateByteBuddyMockMaker.java:330)
	at org.mockito.internal.creation.bytebuddy.InlineByteBuddyMockMaker.createMock(InlineByteBuddyMockMaker.java:58)
	at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:53)
	at org.mockito.internal.MockitoCore.mock(MockitoCore.java:84)
	at org.mockito.Mockito.mock(Mockito.java:1954)

[nihil]

Hi Lennart, no solution for now, we worked around it by replaying the webtests in our CI until they succeed or a replay limit is reached

[lennart_juette]

Thanks Konrad!

I opened a ticket for SAP. Let's see, if they find something.

[lennart_juette]

Do you happen to use the jacoco agent when running the tests?

[nihil]

Yes, we run jacoco for coverage reports as agent

[lennart_juette]

Sorry for causing confusion. I though that removing the jacoco-agent from the JVM args would help, but it was just a fluke. I had a single, successful test execution. After that the tests started to fail again.

[paraspatel13]

Hi,
We are on 2205 Patch 5 and experiencing the exact same issue, were you able to get this working or its still the same for you as well.
Lennart, could you please share if SAP were able to provide any solution on the ticket that you already created,

Thank you.
Regards,
Paras.

[lennart_juette]

Hi there,

SAP pointed me to a few wrong directions. In the end the case was closed, because there apparently was no one else who reported the issue in their issue tracker. At least SAP couldn't identify similar issues. So if none of you reports this issue in launchpad or me.sap.com, they won't acknowledge the issue. At least that's how I understood their response.

My solution: Run tests and disable extensons one-by-one, to find out which combinations of extensions caused the issue. Long story short: I just disabled a certain extension, which didn't have tests anyways. Works for me, YMMV.

Cheers,
Lennart

[pj_delaere]

Hi,

I created a ticket at SAP. Hopefully they will look into it soon

regards