Journey to setup best possible Continuous Integrat...

former_member205561 · ‎2015 Nov 27

Background

After working for several years in the nowadays so called "DevOps" area, developing test automation tools, simulation frameworks, automated performance & stability tests and rampup and maintaining quite complex build, test and delivery pipelines (CI's/CD's) for both Java and .NET stack projects i joined this year SAP. Into an purely ABAP based project.

Idea

Setup CI. Simple. Or ?

Beside normal CI requirements we have additional test activities to do before release

Compatibility to all versions between 7.02 and 7.5x, S4H*
Performance KPI's, especially from customers with huge (x hundred's Mio contracts like Vodafone)
Test with specific customizations (using one config as blueprint for whole industry sector)

so, first sketch (translated Dev/Integration/Release branch model to ABAP development landscape)

ATC

So easy to start, but not all developers consistently run check and remove - at least Prio1&2 - before triggering transport. For *some* machines (which are connected ) it is possible to use the ATC gadget on the Bridge Dashboard.

Follow this guide to setup HTTP IF for ATC Checks.

AUnit

Same. Failing Unit tests do not block any transport. TDD? Yes, most heard about it, but really doing in daily life ? Good practice is to plan as task immediately for each Item from backlog and add to teams DoD a minimum of code coverage for newly created code.

We found it very effective to setup at least nightly runs of our unit tests, each team got an own variant, an own Jenkins Job for the trend charts and results(errors) discussed in each teams daily standup.

Step-By-Step Guide Setup Jenkins Instance wit AUnit Test Jobs

Transport Events

Yeah! After I first heard about it I immediately thought that is the solution to slackers which ignore both ATC and AUnit. But setup is a bit more complicated as thought. First of all - check version of SUT. Write DLM ticket to update to latest version (as of today - 11/2015 - this is 2.12). Use Transport Requests with latest version of SUT Core as reference within the DLM ticket.

Note: It will not work for 7.02.

Now you can configure your systems to run both ATC & AUnit triggered by Import/Export Events (which sends EMail to the author of the change).

ITF - Instant Test Framework

During research I found some references to the ITF, which was developed within ByD landscape and never ported to NW.

The idea is close to CI - first run a "trial" transport into a shadow system, and only if all tests passed there activate the transport for next stage.

See some more ITF details.

eCATT with START

In our project we have SAPUI & WebDynPro, which means for E2E tests we have to use eCATTs (or manual testing). eCATTs are quite expensive (around 2 days per TC) and generate a heavy maintenance load (test scripts need adaptation, test data might need to be changed, errors itself are hard to find cause for some types of crashes there is no way to get a detailed trace). All these reasons have led to running them as less as possible. And of course no test trend chart available too.

Best possible solution: Trigger eCATT test plans (which so far must be configured in ECA system cause START cannot run test plans from GTP) via START command line from Jenkins. Meanwhile START team extended functionality so that results can be retrieved in JUnit format and displayed quite nicely (but without possibility to directly jump to detailed traces).

Code Coverage Trends

Within one of the latest versions of SUT a new feature was introduced to generate Code Coverage Trend Charts.

And again, not availble for NW 7.02.

ABAP code duplication check (Simian, conQAT, …)

http://scn.sap.com/community/abap/blog/2015/02/17/static-abap-code-analysis-using-conqat-and-of-cour...

I tried to configure conQAT in eclipse but gave up after one day without the expected results.

ABAP in Eclipse

Shall be better as ABAP workbench, but not available for NW 7.02.

Open Points

ATC Checks in Jenkins
Code Coverage Trends In Jenkins
START to run tests from GTP, providing more params like "Start profile"

Further Links

js2 · ‎2015 Nov 30

Interesting blog, but all the links seem to be internal to SAP only?!?

ceedee666 · ‎2015 Nov 30

Hi Mathias,

I second Jasons commet. The topic is really interesting and relevant for lots of developer and customers. Is it possible to share some of the SAP internal information here on SCN as well?

Christian

former_member205561 · ‎2015 Nov 30

@Jason & Christian:

As there are a lot of sub-links, which topic would be most interesting for you to add here in SCN?

js2 · ‎2015 Dec 01

These ones would be great if you can make them public:

Step-By-Step Guide Setup Jenkins Instance wit AUnit Test Jobs

These ones too:

I've seen a few references to START now - I'm wondering if this is purely for internal purposes or something that SAP may make available for users of their products..?

Thanks.

former_member184711 · ‎2015 Dec 01

I join the other commenters.

I am very interested in how to setup jenkins with ABAP development

former_member205561 · ‎2015 Dec 01

I need to ask about START.

In short :

It provides a command line to trigger a test plan configured in the ECA system and returns results in a JUNIT format file. As I understood the connection is rather hard coded within the tool.

Update:

Answer from Product Owner of START:

"Currently START is not available for release to customers. START is used only internally."