Headless OPA5 testing with Karma and PhantomJS

[former_member234069]

Introduction

During development it can be sufficient to manually start and check OPA5 tests in the browser. If started this way the tests are normally deployed on a web server, services like the HCP HTML5 app, or bundled with a Java application.

Doing continuos integration the build duration might become a challenge. To match this, OPA5 tests can be run with Karma and PhantomJS. While Karma will serve the files to the browser without the overhead of an actual web server, PhantomJS will not need the overhead created by a web browser like Chrome which needs to be run with a window system.

The Karma run can produce a junit XML file, which can be published at a continuos integration like Jenkins CI. Other formats are supported as well once the karma reporter is installed.

Prerequisites

Karma is based on NodeJS. So a prerequisite is a LTS or an up to date NodeJS installation with its package manager npm. In addition a UI5 project with the recommended folder structure. It needs to include at least one OPA5 test with a mocked back end. In its root folder a npm project needs to be initialized:

- test

  - ...

- webapp

  - controller

  - i18n

  - model

  - view

  - ...

  - Component.js

  - index.html

- package.json

- gruntfile.js

- karma.conf.js

Links

• PhantomJS


• Karma


• Karma OpenUI5


• Karma QUnit


• Karma Junit Reporter


• Karma Coverage

Karma and Plugins

At this point a npm project should be initialized into which we will install additional NodeJS modules.

Karma and the PhantomJS launcher:

npm install --save-dev karma karma-phantomjs-launcher

Plugin needed for test execution:

npm install --save-dev karma-openui5 karma-qunit qunitjs

Reporter plugin which will store the result in JUnit format:

npm install --save-dev karma-junit-reporter

Calculate the test coverage with Istanbul:

npm install --save-dev karma-coverage

karma.conf.js

How to configure the karma.conf.js in general is documented at karma.io. To initialize and empty one run the command:

karma init karma.conf.js

Files

By running karma, a context.html will be generated and the required files will be added into it. The files array specifies which file to load. The karma-openui5 plugin will add the required UI5 files into the files array automatically and in the right order. The other files need to be specified. The JavaScript file which is loaded by the index.html and requires/starts the rest is specified last. It is the only file with the property included set to true.
In case the code coverage should be recorded, replace dist with webapp.

files: [

    {pattern: 'dist/**', included: false, served: true, watched: false, nocache: false},

    {pattern: 'test/**/*.js', included: false, served: true, watched: false, nocache: false},

    {pattern: 'test/AllOPATests.js', included: true, served: true, watched: false, nocache: false}

],

Proxies

By default Karma will serve the files at /base/(...). In case absolute paths are used in the tested application, errors will happen. To overcome Karma´s proxy function can be used.

proxies: {

    '/': '/base/dist/',

    '/test/': '/base/test/'

},

Or in case of testing the uncompressed sources with a mapping to the webapp folder.

proxies: {

    '/': '/base/webapp/',

    '/test/': '/base/test/'

},

UI5 Config

After installing the plugin karma-openui5, it is possible to configure a CDN from which UI5 is loaded and also UI5 itself. The basic setup is documented here. For a fast execution make sure to disable animations and activate async preloading

preload: 'async',

animation: 'false',

resourceroots: {

    'test': '/test/',

    'your.app': '/'

},

'xx-debugModuleLoading': 'true',

'xx-showLoadErrors': 'true',

Recording Test Coverage

In case the karma-coverage plugin is installed and the test is executed against the source files, the coverage of the OPA5 tests can be recorded. Add the plugin to the reporters. Also configure the preprocessors, so that the plugin knows which files should be recorded.

reporters: ['progress', 'junit', 'coverage'],

preprocessors: {

    'webapp/**/*.js': ['coverage']

},

Depending on the CI system the code coverage report needs a different format. Jenkins CI for example can make use of the Cobertura format, while SonarQube prefers lcov. To create the report reports/coverage/cobertura/coverage.xml configure like the following. Add more reporters if needed.

coverageReporter: {

    dir: 'reports/coverage',

    // If only PhantomJS is used, remove the folder with the browser name

    subdir: function() {

        return '';

    },

    reporters: [{

        type: 'cobertura',

        subdir: 'cobertura',

        file: 'coverage.xml'

    }, {

        type: 'lcovonly',

        subdir: 'lcov'

    }]

}

Extending karma.conf.js

There might come a point where the execution of the tests will take a significant amount of time. Another use case is recording the test coverage. Here the raw sources should be tested instead instead of the compressed files in the dist folder. On top the same tests should be executed against the dist folder to ensure the compression did not brake the tests.
For such use cases a base configuration can be used, which is extended by various others. The extending config will be specified by the task runner and load the base config.

Starting karma takes a couple of seconds. If not configured otherwise, the files are cached into ram for speed. This process has to be repeated for every task that is started. Make sure that there is an actual gain in execution time and or process wise, before executing N karma tasks in parallel.

Starting OPA5

OPA5 testing is started with the QUnit function QUnit.start(). If run as a manual test in a browser, it is sufficient to call QUnit.start() at the file `AllOPATests.js` once it has loaded all test files.

However if run with Karma, an issue with the plugin karma-qunit will start the tests even if the config QUnit.config.autostart is set to false. This will lead to a failure of all tests, as the second start command will break the tests. The ticket to this known issue can be found here.
To solve it comment the line with the function call runner.start(); at the file node_modules/karma-qunit/lib/adapter.js or add a check for the autostart property

if (!window.QUnit || !window.QUnit.config || window.QUnit.config.autostart !== false) {

  runner.start()

}

The second challenge will be latency. It might happen that UI5 is not fully loaded yet, but all the test files are loaded. If the QUnit command is executed at this point, the tests will fail because sap.m or another module is still in the process of loading. This can be overcome by a function that checks if the UI5 modules are loaded and if not restarts itself. Use it at the place, where you normally execute QUnit.start(). It could look like:

/**

 * Delayed start of QUnit or KarmaJS might break.

 * @returns {void}

 */

function checkStart() {



    'use strict';



    var aModules, i, iLength;



    if (!window['sap-ui-config'] || !window['sap-ui-config'].libs || !sap) {



        setTimeout(checkStart, 500);



        return;

    }



    aModules = window['sap-ui-config'].libs.replace(/sap./g, '').replace(/\s/g, '').split(',');



    for (i = 0, iLength = aModules.length; i < iLength; i++) {



        if ((aModules[i].indexOf('.') !== -1 && !sap[aModules[i].split('.')[0]]) || (aModules[i].indexOf('.') === -1 && !sap[aModules[i]])) {



            setTimeout(checkStart, 500);



            return;

        }

    }



    QUnit.load();

    QUnit.start();

}

Run Karma as Gulp Task

In order to run Karma as a Gulp task, no plugins are required. N tasks can be defined and grouped by a general task:

const Server = require('karma').Server;



gulp.task('opa5-dist', (done) => {

    new Server({

        configFile: __dirname + '/karma_dist_all.conf.js'

    }, done).start();

});



gulp.task('opa5-webapp', (done) => {

    new Server({

        configFile: __dirname + '/karma_webapp_all.conf.js'

    }, done).start();

});



gulp.task('opa5', ['opa5-dist', 'opa5-webapp']);

Example Files

karma.conf.js

module.exports = function(config) {



    config.set({



        // base path that will be used to resolve all patterns (eg. files, exclude)

        basePath : '',



        // How long will Karma wait for a message from a browser before disconnecting from it (in ms).

        browserNoActivityTimeout: 2000000,



        // frameworks to use

        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter

        frameworks: ['openui5', 'qunit'],



        // list of files / patterns to load in the browser

        files: [],



        // list of files to exclude

        exclude: [],



        // preprocess matching files before serving them to the browser

        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor

        preprocessors: {},



        // In case an absolute URL is used at some point of the code, a proxy configuration is required.

        proxies: {},



        // test results reporter to use

        // possible values: 'dots', 'progress'

        // available reporters: https://npmjs.org/browse/keyword/karma-reporter

        reporters: ['progress','junit'],



        // the default configuration

        junitReporter: {

            outputDir: 'reports', // results will be saved as $outputDir/$browserName.xml

            outputFile: 'junit-opa5.xml', // if included, results will be saved as $outputDir/$browserName/$outputFile

            suite: '', // suite will become the package name attribute in xml testsuite element

            useBrowserName: false, // add browser name to report and classes names

            nameFormatter: undefined, // function (browser, result) to customize the name attribute in xml testcase element

            classNameFormatter: undefined, // function (browser, result) to customize the classname attribute in xml testcase element

            properties: {} // key value pair of properties to add to the <properties> section of the report

        },



        // start these browsers

        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher

        browsers: ['PhantomJS_custom'],



        // Have phantomjs exit if a ResourceError is encountered (useful if karma exits without killing phantom)

        phantomjsLauncher: {

          exitOnResourceError: false

        },



        // you can define custom flags

        customLaunchers: {

            PhantomJS_custom: {

                base: 'PhantomJS',

                options: {

                    viewportSize: {

                        width: 1440,

                        height: 900

                    },

                    customHeaders: {

                        DNT: "1"

                    },

                    windowName: 'my-window',

                    settings: {

                        webSecurityEnabled: false,

                        userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36"

                    },

                },

                flags: ['--load-images=true', '--debug=false', '--disk-cache=false'],

                debug: false

            }

        },



        openui5: {

            path: 'https://sapui5.hana.ondemand.com/1.40.12/resources/sap-ui-core.js',

            useMockServer: false

        },



        // How the client should execute the test, like run in an iframe, capture the console and so on.

        client: {

            captureConsole: true,

            clearContext: true,

            useIframe: false,

            qunit: {

                showUI: false,

                testTimeout: 15000,

                autostart: false,

                autoload: false

            },

            openui5: {

                config: {

                    theme: 'sap_belize',

                    libs: 'sap.m',

                    compatVersion: 'edge',

                    frameOptions: 'deny',

                    preload: 'async',

                    animation: 'false',

                    debug: 'false',

                    resourceroots: {

                        "test": "/test/",

                        "your.app": "/"

                    },

                    'xx-debugModuleLoading': 'true',

                    'xx-showLoadErrors': 'true',

                    'xx-supportedLanguages': ['en']

                }

            }

        },



        // enable / disable colors in the output (reporters and logs)

        colors: true,



        // level of logging

        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG

        logLevel: config.LOG_ERROR,



        // enable / disable watching file and executing tests whenever any file changes

        autoWatch: false,



        // Continuous Integration mode

        // if true, Karma captures browsers, runs the tests and exits

        singleRun: true,



        // Concurrency level

        // how many browser should be started simultaneous

        concurrency: Infinity

    });

};

karma_dist_all.conf.js

//Load the base configuration

var baseConfig = require('./karma.conf.js');

 

module.exports = function(config) {



    // Load base config

    baseConfig(config);

 

    // Override base config

    config.set({



        // list of files / patterns to load in the browser

        files: [

            {pattern: 'dist/**', included: false, served: true, watched: false, nocache: false},

            {pattern: 'test/**/*.js', included: false, served: true, watched: false, nocache: false},

            {pattern: 'test/AllOPATests.js', included: true, served: true, watched: false, nocache: false}

        ],



        // In case an absolute URL is used at some point of the code, a proxy configuration is required.

        proxies: {

            '/': '/base/dist/',

            '/test/': '/base/test/'

        },



        junitReporter: {

            outputFile: 'junit/dist_all.xml',

        },



        port: 9876

    });

};

karma_webapp_all.conf.js

//Load the base configuration

var baseConfig = require('./karma.conf.js');

 

module.exports = function(config) {



    // Load base config

    baseConfig(config);

 

    // Override base config

    config.set({



        // list of files / patterns to load in the browser

        files: [

            {pattern: 'webapp/**', included: false, served: true, watched: false, nocache: false},

            {pattern: 'test/**/*.js', included: false, served: true, watched: false, nocache: false},

            {pattern: 'test/AllOPATests.js', included: true, served: true, watched: false, nocache: false}

        ],



        // In case an absolute URL is used at some point of the code, a proxy configuration is required.

        proxies: {

            '/': '/base/webapp/',

            '/test/': '/base/test/'

        },



        // test results reporter to use

        // available reporters: https://npmjs.org/browse/keyword/karma-reporter

        reporters: ['progress', 'junit', 'sonarqubeUnit', 'coverage'],



        // source files, that you wanna generate coverage for

        // do not include tests or libraries

        // (these files will be instrumented by Istanbul)

        preprocessors: {

            'webapp/**/*.js': ['coverage']

        },



        // Configuration of coverage reporter

        coverageReporter: {

            dir: 'reports/coverage',

            subdir: function() {

                return '';

            },

            reporters: [{

                type: 'cobertura',

                subdir: 'cobertura',

                file: 'coverage.xml'

            }, {

                type: 'html',

                subdir: 'html'

            }, {

                type: 'lcovonly',

                subdir: 'lcov'

            }]

        },



        junitReporter: {

            outputFile: 'junit/webapp_all.xml',

        },



        port: 9877

    });

};