Dynamic image manipulation with UI5 and Promises

vobu · ‎2016 Jan 06

Intro

You know that time when manipulating height and width of an image just doesn't get the job done? This is when a lot of additional players enter the game quickly - HTML5's canvas, Promises and runtime Image objects.

We're talking asynchronous content handling here by retaining control over runtime execution along with operating a drawing canvas - all of that in UI5, via JavaScript.

Good thing that we're going to cover all that ground here :smile:

Although every aspect of client-side image manipulation at runtime would deserve its own blog post (even more so when looked at from a UI5-perspective), let's go for the sweep through.

Why asynchronous?

Sure thing: working over an image with JavaScript could be done synchronous à la (attention, pseudo-code)



var oImage = Page.getImage();

oImage.resizeTo(1500,1000);

Page.put(oImage);

This would result in the image getting resized - with the user potentially closing the browser window in the process.

Why? Because synchronous JS-operations lock the UI.

If they run for a noticeable amount of time (we're talking single-digit secons here), it appears to the user that the browser has frozen. This causes frustration, the user might cancel the operation by killing the browser - bad UX.

So the rule of thumb is: when doing extensive JS operations client-side, do them asynchronous in order not to lock up the UI.

Why Promises?

Sure thing: you could use callback functions to go asychronous with your code (attention, pseduo-code again)



var oImage = new Image();

var oImageNew = new Image();

oImage.src = Page.getImage();

oImage.onload = function() {

  oImageNew = oImage.resizeTo(1500,1000);

  Page.put(oImageNew);

}

Here, the onload event of an image is utilized by an anonymous callback function to resize the image asynchronously (after the original image is ready for processing).

Fine, right? Well, almost.

Because now you certainly want to do other stuff after the resizing has finished.

Which means, you'd have to stuff everything in the anonymous callback function (attention, pseudo-code, you know)



oImage.onload = function() {

  oImageNew = oImage.resizeTo(1500,1000);

  Page.put(oImageNew);

  Page.showPopup("Resizing finished!");

  oDownloadButton.show();

  // anything else

}

Imagine a couple of asynchronous callback-based operations in a process.

Sooner or later, you end up in callback hell (by now, you should know, pseudo-code and such)



getImage(function(a){

    getAnotherImage(a, function(b){

        createDownloadLink(b, function(c){

            showDownloadButton(c, function(d){

                ...

            });

        });

    });

});

Wouldn't it be nice to be able to:



getImage()

  .then( resizeIt )

  .then( createDownloadLink )

  .then( showDownloadButton )

?

Answer: yes, it would be. Awesome even!

And that's what Promises in JS are for: they let you retain control over sequence in asynchronous operations.

(Technical excursion: compared to callbacks, they give you return and throw back, along with a meaningful runtime stack for debugging. Elaborate on that? Not here. See beginning of blog post, "every aspect ... would deserve its own blog post" bla bla).

For the record: Why canvas?

Because literally there is no other way to dynamically manipulate images client-side at runtime.

So, after this lengthy intro:

let's get started

(Upfront: go grab or look at the source over at GitHub. Way easier to have that open simultaneously in order to get the bigger picture.)

Here's an image in UI5:



<Image

                    id="moray_eel"

                    src="moray_eel.jpg"

                    press="transformPic" />

We want to do some show-casing manipulation on it (transformPic) after it is clicked.

First step is to get the base64-string representing the image.

That's where we already dive into the Promise-world (no more pseudo-code from here on, promised (ha ha)):

The Promise



sap.ui.define([

                "sap/ui/core/mvc/Controller"

            ], function (Controller) {

                return Controller.extend("vb.controller.main", {

                    sBase64: '',

                  transformPic: function (oEvent) {

                     

                        this.toBase64(oEvent.getSource().getProperty("src"))

                                // store resulting base64 as property

                                .then(function (sBase64) {

                                    this.sBase64 = sBase64;

                                }.bind(this)) // handle context!

                        // ...

                  }

          })

});

Quick excursion: notice the bind(this) above: in the async world, one key to success is to always watch your context. Meaning, to what scope the this keyword applies to when the code is executed. One way of doing this is to bind the current scope via (duh) bind(this) to a function that is executed at a later point in time, in a different scope.

Back to toBase64(): it is defined as a Promise. Stripped to bare minimum, it looks like:



toBase64: function (sBinaryImgSrc) {

                        return new Promise(

                                    // do some stuff

                                    // ...

                                    if (everythingOk === true) {

                                    // resolve a.k.a "return" the Promise

                                        resolve(someValue);

                                    } else {

                                   // sth went wrong

                                    // -> reject the Promise a.k.a "report" the error

                                        reject(someError);

                                    }

                                }

                        );

                    }

Promises give you exactly 2 options: resolve or reject them.

When resolving, return a value. Or just return. Important thing is, that you resolve() at all.

When rejecting, think of it as of throw-ing an error.

This is where you regain control: the Promise "spawns" its own thread, executes the stuff you want it to, and then tells you afterwards whether it was successful (resolve) or not (reject). So you get to know, when your async operation has finished - now you can continue doing other stuff (asynchronously) or handle the error:



this.toBase64(oImgSrc)

    .then( function(resultOfToBase64) {

        // do sth with resultOfToBase64

        return doWhatEver(resultOfToBase64);

    })

    .then( function(resultofWhatEver) {

        return doMore(resultofWhatEver); // doesn't have to be a Promise, can be any function returning stuff

    })

    .then( doEvenMore ) // shorthand notation of calling the Promise doEvenMore with the result of doMore

    .catch(function (oError) {

        console.log(oError);

    });

Following this principle, you can either continue then-ing by

- calling other Promise functions or

- just returning values/functions.

The entire promisey way of then-ing depends on returning values/functions or calling other Promises (which themselves just return stuff).

Read the above again. "The entire...". And again. "The ...".

Because this is where most errors happen: if your functions/events/Promises are not executed in order, but somehow different, it is most likely you forgot to return/resolve at one point.

(Technical excursion: there's a whole lot of variations on how to call functions, use more than one parameter for the subsequent then-d call (named functions!) etc pp. As mentioned at the beginning of the post: "every aspect ... would deserve its own blog post" bla bla).

Now that we know how to control the sequence of the asynchronous operations, here's how to manipulate images in UI5 at runtime.

The image manipulation

As mentioned above, using HTML5's canvas for the purpose is a given. Only so can an image be repainted on a canvas (ha ha).

Here's where we use the Promises: we load a base64-string onto a canvas, make modifications and then return the modified image, again as base64-string.

Let the source code and the comments speak for themselves:



return new Promise(

                                function resolver(resolve, reject) {

                                    // construct an "anonymous" image at run-time

                                    var oImage = new Image();

                                    // trigger onload

                                    oImage.src = sBinaryImgSrc;

                                    oImage.onload = function () {

                                        // construct canvas

                                        var oCanvas = document.createElement("canvas");

                                        // make canvas fit the image

                                        oCanvas.width = this.width;

                                        oCanvas.height = this.height;

                                        // reduce opacity of context, then applied to the image

                                        var oContext = oCanvas.getContext("2d");

                                        oContext.globalAlpha = 0.2;

                                        oContext.drawImage(this, 0, 0); // paint the image onto the canvas

                                        // retrieve the manipulated base64-represenation of the image from the canvas

                                        var sBase64 = oCanvas.toDataURL("image/jpeg", 1.0); // retrieve as JPG in 100% quality

                                        // "return" it

                                        resolve(sBase64);

                                    };

                                    // sth went wrong

                                    // -> reject the Promise a.k.a "report" the error

                                    oImage.onerror = function (oError) {

                                        reject(oError);

                                    };

                                }

                        )

Following this approach, you can basically manipulate the image any way HTML5's canvas allows for.

See the example for reducing opacity, inverting color scheme and resizing images (resulting in new images).

Speaking of resizing:

1. the use case for this is simple - say, you allow for picture uploads by your users. For performance reasons, you don't want to use the original uploaded image everywhere in your UI5 application. Instead, you want a lower-quality version, e.g. for previews. That's when you need store additional versions of the uploaded image for later use. You can generate as many variations of the uploaded picture at runtime with UI5 and ...

2. ...the help of using Promises in batch-mode.

What?

Yes, batch-mode. Meaning: trigger multiple asynchronous operations simultaneously and get notified when the last one(!) has finished.

The asynchronous equivalent of the synchronous forEach-loop so to speak.

This is what Promise.all(aPromises) is for. It takes an array of Promises as argument, executes each one and will only resolve() after the last one has finished - returning you the result of every Promise of the argument-array in return. Isn't that awesome?!?

Look at line 124ff. of the example file for an (duh) example of using Promise.all() - for batch-resizing an image.

Conclusion

So here we are: having used UI5, canvas and Promises in order to manipulate images at runtime.

All of this in an asynchronous way, keeping the UI unlocked, allowing for simultaneous application behavior.

Good stuff!

But especially Promises in JS are hard to understand just by looking at them once. At least for me. It took me some time to get my head around the concept and the coding embedded in UI5. So I encourage you to get your hands dirty as well, fiddle with and/or look at the code and/or look at the screenshots attached.

Happy coding!

tl;dr

use JS Promises in UI5 to load base64-strings on canvas to generate new images.

Example source