Headless Cordova Android builds on Jenkins

This is a fairly old story, nevertheless I still think it’s worthwhile telling, just in case someone else wants to build Android APKs from Cordova within their headless CI/CD Jenkins environment.

The story goes back to the end of September 2019 where at my work (Drift+Noise) we were just starting to build an Android app to provide satellite-based sea-ice information to people navigating in and around Svalbard. We had decided to use Cordova to build the app because it uses technologies that we as a team were already familiar with: HTML, CSS and JavaScript. It also gave us the possibility to target multiple platforms from a single codebase (which would allow us to reach more users), so it looked like the tool to use. In our particular case, it turned out that Progressive Web App technologies were a much better fit for what we were trying to do and the app has evolved significantly since then, turning into what is now called IcySea, covering now the entire Arctic and Antarctic with sea-ice related satellite information.

But I’m getting ahead of myself: let’s go back in time to the last quarter of 2019.

Now, I’m a big fan of CI/CD systems: being able to build, test and package software in a reproducible way on system other than my development box is a great way to avoid “it-works-on-my-box-itis”. Also, having another system check that the application dependencies are configured correctly, that the operating system has the required packages installed, that the tests don’t fail, that the software can be built and packaged at all, is a huge help and has saved my backside on several occasions. Therefore, it made sense to build, test and bundle our new app on our existing Jenkins infrastructure. Unfortunately, at the time, it wasn’t obvious how one should do this from the command line, potentially repeatably, and in a headless environment (i.e. without some kind of GUI to interact with). At the time, Google obviously preferred that users used Android Studio (a GUI application) to build Android applications (and I think they still do) and had not published the location of their command line tools very prominently. The situation has changed somewhat now, as the command line tools are now listed on the Android Studio downloads page. At the time I was wanting to use just the SDK, there was a lot of googling involved (oh, the irony!) to try and work out how to set up an Android environment that Cordova could then use without needing to interact with a large GUI application over the network on our cloud-based infrastructure in order to do it. What follows is the procedure I came up with to get this to work.

Let’s split up the procedure into separate chunks: setting up the Android environment, setting up the Cordova environment, running the test suite, and building the Android APK. Not only does this make each chunk easier to digest, but it fits nicely with the stages we created in the Jenkinsfile to drive the build and test process.

Setting up the Android environment

First things first: let’s install the Android SDK command line tools. Why, you ask? How do we know to start here? Let’s go down that rabbit hole first.

Our goal is to be able to build an APK (Android package file) of the Cordova app we’re developing. To do that, we need to install and set up Cordova so that it can build Android packages. To do that, we need to create an Android virtual device so that we can emulate an Android device on our development system. In order to do that we need to set up an Android system image to use within the Android virtual device. And in order to do that we need to install the Android SDK command line tools. Phew!

So, effectively popping the why stack from our goal to where we are now tells us what to do first. Let’s do it!

Installing the Android SDK command line tools

This package used to be called sdk-tools but has more recently been renamed to commandlinetools which is probably a more descriptive name. Either way, it’s a zip archive containing the software development kit (SDK) command line tools required to build Android applications on your platform. I’m going to focus on a Linux-based environment here because it’s the one I know best (and what’s deployed on our CI/CD infrastructure).

Let’s install the Android SDK command line tools within the Jenkins project’s workspace, e.g. in a directory such as: ${env.WORKSPACE}/android_sdk. This is a non-standard location, which will have repercussions for how we call sdkmanager later, however this allows us to fairly easily start from a completely clean slate if we want to, by simply removing the workspace directory for this project. If we install these tools in a central location then we can’t manage multiple installations with (say) different versions and setups. The encapsulation we have here gives more flexibility but at the price of a bit more complexity in the project setup. Although Jenkins will clean up project workspace directories if they haven’t been touched for a while, for reasonably fresh projects it will (usually) leave any previous checkouts and installed files in place. Therefore, we only need to install the Android SDK tools if we haven’t already done so; this we test by checking if the installation directory exists or not.

To install the command line tools, it’s as simple as creating the android_sdk directory, changing into it, downloading the command line tools zip archive and extracting it in place. Pulling all of these threads together, we get the following shell code:

# install Android SDK tools if necessary
export ANDROID_HOME="${env.WORKSPACE}/android_sdk"
export PATH="${env.PATH}:${env.ANDROID_TOOLS}"
if [ ! -d "${env.ANDROID_HOME}" ]
then
    mkdir "${env.ANDROID_HOME}"
    cd "${env.ANDROID_HOME}"
    wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
    unzip commandlinetools-linux-7583922_latest.zip
    cd "${env.WORKSPACE}"
fi

where we also ensure that we return to the workspace directory after having installed the files. Note that ${env.ANDROID_HOME} and ${env.PATH} won’t work outside of a Jenkinsfile environment; I’ve mentioned them (and exported them) here for context and completeness.

Accepting the SDK licenses

In later steps, we’ll want to install various packages via the sdkmanager tool; installing a package requires us to explicitly accept the package’s license (and some packages have different licenses, hence there are several licenses). This step usually requires a human to type “yes” or “y” at the command line. However, we want this to run in a scripted environment without the need for human intervention, hence we need to accept all licenses ahead of time. The sdkmanager command has the --licenses option just for this case, however it also requires human interaction in that the licenses have to be accepted. It turns out, though, that one can simply pipe the command yes¹ into sdkmanager. So, to programmatically accept all Android SDK licenses, we have this one-liner:

# accept all SDK licenses, otherwise later processes will hang waiting for input
yes | sdkmanager --sdk_root=${env.ANDROID_HOME} --licenses

where we had to tell sdkmanager where its root directory is located because we installed the command line tools in a non-standard location.

Installing the required Android packages and system image

My first ever Android phone was a Nexus 5. Why is this relevant? Well, the app we wanted to build needed to run on old hardware (we couldn’t make too many assumptions about how up-to-date our users’ devices were) and I knew that this phone was old, so why not use it as a test device to ensure everything is working well? That phone had used Android 19, so what we need to do is install the Android 19 system image which can run on x86 hardware. Specifically, we need to install the system-images;android-19;default;x86 package. The command to do this is:

sdkmanager --sdk_root=${env.ANDROID_HOME} "system-images;android-19;default;x86"

We won’t be running this image on an actual device; we’ll be emulating it, hence we need to install the emulator package as well:

sdkmanager --sdk_root=${env.ANDROID_HOME} emulator

Because we want to build an APK of our app (we’ll get Cordova to do this for us later), we’ll also need the latest version of the build-tools package, i.e. we need this command:

sdkmanager --sdk_root=${env.ANDROID_HOME} "build-tools;31.0.0"

Now that the basic packages are installed, we’re ready to create an Android virtual device with which we can emulate the system image we just installed.

Creating an Android virtual device

Since we’re only emulating an Android device and not actually running code on a real one, we need to create an Android virtual device (AVD) to run the system image we set up in the previous section. Just to keep with the nostalgic feel of trying to target my old phone, let’s call this virtual device “nexus5” and use the “Nexus 5” device. One can get a list of all available devices by using avdmanager list. To create the virtual device we pass the create avd subcommand to the avdmanager command:

avdmanager create avd --name nexus5 --device "Nexus 5" --package "system-images;android-19;default;x86"

Although that will do the job the first time we run the command, it’ll cause an error the next time we run it because the device with the name “nexus5” will already exist. Since we’re scripting this, we want to avoid such unnecessary errors; hence we need to find out if the AVD already exists and only create it if it doesn’t. To find out which AVDs are available, we can use:

avdmanager list avd

which unfortunately produces output that is clumsy to parse in scripts. Fortunately, the developers have also added the --compact option which is specifically for use in scripts, therefore we only need to work out if the word “nexus5” appears in this output, and if not create the virtual device. The code we end up with looks like this:

# create an Android virtual device to emulate with this system image
AVDS=$(avdmanager list avd --compact)
nexus5_exists=$( echo "$AVDS" | grep nexus5 || [ "$?" = 1 ] )
if [ "$nexus5_exists" = "" ]
then
    avdmanager create avd --name nexus5 --device "Nexus 5" --package "system-images;android-19;default;x86"
fi

where setting the nexus5_exists variable is probably the only tricky bit.

After getting the list of AVDs, if we just grep through this list and don’t find the text we’re looking for, then grep will exit with an error (i.e. the shell variable $? will be set to a value other than 0) and the entire script will fail (which isn’t what we want). However, if the grep does find a match, then we want to use that output. To work around this issue, we use an or (||) to run code if the grep fails. The full explanation about how this construct works is in this StackExchange answer, but basically what happens is that if the grep matches, then we’ll get the matched text back; if it doesn’t match, then we get the empty string back. In the case that something goes horribly wrong with the grep command itself, then $? is set (correctly, this time) to a non-zero value so that the shell can exit.

To cut a long story short, we only need to create the AVD if the value of nexus5_exists is empty, which is what the body of the if conditional implements.

With this step complete, we’re now ready to set up the Cordova environment!

Setting up the Cordova environment

Getting Cordova installed is quite simple and painless. However, getting everything set up within a headless and scripted environment so that repeated runs “just work” can be a bit tricky. Let’s break this task down into smaller chunks, namely: installing Cordova and any required plugins; and working out which platforms have already been installed so that we can then install the Android platform if it hasn’t been installed already.

Installing Cordova and its plugins

A Cordova project is usually set up such that there’s a package.json file to install Cordova itself in the main project directory, then one changes into the app directory (usually one level deeper than the project directory) and there’s another package.json file which handles installation of the Cordova plugins as well as any other dependencies the app might have. Translating these steps into shell code, we get²:

# install cordova
npm install
cd aimee  # change into the app's main directory

# install cordova plugins
npm install

Note that the app’s name I’m using here is “AIMEE - the Arctic/Antarctic Ice Map EnginE”; this was the proof of concept app that I initially built while we were trying to work out if our ideas for such an app made sense and were worthwhile pursuing.

Working out which platforms have already been installed

Before we can build Android APKs with Cordova, we need to add the Android platform to Cordova. Now, because we’re doing this programmatically and want the process to run multiple times (we’re using this in our CI/CD environment after all, so this process will be run many times per day), we need to avoid trying to add the Android platform multiple times (otherwise we’ll get an error). Therefore, we need to work out which platforms have already been installed first. Cordova provides a command to display the installed and the available platforms:

export PATH="${env.PATH}:${env.CORDOVA_PATH}"  # add cordova programs to path
cordova platforms list

Unfortunately, this output doesn’t have output which is easily digestible by scripts, for instance (where the Android platform has already been installed):

Installed platforms:
  android 7.1.4
Available platforms:
  browser ~5.0.1
  ios ~4.5.4
  osx ~4.0.1
  windows ~6.0.0

We just want to know which platforms have been installed (if any) and one way to do this is to process this output line by line; if we see “Installed” on the line, then ignore it and go to the next line; if the next line does not start with “Available”, then we know we have an installed platform and can add this to a list of known installed platforms; as soon as we see “Available”, then we can ignore the rest of the output as no further installed platforms will be listed.

That was hard enough to describe in English, let alone try to tell a computer how to do it! Nevertheless, it’s possible. Here’s the solution I came up with:

# work out which platforms have already been installed
installed_platforms="";
cordova platforms list > platforms_list
while read -r line
do
    is_installed_section=\$( echo "\$line" | grep Installed || [ "\$?" = 1 ] )
    is_available_section=\$( echo "\$line" | grep Available || [ "\$?" = 1 ] )
    if [ "\$is_installed_section" != "" ]
    then
        continue
    elif [ "\$is_available_section" != "" ]
    then
        break
    else
        line=\$(echo "\$line" | sed 's/^\\s+//g')
        installed_platforms=\$(echo -e "\$installed_platforms\\n\$line")
    fi
done <platforms_list

We first initialise a variable to store the installed platforms that we find. Then we redirect the output of cordova platforms list into a file, which we read into the while loop (see the redirection at the end of the loop) and process line by line.

For each line read, we check to see if the line denotes the “Installed platforms:” or “Available platforms:” section. If we detect the “Installed platforms:” section, we skip the line and continue to the next line in the input. If we detect the “Available platforms:” section, we break out of the loop entirely. Otherwise we append the current line to the installed_platforms variable so that we can keep track of the currently installed platforms (if any).

One would be forgiven for thinking that just piping the output of the cordova platforms list command into the while loop would be sufficient and that the redirection to a file would be unnecessary, i.e. by doing something like this:

cordova platforms list | while read -r line ...

This would certainly make the order of processing more obvious. However, any variables set within the while loop (such as installed_platforms) are lost because a pipe starts a subprocess and any variables created within the pipe are lost to the parent process and hence we won’t have access to the list of platforms that the while loop detected. This is why the command’s output is redirected to a file and then read to the while loop (also by redirection).

One would also be forgiven for thinking that one could use process substitution to redirect the output of cordova platforms list directly into the while loop, i.e. something along these lines:

while read -r line
do
    # stuff to do
done < <(cordova platforms list)

however this is only a Bash construct and is hence not POSIX compliant. Another reason for avoiding process substitution is because the machine I’m running Jenkins on is Debian, and Jenkins uses plain Bourne shell (a.k.a ‘sh’) for its shell-executed regions, and Debian uses dash as its preferred variant of sh, and dash doesn’t support process substitution. I realise I could have added a bash shebang line to force bash to be used, i.e. by adding a line like the following to the beginning of the Jenkinsfile shell block:

!#/bin/bash

but that would have removed all of the shell options that were set up by Jenkins to warn me when something goes wrong, or I could have configured Jenkins to use bash instead of sh (but that could have caused problems in lots of other projects), however (in my experience) it’s much easier long-term to stick to the defaults a given environment provides. Also, I got to learn a lot about why various things I tried to do didn’t work and hence have a better understand of how shells work now.

Another way to get around the more restrictive nature of the default shell would be to wrap all of the shell code into a script and just call the script rather than go through the pain of trying to get everything to work in a Jenkinsfile shell block. One goal of this article is to provide a full Jenkinsfile with the entire implementation so that one can see everything within the one context, therefore I didn’t want to hide everything in shell scripts, but wanted the gory details displayed for all to see.

Ok, I think that’s enough prattling on from my side about why I chose to do the extra work here. We’re now ready do install the Android platform if we need to, so let’s do that.

Installing the Android platform for Cordova

Now that we know which platforms have been installed, we just need to see if the word “android” appears and add the platform to Cordova if it’s not present, i.e.:

# add Android platform if it isn't installed
android_is_installed=$( echo "$installed_platforms" | grep android || [ "\$?" = 1 ] )
if [ "$android_is_installed" = "" ]
then
    cordova platforms add android
fi

And that’s it! Now Cordova is set up to use Android as its target platform.

Running the test suite

This is just a simple matter of running

npm run test

so this isn’t really as interesting as getting everything set up.

Building the distribution APK

Since we’ve done all of the hard work of setting up Android and Cordova, building the Android APK of the app is a simple matter of running

cordova build android

which will build a “debug” version of the app; one that is unsigned and hence not able to be uploaded to the Google Play Store. However you will be able to install it on your own Android device if you download the APK file and grant permissions for it to be installed.

Because the output of the build process is a debug version of the app, it will be called (rather unimaginatively) app-debug.apk irrespective of how you set things up in Cordova, therefore I have an extra step to rename the file to something sensible when building APKs, e.g.:

# give the output file a more sensible name
OUTPUT_PATH=${env.WORKSPACE}/aimee/platforms/android/app/build/outputs/apk/debug
mv \$OUTPUT_PATH/app-debug.apk \$OUTPUT_PATH/aimee-debug.apk

If we use the archiveArtifacts directive within the Jenkinsfile, we can make the APK available for download from the Jenkins server, this way we can download and install the latest build on a real Android device to ensure that it works as expected on real hardware.

Putting it all together

Now that we have all of the pieces of the puzzle, let’s put them together as part of a Jenkinsfile. The following file is what I ended up using to build our original proof-of-concept application; the pre-pre-cursor to what has since become IcySea.³

pipeline {
    agent any

    options {
        lock('aimee-test-and-build')
    }

    triggers {
        pollSCM('H/5 * * * *')
    }

    environment {
        ANDROID_HOME = "${env.WORKSPACE}/android_sdk"
        ANDROID_TOOLS = "${env.ANDROID_HOME}/cmdline-tools/bin"
        CORDOVA_PATH = "${env.WORKSPACE}/node_modules/cordova/bin"
        PATH = "${env.PATH}:${env.ANDROID_TOOLS}:${env.CORDOVA_PATH}"
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        stage('Set up Android environment') {
            steps {
                echo 'Preparing Android environment..'
                sh """
                # install Android SDK tools if necessary
                if [ ! -d "${env.ANDROID_HOME}" ]
                then
                    mkdir "${env.ANDROID_HOME}"
                    cd "${env.ANDROID_HOME}"
                    wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
                    unzip commandlinetools-linux-7583922_latest.zip
                    cd "${env.WORKSPACE}"
                fi

                # accept all SDK licenses, otherwise later processes will hang waiting for input
                yes | sdkmanager --sdk_root=${env.ANDROID_HOME} --licenses

                # install the Android system image to use
                sdkmanager --sdk_root=${env.ANDROID_HOME} "system-images;android-19;default;x86"
                # install the supporting packages
                sdkmanager --sdk_root=${env.ANDROID_HOME} emulator  # required so images can be emulated
                sdkmanager --sdk_root=${env.ANDROID_HOME} "build-tools;31.0.0"  # required so cordova can build app

                # create an Android virtual device to emulate with this system image
                AVDS=\$(avdmanager list avd --compact)
                nexus5_exists=\$( echo "\$AVDS" | grep nexus5 || [ "\$?" = 1 ] )
                if [ "\$nexus5_exists" = "" ]
                then
                    avdmanager create avd --name nexus5 --device "Nexus 5" --package "system-images;android-19;default;x86"
                fi
                """
            }
        }
        stage('Prepare Cordova') {
            steps {
                echo 'Preparing Cordova environment..'
                sh """
                # install cordova
                npm install
                cd aimee  # change into the app's main directory

                # install cordova plugins
                npm install

                # work out which platforms have already been installed
                installed_platforms="";
                cordova platforms list > platforms_list
                while read -r line
                do
                    is_installed_section=\$( echo "\$line" | grep Installed || [ "\$?" = 1 ] )
                    is_available_section=\$( echo "\$line" | grep Available || [ "\$?" = 1 ] )
                    if [ "\$is_installed_section" != "" ]
                    then
                        continue
                    elif [ "\$is_available_section" != "" ]
                    then
                        break
                    else
                        line=\$(echo "\$line" | sed 's/^\\s+//g')
                        installed_platforms=\$(echo -e "\$installed_platforms\\n\$line")
                    fi
                done <platforms_list

                # add Android platform if it isn't installed
                android_is_installed=\$( echo "\$installed_platforms" | grep android || [ "\$?" = 1 ] )
                if [ "\$android_is_installed" = "" ]
                then
                    cordova platforms add android
                fi
                """
            }
        }
        stage('Test') {
            steps {
                echo 'Running tests..'
                sh """
                cd aimee
                npm run test
                """
            }
        }
        stage('Build Android APK') {
            when {
                expression {
                    "${env.BRANCH_NAME}" == 'master'
                }
            }
            steps {
                echo 'Building distribution package..'
                sh """
                cd aimee
                cordova build android

                # give the output file a more sensible name
                OUTPUT_PATH=${env.WORKSPACE}/aimee/platforms/android/app/build/outputs/apk/debug
                mv \$OUTPUT_PATH/app-debug.apk \$OUTPUT_PATH/aimee-debug.apk
                """
            }
        }
    }

    post {
        failure {
            emailext to: 'aimee@example.com',
                subject: "Jenkins build failed: ${JOB_NAME} ${BRANCH_NAME} #${BUILD_NUMBER}",
                body: "Please go to ${BUILD_URL} and verify the build",
                attachLog: true,
                recipientProviders: [[$class: 'CulpritsRecipientProvider']]
        }
        fixed {
            emailext to: 'aimee@example.com',
                subject: "Jenkins build back to normal: ${JOB_NAME} ${BRANCH_NAME} #${BUILD_NUMBER}",
                body: "Jenkins build back to normal: ${JOB_NAME} ${BRANCH_NAME} #${BUILD_NUMBER}",
                attachLog: true,
                recipientProviders: [[$class: 'CulpritsRecipientProvider']]
        }
        success {
            archiveArtifacts artifacts: "aimee/platforms/android/app/build/outputs/**/*.apk", fingerprint: true
        }
    }
}

// vim: expandtab shiftwidth=4 softtabstop=4

Let’s go through this quickly:

• the entire Jenkins job is wrapped in a pipeline, including job settings, environment variables, the stages to run in the pipeline, and finally what to do at the end of a pipeline if it works, or if it doesn’t.
• agent any just tells Jenkins to run this pipeline on any available agent.
• the lock option is so that we don’t inadvertently run multiple builds of the same project simultaneously; this was before we’d moved to running builds in Docker containers.
• the triggers section on line 8 tells Jenkins to poll the Git repository every 5 minutes for any new commits. At the time we were doing this, we were using gitolite to host our Git repositories and it wasn’t possible to trigger a Jenkins build from a push event to the repository, hence the somewhat less efficient polling method.
• line 12 starts the environment block which holds all environment variables that we want available in the later shell (sh) directives. Important things here are the location of the Android “home” directory, the location of the Android SDK command line programs, the location of the Cordova programs, as well as extending the shell’s PATH with these locations so that the programs can be found.
• the stages directive on line 19 contains all of the stages to run in the pipeline.
• we check out the source code from Git (a.k.a. SCM: source code management).
• then we set up the Android environment as described in detail earlier.
• Cordova is prepared from line 58 onwards as described earlier.
• we run the test suite in the ‘Test’ stage.
• and then we finally build the Android APK from line 106 onwards as described above. Note that this is only done if we’re on the master branch (we didn’t need feature branches to create an APK; only code that had gotten through review made its way to master and hence only master needed to build any packages).
• the post section on line 126 shows who should be notified if a failure occurs, or who to notify if things get back to normal (i.e. a build has been fixed).
• all successful builds (line 141) archive the APK so that it can be downloaded from the project’s page in Jenkins.

If you were reading carefully, you will have noticed many dollar signs ($) being escaped by a backslash (\) within the shell directives (sh """ ...). The reason for this is that Groovy (the language that is used for the Jenkinsfile) uses dollar signs for variables as well as the shell, hence in order to pass the dollar signs through to the shell without being unnecessarily and incorrectly interpreted, they have to be escaped. This is also the case for instances of command substitution ($()); they need to have their dollar sign escaped so that they don’t get interpreted as being for Groovy.

All of this escaping does mean that one has to be very careful when mixing shell code with Jenkinsfile Groovy constructs. In normal use it is likely to be much better to wrap the shell code into a shell script and just call the script directly rather than have everything explicit in the Jenkinsfile. However, for the purposes of this article, it’s helpful to present the Jenkinsfile as a standalone document. This allows one to see how everything works at a single glance; it also highlights the issues one can bump into when mixing shell code within a Jenkinsfile.

I definitely learned something in the process of getting this to work and I hope you learned something too!

Summary

The main take-aways from this wall of text are:

• it’s not necessary to rely on a full GUI application such as Android Studio to build an Android app; it’s possible to do so with command line tools.
• Android apps can be built rather simply with Cordova in a headless Jenkins environment.
• spending the time to work out why particular constructs don’t work within a command line environment such as the shell and working out how to solve a problem using different constructs can be very illuminating and can lead to a much deeper understanding of the shell and the differences between shells.

Is there anything that I’ve missed? Was this post helpful? How could I make it better? Let me know by dropping me a line via email or pinging me on Mastodon.

1. Yes, that’s an actual Linux command; it repeatedly outputs the text ‘y’ followed by a newline to stdout, thus simulating a user typing ‘y’ into the terminal. ↩

2. I’m assuming you’re familiar with creating JavaScript apps with NodeJS and NPM. ↩

3. With a version update to use the most recent Android SDK command line tools package. ↩

Support

If you liked this post and want to see more like this, please buy me a coffee!