In the first part of this article, we described how to quickly and easily build an infrastructure for running UI tests on Android using Appium and Selenoid. We are continuing our story to explain how we incorporated the launch of UI tests in iOS into the process.

Scaling with GGR

The maximum number of parallel workflows within a host is limited by its resources. Thus, we needed a tool for combining multiple hosts into one cluster. For this, we use the Go Grid Router (GGR) from the folks at Aerokube. Based on the description in the documentation, GGR is a load balancer used for creating scalable and highly available Selenium clusters.


The project with the tests runs a query in GGR as part of the process used. It polls the Selenoid parameters specified in its configuration and distributes the load among them based on the platform used, the availability of free flows, and the predefined specific weight of each Selenoid.

Deploying GGR and GGR UI is easy:

$ cat /etc/grid-router/quota/test.xml
<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="android" defaultVersion="10.0" defaultPlatform="android">
  <version number="10.0">
    <region name="1">
      <host name="0.0.0.0" port="4444" count="1"/>
    </region>
  </version>
</browser>
</qa:browsers>


docker run -d \
  --name ggr \
  -v /etc/grid-router/:/etc/grid-router:ro \
  --net host aerokube/ggr:latest-release \
  -listen=:4445 -guests-allowed


val driver = AndroidDriver(URL("http://localhost:4445/wd/hub"), capabilities)


docker run -d \
  --name ggr_ui \
  -p 8888:8888 \
  -v /etc/grid-router/quota:/etc/grid-router/quota:ro \
  aerokube/ggr-ui:latest-release


docker run -d \
  --name selenoid-ui \
  -p 4446:4446 \
  --link selenoid:selenoid \
  aerokube/selenoid-ui:1.10.4 \
  --selenoid-uri "<http://ggr-ui:8888>"

Our Selenoid UI should now display the status of all Selenoid clusters connected to GGR.

Now, we proceed to the running of tests on iOS

We use our own Mac mini farm to run UI tests on iOS. The farm can similarly be assembled from decommissioned but operational MacBooks. Alternatively, they can be rented. The following needs to be installed on each host:

Some early problems

We were unable to replicate the structure used in running Android tests because we couldn’t find a way to run iOS simulators in Docker containers. One option that we considered was running Docker-OSX, but we ran into doubts about the legality of its use for any purposes unrelated to OS X Security Research. So, we decided to go a different route.

Iteration #1: GGR→Appium

We added Appium (port 4723) as a Selenoid host for iOS tests in the previously created GGR config file:

<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="android" defaultVersion="10.0" defaultPlatform="android">
  <version number="10.0">
    <region name="1">
      <host name="0.0.0.0" port="4444" count="1"/>
    </region>
  </version>
</browser>
<browser name="iPhone 14" defaultVersion="16.2" defaultPlatform="iOS">
  <version number="16.2">
    <region name="1">
      <host name="0.0.0.0" port="4723" count="1"/>
    </region>
  </version>
</browser>
</qa:browsers>

In such a case, the iOS scheme looks like this:

The structure used in this iteration is operational. The problem is that, in this case, we can only run tests in one workflow on each Mac mini, which is wasteful. Also, the cluster will not be displayed in the Selenoid UI.

Iteration #2: GGR→Selenoid→Appium

Selenoid allows you to work with more than just containers. The above problems informed our decision to use Selenoid when running iOS tests as well, though as an executable file:

{
  "iPhone 14": {
    "default": "16.2",
    "versions": {
      "16.2": {
        "image": ["appium", "--log-timestamp", "--log-no-colors", ...]
      }
    }
  }
}

Now, the process of the cluster looks like this:

With this approach, we partially achieve Selenoid UI functionality and the ability to run tests across multiple flows on the same host.

The downside is that, on each Mac mini, you have to manually carry out a multitude of routine tasks to create a simulator and link it with Appium by specifying the UUID and port assignment. This can become a problem if you need to upgrade to a new iOS version later.

Iteration #3: GGR→Selenoid→Bash→Appium

We have a large Mac mini farm that will continue growing as time goes on. With this in mind, we were looking for a way to make scaling easier so that we wouldn’t have to create simulators by hand and then connect them to Appium. With the previous schema in place, Appium and simulators would have had long lifetimes, which could have led to unpredictable consequences.

Searching for a solution, we discovered that a bash script can be specified as a host in the Selenoid configuration file:

{
  "iPhone 14": {
    "default": "16.2",
    "versions": {
      "16.2": {
        "image": ["~/bin/config/start_appium.sh", "iPhone 14"]
      }
    }
  }
}

This is what ours looks like:

#!/bin/bash

set -ex

DEVICE_NAME=$1
APPIUM_PORT=$(echo $2 | cut -d '=' -f 2)

function clean() {
  if [ -n "$APPIUM_PID" ]; then
      kill -TERM "$APPIUM_PID"
  fi
  if [ -n "$DEVICE_UDID" ]; then
      xcrun simctl delete $DEVICE_UDID
  fi
}

trap clean SIGINT SIGTERM

# Each simulator has a udid, so to run the same devices in parallel - clone and run
# only clones. You cannot clone a running device. After closing the session, delete the clone.
cloned_device_name="[APPIUM] ${DEVICE_NAME} ($(date +%Y%m%d%H%M%S))"
DEVICE_UDID=$(xcrun simctl clone "$DEVICE_NAME" "$cloned_device_name")

# https://github.com/appium/appium-xcuitest-driver#important-simulator-capabilities
WDA_LOCAL_PORT=$(($APPIUM_PORT+1000))
MJPEG_SERVER_PORT=$(($WDA_LOCAL_PORT+1000))
DEFAULT_CAPABILITIES='"appium:udid":"'$DEVICE_UDID'","appium:automationName":"'XCUITest'","appium:wdaLocalPort":"'$WDA_LOCAL_PORT'","appium:mjpegServerPort":"'$MJPEG_SERVER_PORT'"'

appium --base-path=/wd/hub --port=$APPIUM_PORT --log-timestamp --log-no-colors --allow-insecure=get_server_logs,adb_shell \
       --allow-cors --log-timestamp --log {choose_directory_for_logs} \
       --default-capabilities "{$DEFAULT_CAPABILITIES}" &
APPIUM_PID=$!

wait

If the script is used, pay close attention to the stated capabilities and Appium startup settings. These are set up here assuming that Appium 2.x is used for the run ‒ Appium 1.x does not require the vendor to be specified in capabilities, and there is no option of specifying --base-pat.

The script solves the problem of simulators running in parallel:

The use of Selenoid makes it possible to simplify this process down to a single script that does not create conflicts between multiple Appium + Simulator pairs within a single host. It launches Appium and kills it when receiving a corresponding signal from Selenoid, and it dynamically clones the simulators at startup and deletes them once the session ends.

The process we developed looks like this:

Next, we add the Selenoid addresses of each Mac mini to the configuration file of the deployed GGR, merging the Android and iOS structures:

Bottom-line summary

The assembled infrastructure allows us to run a total of about 500 UI tests per hour across 36 workflows on both platforms. Addition of a new host for Android tests can be fully automated by using the workflow on GitHub Actions, and takes about two minutes. In the very near future, we plan to automate the deployment of the Selenoid cluster on a Mac mini as well.

Further down the road, we would like to try out running Docker-OSX containers on a Mac mini with Linux to unify all the processes and make it easier to deploy to them without breaking any MacOS usage rules. If you have had any related experience, we'd love for you to share it in the comments.

Posted by Ivan Grigoriev.