Running headless Selenium WebDriver tests in Docker containers

Grig Gheorghiu
7 min readJan 8, 2016

--

In my previous post, I showed how to install firefox in headless mode on an Ubuntu box and how to use Xvfb to allow Selenium WebDriver scripts to run against firefox in headless mode.

Here I want to show how run each Selenium test suite in a Docker container, so that the suite gets access to its own firefox browser. This makes it easy to parallelize the test runs, and thus allows you to load test your Web infrastructure with real-life test cases.

Install docker-engine on Ubuntu 14.04

We import the dockerproject.org signing key and apt repo into our apt repositories, then we install the linux-image-extra and docker-engine packages.

# apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D# echo “deb https://apt.dockerproject.org/repo ubuntu-trusty main” > /etc/apt/sources.list.d/docker.list# apt-get update# apt-get install linux-image-extra-$(uname -r)# apt-get install docker-engine

Start the docker service and verify that it is operational

Installing docker-engine actually starts up docker as well, but to start the service you do:

# service docker start

To verify that the docker service is operational, run a container based on the public “hello-world” Docker image.

# docker run hello-worldUnable to find image ‘hello-world:latest’ locallylatest: Pulling from library/hello-worldb901d36b6f2f: Pull complete0a6ba66e537a: Pull completeDigest: sha256:8be990ef2aeb16dbcb9271ddfe2610fa6658d13f6dfb8bc72074cc1ca36966a7Status: Downloaded newer image for hello-world:latestHello from Docker.This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps:1. The Docker client contacted the Docker daemon.2. The Docker daemon pulled the “hello-world” image from the Docker Hub.3. The Docker daemon created a new container from that image which runs theexecutable that produces the output you are currently reading.4. The Docker daemon streamed that output to the Docker client, which sent itto your terminal.To try something more ambitious, you can run an Ubuntu container with:$ docker run -it ubuntu bashShare images, automate workflows, and more with a free Docker Hub account:https://hub.docker.comFor more examples and ideas, visit:https://docs.docker.com/userguide/

Pull the ubuntu:trusty public Docker image

# docker pull ubuntu:trustytrusty: Pulling from library/ubuntufcee8bcfe180: Pull complete4cdc0cbc1936: Pull completed9e545b90db8: Pull completec4bea91afef3: Pull completeDigest: sha256:3a7f4c0573b303f7a36b20ead6190bd81cabf323fc62c77d52fb8fa3e9f7edfeStatus: Downloaded newer image for ubuntu:trusty# docker imagesREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEubuntu trusty c4bea91afef3 3 days ago 187.9 MBhello-world latest 0a6ba66e537a 12 weeks ago 960 B

Build custom Docker image for headless Selenium WebDriver testing

I created a directory called selwd on my host Ubuntu 14.04 box, and in that directory I created this Dockerfile:

FROM ubuntu:trustyRUN echo “deb http://ppa.launchpad.net/mozillateam/firefox-next/ubuntu trusty main” > /etc/apt/sources.list.d//mozillateam-firefox-next-trusty.list
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE49EC21
RUN apt-get update
RUN apt-get install -y firefox xvfb python-pip
RUN pip install selenium
RUN mkdir -p /root/selenium_wd_tests
ADD sel_wd_new_user.py /root/selenium_wd_tests
ADD xvfb.init /etc/init.d/xvfb
RUN chmod +x /etc/init.d/xvfb
RUN update-rc.d xvfb defaults
CMD (service xvfb start; export DISPLAY=:10; python /root/selenium_wd_tests/sel_wd_new_user.py)

This Dockerfile tells docker, via the FROM instruction, to create an image based on the ubuntu:trusty image that we pulled before (if we hadn’t pulled it, it would be pulled the first time our image was built).

The various RUN instructions specify commands to be run at build time. The above instructions add the Firefox Beta repository and key to the apt repositories inside the image, then install firefox, xvfb and python-pip. Then they install the selenium Python package via pip and create a directory structure for the selenium tests.

The ADD instructions copy local files to the image. In my case, I copy one Selenium WebDriver Python script, and an init.d-type file for starting Xvfb as a service (by default it starts in the foreground, which is not something I want inside a Docker container).

The last two RUN instructions make the /etc/init.d/xvfb script executable and run update-rc.d to install it as a service. The xvfb script is the usual init.d wrapper around a command, in my case this command:

PROG=”/usr/bin/Xvfb”
PROG_OPTIONS=”:10 -ac”

Here is a gist for the xvfb.init script for reference.

Finally, the CMD instruction specifies what gets executed when a container based on this image starts up (assuming no other commands are given in the ‘docker run’ command-line for this container). The CMD instruction in the Dockerfile above starts up the xvfb service (which connects to DISPLAY 10 as specified in the xvfb init script), sets the DISPLAY environment variable to 10, then runs the Selenium WebDriver script sel_wd_new_user.py, which will launch firefox in headless mode and execute its commands against it.

Here’s the official documentation for Dockerfile instructions.

To build a Docker image based on this Dockerfile, run:

# docker build -t selwd:v1 .

selwd is the name of the image and v1 is a tag associated with this name. The dot . tells docker to look for a Dockerfile in the current directory.

The build process will take a while intially because it will install all the dependencies necessary for the packages we are installing with apt. Every time you make a modification to the Dockerfile, you need to run ‘docker build’ again, but subsequent runs will be much faster.

Run Docker containers based on the custom image

At this point, we are ready to run multiple containers based on the selwd image we created above.

Here’s how to run a single container:

# docker run --rm selwd:v1

In this format, the command specified in the CMD instruction inside the Dockerfile will get executed, then the container will stop. This is exactly what we need: we run our Selenium WebDriver tests against headless firefox, inside their own container isolated from any other container.

The output of the ‘docker run’ command above is:

Starting : X Virtual Frame Buffer .
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Ran 1 test in 40.441s
OK

(or a traceback if the Selenium test encountered an error)

Note that we also specified the rm flag to ‘docker run’ so that the container gets removed once it stops — otherwise these short-lived containers will be kept around and will pile up, as you can see for yourself if you run:

# docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c9673e59585 selwd:v1 “/bin/bash” 5 minutes ago Exited (130) 5 seconds ago modest_mccarthy
980651e1b167 selwd:v1 “/bin/sh -c ‘(service” 9 minutes ago Exited (0) 8 minutes ago stupefied_turing
4a9b2f4c8c28 selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago nostalgic_ride
9f1fa953c83b selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago admiring_ride
c15b180832f6 selwd:v1 “/bin/sh -c ‘(service” 13 minutes ago Exited (0) 12 minutes ago jovial_booth
.....etc

If you do have large numbers of containers that you want to remove in one go, use this command:

# docker rm `docker ps -aq`

For troubleshooting purposes, we can run a container in interactive mode (with the -i and -t flags) and specify a shell command to be executed on startup, which will override the CMD instruction in the Dockerfile:

docker run -it selwd:v1 /bin/bash
root@6c9673e59585:/#

At the bash prompt, you can run the shell commands specified by the Dockerfile CMD instruction in order to see interactively what is going on. The official ‘docker run’ documentation has lots of details.

One other thing I found useful for troubleshooting Selenium WebDriver scripts running against headless firefox was to have the scripts take screenshots during their execution with the save_screenshot command:

driver.save_screenshot(“before_place_order.png”)# Click Place Order
driver.find_element_by_xpath("//*[@id='order_submit_button']").click()
driver.save_screenshot(“after_place_order.png”)

I then inspected the PNG files to see what was going on.

Running multiple containers for load testing

Because our Selenium WebDriver tests run isolated in their own Docker container, it enables us to run N containers in parallel to do a poor man’s load testing of our site.

We’ll use the -d option to ‘docker run’ to run each container in ‘detached’ mode. Here is a bash script that launches COUNT Docker containers, where COUNT is the 1st command line argument, or 2 by default:

#!/bin/bashCOUNT=$1
if [ -z “$COUNT” ]; then
COUNT=2
fi
for i in `seq 1 $COUNT`; do
docker run -d selwd:v1
done

The output of the script consists in a list of container IDs, one for each container that was launched.

Note that if you launch a container in detached mode with -d, you can’t specify the rm flag to have the container removed automatically when it stops. You will need to periodically clean up your containers with the command I referenced above (docker rm `docker ps -aq`).

To inspect the output of the Selenium scripts in the containers that were launched, first get the container IDs:

# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6fb931689c03 selwd:v1 “/bin/sh -c ‘(service” About an hour ago Exited (0) About an hour ago grave_northcutt
1b82ef59ad46 selwd:v1 “/bin/sh -c ‘(service” About an hour ago Exited (0) About an hour ago admiring_fermat

Then run ‘docker logs <container_id>’ to see the output for a specific container:

# docker logs 6fb931689c03
.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Ran 1 test in 68.436s
OK
Starting : X Virtual Frame Buffer

Have fun load testing your site!

--

--

Grig Gheorghiu

DevOps, cloud computing, Python and Golang programming, data science, automated testing.