Developing reliable integration tests requires you to control every part of your execution environment, from the operating system up to the source code itself. Running your tests inside containers takes extra work up front, so why do it?
Your environment is defined
Could you tell me what packages are on your local machine? Could you tell me what your version of chrome is at this moment? Me neither. Could I tell you what I’m running in my containers?
In a heartbeat.
There's no conflicting node packages, no chance that the browser updates, no chance you installed a conflicting node package, you know exactly what is there.
You limit extraneous problems in your testing
A problem I used to get constantly was
(Webdriver Exception: Cannot find a matching set of capabilities).
This is a useless error message.
If you don’t know what the problem is (this error was trying to tell me my firefox, geckodriver and/or selenium versions were incompatible), it gives you no guidance on solving the problem.
You can match your environment to your pipeline
When you run your tests in a pipeline you spin up a container. That container is defined.
How can your tests truly align to what’s in the pipeline if you can't even know your environment?
You can create a container that matches your pipeline.
The hurdles
As always when trying new things there are always hurdles. Here are some problems that I faced when I started this journey.
You need to learn how to containerize your environment
You need to get used to the concept of thinking about your environment. You can't take it for granted.
What version of Firefox do you want to use?
What version of python or node?
You should already know the answer to these, but until you do you can't easily containerize your tests.
You need to know the environment of the base image
The first image I used was fedora. I picked this image because I thought it
was light weight and it would be similar to Ubuntu.
Unfortunately I didn’t know how to do things different in Fedora. It caused a few problems and slowed me down.
So if you are creating an image make sure you understand the environment you are using.
How to containerize your first test
First you need to install docker.
Once you have installed Docker you should get to building your Dockerfile. I
chose ubuntu for my base image but if you wanted to go somthing a little more
lightweight you could go alpine or mint.
FROM ubuntu:16.04Install your dependencies. Robot Framework
requires python, so we need to install that first.
RUN apt-get update && apt-get install --quiet --assume-yes \
python3 python3-pip unzipNext install robot via pip3.
RUN pip3 install --upgrade robotframeworkLast step is to copy your tests into the container. In this case, copy
everything in the same directory as the Dockerfile.
COPY . /robotYour Dockerfile should look like this. Name it Dockerfile and place in the
base directory.
FROM ubuntu:16.04
RUN apt-get update && apt-get install --quiet --assume-yes \
python3 python3-pip unzip firefox wget
RUN pip3 install --upgrade robotframework
COPY . /robotThis blog isn't about robot, so I'm going to just supply a basic test that
builds and passes right away.
Create a file called my-first-test.robot and place it next to the Dockerfile
you just made.
*** Variables ***
${test} Pass
*** Test Cases ***
Test
Pass Execution ${test}Finally with all this done you can build and run the container.
# Build the image and tag it as myfirstdockertest
$ docker build -t myfirstdockertest .
# Start the container with the image we just built
# We're also mounting our current directory
$ docker run --name myfirstdockertest -td -v ${PWD} myfirstdockertest
# Jump into the container and run the robot test
$ docker exec -t myfirstdockertest robot robot/my-first-test.robot
# Stop the container
$ docker stop myfirstdockertest
# Remove the container
$ docker container rm myfirstdockertestAfter it runs, you should see this in the console.
===========================================================================
My-First-Test
===========================================================================
Test | PASS |
Pass
---------------------------------------------------------------------------
My-First-Test | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
===========================================================================Thats a pass!
Further reading
If you want to understand some more of what's going on here I have compiled a list of further reading of things that I found interesting.
