Testing SMS 2FA with Twilio and Playwright

A few years ago I had the experience of having a login page with mandatory two-factor auth (2FA) for a client. A few devs and I got into a room and talked about possible solutions, all the standard ones came out:

• Turn off 2FA for the test user.

• Turn off the 2FA feature for the solution in development environments.

• Accept a default 2FA token in development environments.

All of these solutions have two glaring problems:

• they don't actually test the 2FA System Under Test (SUT to us cool kids).

• they increase the complexity of the SUT for the purposes of testing the Application.

Our gold standard should be to test all parts of the solution and avoid working around areas that feel too complex to test. While we were discussing the pros and cons of the above solutions, one of the devs in the room - Dave Wales, a great guy - came out and said "Why don't we just use Twilio?".

Enter Twilio

Twilio is a SaaS solution for automating aspects of your customer relationship. cool platform, nothing to do with testing. However, among the suite of products that Twilio offers is a neat little automation tool called Twilio SMS. This is a programmable API that is attached to a phone number from any country or region.

The messages API allows us to get a notification using a listener event when we are running a test, get the message and because we know the format we can extract the 2FA code from it and input it into the application.

This solves the problem we had above where we wanted to test the login into our site but didn't want to add unnecessary complexity to the solution.

How to get started

Firstly and most importantly, you will need a Twilio account with an active number. If you can send messages to the U.S. you can sign up for a trial account with a U.S. or Canadian number.

Then you will need to go into the console and extract the Account SID and the Auth Token. These should be sitting right there on the console. Add them to wherever you store your environment secrets.

You can use different mobile numbers to run parallel tests but as I'm using just one, I don't have that extra step in the testing.

Installing components

We are going to be using playwright for this example, mainly because that's the the framework I use and I'm too lazy to change it.

So if you are starting a new project:

npm init playwright

Just use the defaults unless you have a reason not to.

Then we want to install Twilio:

npm install twilio

You can also use the quick start guide if you need some more context on installation.

Building the tests

First, we need to do all of our imports and set our constants we will be using for the application:

import { test, expect } from "@playwright/test";
import { checkDifferentSid, sleep } from "../src/helpers";
const accountSid = process.env.accountSid;
const authToken = process.env.authToken;
const client = require("twilio")(accountSid, authToken);

We want to create a helper function, a function we use to poll the SMS feature:

export async function checkForDifferentSid(
  client: any,
  record: any
): Promise {
  const response = await client.messages.list();
  return response[0].sid != record.sid;
}

Finally, let's create the test. The pseudocode would be as follows:

• Before the test is run, we want to get the latest record sid.

• When the test starts we want to send off a 2FA code to the twilio number.

• We then want to poll the messages API to detect a difference in the latest message.

• When we detect a difference we want to extract the code and use it on the page.

Here is what the final code might look like:

import { test, expect } from "@playwright/test";
import { checkDifferentSid, sleep } from "../src/helpers";
const accountSid = process.env.accountSid;
const authToken = process.env.authToken;
const client = require("twilio")(accountSid, authToken);
 
let record: any;
 
test.beforeAll(async () => {
  const response = await client.messages.list();
  record = await response[0];
});
 
test("Testing a 2fa message", async () => {
  test.setTimeout(10000);
 
  //Here is where you would send off the 2FA test and wait for a response
 
  let tries = 0;
  let result: boolean;
  while (tries < 5 && !(result = await checkForDifferentSid(client, record))) {
    tries++;
    await sleep(1000);
  }
  expect(result).toEqual(true);
 
  // Here you could add an expect body to match a predetermined string
  // Here is where we would extract the code from the SMS and use it in the SUT
});

Now there are some gaps here for you to add your own features. Even at this level you can test that the SMS 2FA has been received by Twilio.

Running the test

Now we run the test. For my example in the middle of a test I am sending a text message to prove the concept, but you would add in sending the 2FA.

Adding in things such as, checking the body for the correct copy, and making sure the code is being generated correctly. Finally checking that the incorrect code and resend functions are all working are all reasonable tests to add to this feature.

Limitations of this approach

There are a few limitations with this approach. This is fine for running serial tests, but polling the SMS will fail on parallel runs. Standing up an event listener, or using multiple SMS numbers would alleviate this problem but scaling can be an issue.

Another limitation is performance. This approach uses infrastructure that can have poor performance (anyone who has worked in telecommunications knows this). If performance is more important to you than complete coverage or if the 2FA feature is not a critical feature, this approach might be too slow for your test suite.

Finally, this solution makes use of a polling approach that implements a sleep function. This is fine as a proof of concept but for a more robust approach using the above-mentioned listener function would result in much more reliable performance.

Source Files

If you want to have a look at a repo that shows this code in action. Look no further than here!

Anyway, thank you for reading this little blog. I hope that it shone a light on an often overlooked testing activity.