When Playwright met PollyJS...
Writing E2E tests with minimal dev efforts
The Beginning:
Hi there,
I recently worked on building an E2E testing framework with the only objective in my mind - "I want to write E2E tests fast. Very fast."
I am sharing my approach and learnings in this blog post. This approach is amalgam of 2 amazing softwares - Playwright and PollyJS. Hope you learn something new.
Chapter 1: Why write E2E tests?
E2E tests make sure that our app is working as intended to work.
I have seen many cases where major production issues could have been prevented if we had simple E2E tests in place such as:
- Button is clickable
- URL Route is changing after clicking a tab
- There is no infinite loader/spinner sitting on top of our page.
- Many more..
Does E2E tests means testing with actual Backend API's?
- Yes that is ideal! But let's be honest, it is not possible for backend to provide a stable testing environment (mostly).
- Hence, since Backend env is not available - we go with mocking API’s for E2E tests.
- In our case, along with testing UI we also had to test our BFF (Backend For Frontend) server which is actually a graphQL server and has a heavy resolver logic.
In summary, E2E tests prevents you from writing RCA's! 💁
Chapter 2: Developer's Pain
Following are the most popular pain points faced by a Developer while writing E2E tests:
Time Consuming:
- Writing tests is time consuming as we have to add a custom attribute like "data-testid", etc. to so many UI element (div, span, button, etc) in code so that we can easily find that UI element for writing E2E tests.
- Then next major pain point is to mock each and every API call used in the flow you are testing! This activity becomes more difficult in case same API returns different response based on request payload!
Flakiness:
- The E2E tests fail when ran multiple times (example: due to memory leak).
Debugging:
- Debugging E2E tests is usually hard where developer has no access to Logs, screenshots, videos, Test reports, node-inspector, etc.
Chapter 3: Generating E2E tests - Automatically!
For generating E2E tests, I went with Playwright as it has a codegen plugin which generates the E2E tests for you:
Advantages of using Playwright:
- Great community support
- Amazing debugging capabilities (Playwright Inspector)
- Easy to mock API's (client side mocking)
- No Flakiness (atleast none faced yet)
- Cross-browser (web and mobile) and cross-platform support
Setting up Playwright for your project is very fast. See installation doc here.
Install using yarn
:
yarn create playwright
Once installed, run:
yarn playwright codegen
Which will open a Chromium browser and an inspector window. Now all you gave to do is to open your UI in localhost and go through your flow. In the inspector window you will see Playwright tests getting generated automatically.
The code generated is like:
import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
await page.goto('https://www.google.com/');
await page.getByRole('img', { name: 'Google' }).click();
await page.getByRole('combobox', { name: 'Search' }).click();
await page.getByRole('combobox', { name: 'Search' }).fill('Twiiter');
await page.getByRole('option', { name: 'Twitter' }).getByText('Twitter').click();
await expect(page).toHaveURL('https://www.google.com/search?q=twitter');
});
Now all you have to do is to copy the code and paste in your codebase!
Chapter 4: Mocking API's - Automatically!
Till now you have generated the E2E tests, but these tests are of no use without mocking the API's. For mocking the API's there are many options out there:
Mocking at client side:
- Using MSW (Mock Service Worker)
- Using Playwright's client side mocking - this records any http request and saves as a HAR file (HTTP Archive file). Use this HAR file next time you run tests.
Mocking at server side:
- MSW (Mock Service Worker) for node
- Record and Replay API calls and re-use it to run tests next time.
- Based on NODE_ENV = "test", mock API response manually.
My use-case was to test BFF (Backend For Frontend) server as well but with the aim of spending minimal time in mocking the API's. This is where Polly.JS came into the picture.
Why PollyJS?
- It works as expected: record -> save -> replay
- Easy to configure if we want to record all API's or only record those API's which are not recorded yet.
- Easy to configure as to what parameters to use in order to replay an API (like match request params but do not match headers, etc.)
- Set the expiry after which your recording should expire.
Some Polly.JS Terminology:
- Record: Save the request, reponse, headers, etc of an API call.
- Replay: Mock the API calls from a saved recording.
- Persister: Polly offers many ways in which you can save the recordings. For our approach, we will use "File System" persister which will store recording as a HAR (HTTP Archive) File which you can commit to your codebase.
- Adaptors: Adapters provide functionality that allows Polly to intercept requests via different sources (e.g. XHR, fetch, Puppeteer). For our approach, we will use "Node HTTP" adaptor which will intercept http calls from our nodejs server.
Let's dive into code:
- Install Polly:
yarn add @pollyjs/core
yarn add @pollyjs/adapter-node-http
yarn add @pollyjs/persister-fs
- Run Polly in your node server:
import { Polly, PollyConfig } from '@pollyjs/core';
import FSPersister from '@pollyjs/persister-fs';
import NodeHttpAdapter from '@pollyjs/adapter-node-http';
export default function autoSetupPolly(flowName: string) {
Polly.register(NodeHttpAdapter);
Polly.register(FSPersister);
const recordIfMissing = true;
const mode: PollyConfig['mode'] = 'replay';
console.log('[PollyJS] Starting flow: ', flowName);
const polly = new Polly(flowName || 'default');
polly.configure({
adapters: ['node-http'],
mode,
// expiresIn: '30d5h10m', // expires in 30 days, 5 hours, and 10 minutes
recordIfMissing,
flushRequestsOnStop: true,
logLevel: 'INFO',
recordFailedRequests: true,
persister: 'fs',
persisterOptions: {
keepUnusedRequests: true,
disableSortingHarEntries: false,
fs: {
recordingsDir: '__recordings__',
},
},
matchRequestsBy: {
method: true,
headers: false,
body: true,
order: false,
},
});
return polly;
}
The above configuration of Polly records any new API call and replays already saved API call. The mocks will be saved in recordings folder.
Now execute the above function in your TEST env:
if(__TEST__) {
let _polly = autoSetupPolly("Google_Home_Page_Flow");
}
Note: After you have done recording, the API's will be saved in recordings folder only when you execute:
" _polly.stop()"
in your node server!
Chapter 5: Stitching Playwright and PollyJS:
Now we have Playwright which generates tests for us and now we also have Polly that will record and save API calls in our node server for us. Let's see how they work together!
Now as a good practice we want our E2E tests to be small and concise, also we want all the Polly recordings for different flows to be stored separately. Storing recordings separately will help us to easily change/override a recording if one of the E2E test changes due to change in any UI flow.
Steps:
- In order to store recordings separately, let's expose 2 simple API's in our node server (expose only in
test
env):
if (__TEST__) {
let _polly = null;
/* Polly Stop API */
app.use('/polly/stop', (_, res) => {
_polly.stop();
res.status(200).json({ status: 'Polly Stopped in Server!' });
});
/* Polly Start API */
app.use('/polly/start/:flow_name', (req, res) => {
const promised = !!_polly ? _polly?.stop() : Promise.resolve();
promised
?.then(() => {
_polly = autoSetupPolly(req?.params?.flow_name);
res.status(200).json({
status: `Polly Started in Server for: <<< ${req?.params?.flow_name} >>>!`,
});
})
?.catch(() => {
res.status(200).json({
status: `Polly Errored in Server for: <<< ${req?.params?.flow_name} >>>!`,
});
});
});
}
These 2 API's are to start Polly and stop Polly. In start Polly API, we pass the flow_name
which is the name of the file this recording will save as (example: Google_Home_Page_Flow
). The stop Polly API, will stop Polly instance and save all recordings in HAR file with the name: flow_name
- After having these 2 API's, now while generating tests via Playwright codegen:
- Before starting the Test Flow, fire Polly Start API call in our node server:
This will start Polly instance with the
flow_name
and will start recording our API's. - Go through the flow and generate Playwright E2E tests.
- After the flow is done, Fire Polly Stop API call in the node server:
- Before starting the Test Flow, fire Polly Start API call in our node server:
This will start Polly instance with the
The Playwright code generated will be like:
import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
/* Start POLLY */
await page.goto('http://localhost:3001/polly/start/Google_Home_Page_Flow');
await page.getByText(/\{"status":"Polly\s+Started\s+in\s+Server\s+for:\s+<<<\s+Google_Home_Page_Flow\s+\>\>\>!"\}/).click();
/* Your application testing code */
...
...
/* Stop POLLY */
await page.goto('http://localhost:3001/polly/stop');
await page.getByText(/\{"status":"Polly\s+Stopped\s+in\s+Server!"\}/).click();
});
That's it folks, now disconnect your node server to actual backend and run the generated test code using
yarn playwright test
. It should just run!
Chapter 6: Running the tests via Github Action!
During Playwright installation, you will see the playwright.yml
generated automatically. You just need to tell playwright how to run your local server in playwright.config.ts
:
/* Run your local dev server before starting the tests */
webServer: {
command: 'npm run start',
port: 3000,
},
However, if your server bootstrap time is more (like it generates some stubs, etc before starting server in dev) the above approach might give some issue where tests start running before the server starts.
The workaround is to run your dev server as part of github job config file in background and sleep for some seconds before running your tests.
Example playwright.yml
:
name: Playwright Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
react-app:
timeout-minutes: 20
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 16
- name: Install dependencies
run: yarn
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run React and node server
run: |
yarn dev:test --filter=react-app --filter=node-server &
sleep 80
yarn playwright test
env:
CI: true
- uses: actions/upload-artifact@v3
if: always() # https://github.com/actions/upload-artifact#conditional-artifact-upload
with:
name: playwright-web-report
path: apps/react-app/playwright-web-report/
retention-days: 5
The last step in Playwright Github Job config file, uploads the HTML reporter report to Github so that we can see where test failed with screenshot!
Chapter 7: The Conclusion
This setup will allow you to quickly write E2E tests and run those tests as part of each PR. Using this you can quickly change your E2E tests as well in case UI flow changes or API changes. The main selling point is that this setup significantly reduces the time taken to write tests. Moreover, even a non-engineer can be taught to generate these tests.
Thanks for reading !!
Alright! You have reached the end of this article. If you liked this article, kindly give likes and comment the best part you liked about this article.