Go to the profile of jsilvax
jsilvax
Web Developer @NBCUniversal. #000 Coffee Addict.

Getting Started with Jest and Puppeteer

Jest is a testing framework maintained by Facebook that works great with Puppeteer, a library for controlling Headless Chrome. In this post we will go over how to:

  • Setup Jest, Puppeteer and Babel
  • Use Puppeteer’s API to emulate a mobile device, use selectors, scroll a page, listen to requests, and return response data
  • Write tests that are asynchronous and make assertions in Jest

Puppeteer is still under development so, stay up to date with their API, as it is subject to change.

If you’d like to check out the code for this post, go here

Update: This post has been updated to reflect the suggestions made by @xfumihiro to use the new globalSetup and globalTeardown API in Jest v22.0.0. This allows for a single Puppeteer Instance to be reused throughout all test suites.

Setting Up The Project

This post assumes you’re using Node v7.6.0 or greater

Let’s get started with some dependencies:

yarn add --dev puppeteer

Puppeteer will download a recent version of Chrome that’s meant to work with the puppeteer API.

yarn add --dev jest jest-environment-node

jest-environment-node will allow us to extend the Node Environment to use a custom setup hook that enable us to reuse the Puppeteer browser instance throughout our tests. This method is more convenient than using setupFiles which run before each test.

Then, rimraf to delete a directory we will be creating:

yarn add --dev rimraf

Finally, we need Babel to transpile our code based on our environment

yarn add --dev babel-core babel-preset-env

Setting up Jest

Jest, by default, will set the testEnvironment to JSDOM. We can configure this to be a custom Node Environment that will work alongside globalSetup and globalTeardown. (globalSetup executes before all test suites and globalTeardown executes after all test suites)

{
  "scripts": {
    "test": "jest"
  },
  "jest": {
    "globalSetup": "./setUpPuppeteer.js",
    "globalTeardown": "./tearDownPuppeteer.js", 
    "testEnvironment": "./CustomNodeEnvironment.js"
  }
}

*Alternatively, you can configure this value in a jest.config.js

In setUpPuppeteer.js we launch Chromium in headless mode. We then assign the browser instance to the global scope so that in tearDownPuppeteer.js we can close the browser / Chromium process. In order to share the Chromium instance in CustomNodeEnvironment.js, we file the browser’s web socket endpoint so that the sandboxed testEnvironment can access it. That will allow us to reuse that single Puppeteer Instance throughout all test suites.

// setUpPuppeteer.js
const puppeteer = require('puppeteer');
const fs = require('fs');
const mkdirp = require('mkdirp');
const os = require('os');
const path = require('path');
const DIR = path.join(os.tmpdir(), 'jest_puppeteer_global_setup');
module.exports = async function() {
    const browser = await puppeteer.launch({ headless: true });
    global.__BROWSER__ = browser;
    mkdirp.sync(DIR);
    fs.writeFileSync(
        path.join(DIR, 'wsEndpoint'),     
        browser.wsEndpoint()
    );
};

If you want to see what puppeteer is doing, which is useful for debugging, you can set headless to true

In CustomNodeEnvironment.js the following is needed:

const puppeteer = require('puppeteer');
const NodeEnvironment = require('jest-environment-node');
const fs = require('fs');
const os = require('os');
const path = require('path');
const DIR = path.join(os.tmpdir(), 'jest_puppeteer_global_setup');
const wsEndpointDir = path.join(DIR, 'wsEndpoint');
class CustomNodeEnvironment extends NodeEnvironment {
    constructor(config) {
        super(config);
    }
    async setup() {
        await super.setup();
        const wsEndpoint = fs.readFileSync(wsEndpointDir, 'utf8');
        if (!wsEndpoint) throw new Error('wsEndpoint not found');
        this.global.browser = await puppeteer.connect({    
            browserWSEndpoint: wsEndpoint
        });
    }
}
module.exports = CustomNodeEnvironment;

In setup, we connect Puppeteer to the existing Chromium instance we launched in setUpPuppeteer.js. This is done using puppeteer.connect() which passes the wsEndpoint. We then assign the browser instance to the test suite’s global scope so our tests have access to that instance.

Finally, tearDownPuppeteer.js is responsible for closing the browser / Chromium process and cleaning up the temp directory made for wsEndpoint.

// tearDownPuppeteer.js
const puppeteer = require('puppeteer');
const os = require('os');
const path = require('path');
const rimraf = require('rimraf');
const DIR = path.join(os.tmpdir(), 'jest_puppeteer_global_setup');
module.exports = async function() {
    await global.__BROWSER__.close();
    rimraf.sync(DIR);
};

Since these are all configuration files, they don’t get transpiled by Babel, and we must use CommonJS.

Setting up Babel

Next, we need to configure babel with a .babelrc in the root of our project

The .babelrc will use babel-preset-env which include polyfills and transforms that are needed for the targeted node version. For convenience, we will use “current” which is equivalent to process.versions.node

{
 "presets": [
   ["env", {
    "targets" : {
     "node" : "current"
    }
   }]
 ]
}

Writing tests

Now that our Puppeteer Browser instance gets set up at the beginning of our test run, we can just worry about writing specific tests and no longer about setting up and tearing down for each file we are testing.

Let’s ensure a widget lazy loads and inserts more list items when a user scrolls beyond a certain point.

Start off by writing a beforeAll block that passes an async function. This allows us to use await inside the block. We then open up a new page by using puppeteer’s browser.newPage API and cache that to a local variable.

let page;
beforeAll(async () => {
    page = await browser.newPage();
});

We also want to close the page after all tests run, this is done using an afterAll block.

afterAll(async () => {
    await page.close();
});

Now we can begin writing a test block. The second argument increases Jest’s default timeout of 5000ms to 10000ms. This will solve the frequent Timeout — Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout errors that can occur when puppeteer takes too long to complete its procedures.

test('should lazy load new list items', async () => {
}, 10000);

Inside of the async function we want to define some constants. The first is the selector name for the widget

const selector = '[data-hook="dynamic-list"]';

Then, once we lazy load the list elements, this will be the selector name for the first dynamically inserted element (21st element)

const firstDynamicEl = `${selector} > div:nth-child(21)`;

We also need the request URL that we will be listening for

const requestUrl  = "http://m.eonline.com/us/category/dynamicList/lady_gaga/json?page=2&pageSize=20";

To emulate an iPhone, we use puppeteer’s emulate API method with viewport and userAgent options passed.

// Emulate an iPhone
const userAgent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25';
await page.emulate({ 
    viewport: { 
       width: 375,
       height: 667,
       isMobile: true 
    }, 
    userAgent 
});

*Only do this if you want to emulate a device, otherwise it’s unecessary

To navigate to a URL, we use page.goto and wait until the DOMContentLoaded event is fired.

await page.goto('http://m.eonline.com/news/lady_gaga', { 
    waitUntil: 'domcontentloaded' 
});

By default, waitUntil is set to “load” but in our case we don’t need to wait for a fully-loaded page.

The selector [data-hook="dynamic-list"] is the parent element that contains the list items. Puppeteer lets us have access to the NodeElement using page.$eval, which runs document.querySelector and takes a function that passes the element as its first argument. Using this lets us return the children’s length.

const numOfItems = await page.$eval(selector, el => el.children.length);

If you want to select multiple elements, you’d use page.$$eval, which runs document.querySelectorAll

Now that we have the initial number of elements, we need to trigger the lazy loading by scrolling down the page.

await page.evaluate(() => window.scrollTo(0, document.querySelector('footer').offsetTop));

page.evaluate takes a function that runs in the browser context. This lets us use window.scrollTo, to scroll to the footer’s offsetTop. This will kick off the lazy loading request.

We want to be able to get the response data from our request, so we need to create a helper function that will resolve a promise when our request is successful.

const waitForResponse = (page, url) => {
 return new Promise(async (resolve, reject) => {
  page.on('response', async function responseDataHandler(response){
   if(response.url === url){
    if(response.status !== 200) {
     reject(`Error Status: ${response.status}`);
    }
    let data = await response.text();
    data = JSON.parse(data);
    page.removeListener('response', responseDataHandler)
    resolve({ data });
   }
  });
 });
}

The function waitForResponse takes in the page and the request url. Since page.on uses a callback to execute every time a new response comes in, we need to use a Promise to resolve when the response we’re waiting for comes in and passes along the data. Once we get the data and parse it, we remove the event listener by using page.removeListener and passing in the event name and callback handler used.

The nice thing about async/await is you can mix it in with Promises when you need to.

Once back in our test, we just use await to get the data

let { data } = await waitForResponse(page, requestUrl);

We then have to wait for the 21st list element to be dynamically inserted using page.waitForSelector.

await page.waitForSelector(firstDynamicEl);

Now we are able to grab the new number of list items by once again using page.$eval

const numWithLazyLoadedItems = await page.$eval(selector, el => el.children.length);

and we can grab the href of the dynamically inserted list element’s link

const href = await page.$eval(`${firstDynamicEl} a`, el => el.href);

Finally, we want to begin making assertions. In this case, we want to verify that only 2 assertions are called during the test

expect.assertions(2);

This isn’t necessary, but recommended when testing async code. It allows you to know that the number of assertions expected really got called.

Next, we assert that the number of lazy loaded list items is greater than the initial number of items

expect(numWithLazyLoadedItems).toBeGreaterThan(numOfItems);

and that the first dynamically inserted list element’s <a> link matches the response data

expect(href).toContain(data[0].uri);

Final Thoughts

Puppeteer makes testing UI a pleasant experience with its robust API and latest browser and JS features. Together with Jest, it can be a great addition to any developer’s toolset.

This story is published in Noteworthy, where 10,000+ readers come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.