My FeedDiscussionsHeadless CMS
New
Sign in
Log inSign up
Learn more about Hashnode Headless CMSHashnode Headless CMS
Collaborate seamlessly with Hashnode Headless CMS for Enterprise.
Upgrade ✨Learn more

React Code Coverage without ejecting from CRA

Aidan Reel's photo
Aidan Reel
·Jul 3, 2019·

13 min read

This story outlines the scaffolding needed to produce code coverage reports on a TypeScript based React application created using create-react-application (cra) without having to eject your application from cra.

The testing framework used is Cypress with Istanbul and nyc used to provide code coverage.

The complication is that cra does not expose access to webpack or babel configuration files (webpack.config.js or .babelrc) BUT istanbul/nyc require that access.

Create React App Configuration Override (craco) is used to avoid ejecting from cra by providing a configuration layer.

A repo containing the scaffolding is available here. It has separate commits for each major step.

Readership: The reader is assumed to be familiar with the technologies mentioned (create-react-app, TypeScript, Cypress, Istanbul) perhaps less so with react-scripts, craco. The focus is on getting these working together rather than explaining their individual merits.

Create and build react application via CRA

I tend to use yarn so the following commands are all yarn based. To see larger versions of screenshots, open the image in a separate tab/window.

We are going to use the demo code created by cra with a very minor change (made later) to add a function to display the strap line text, this just confirms code coverage is being executed.

yarn create react-app cra-typescript-cypress-istanbul-craco --template typescript
cd cra-typescript-cypress-istanbul-craco
yarn start

We now have a running typescript react application, a very smooth experience.

01-BasicAppRunning-fs8.png

Introduce Cypress testing framework

Install Cypress

yarn add -D cypress

Add testing script to package.json

...
"scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "cypress:open": "cypress open"
  }
...

At this point cypress is available and able to execute JavaScript based tests over our TypeScript code base, see commit B in repo. We want our tests to be written in TypeScript too, so we need to configure Cypress in order to accept and execute tests written in TypeScript.

Configure Cypress to execute TypeScript tests

Currently the easiest way to achieve this is to execute

yarn add -D @bahmutov/add-typescript-to-cypress ts-loader

This will modify some existing cypress configuration files. You can edit the newly added cypress/plugins/cy-ts-preprocessor.js by including "tsx" and "jsx" extensions.

...
const webpackOptions = {
  resolve: {
    extensions: [".ts", ".tsx", ".js", ".jsx"]
  },
...

Modify cypress/tsconfig.json to be

{
  "compilerOptions": {
    "noEmit": false,
    "isolatedModules": false
  },
  "extends": "../tsconfig.json",
  "include": ["../node_modules/cypress", "*/*.ts"]
}

While doing this investigation I have noticed that the isolatedModules: false is sometimes not required but have been unable to locate the reason for the variation in behaviour. Given this is configuring just the Cypress Test Files I am comfortable leaving this override in place (for now).

At this point we have our TypeScript based React application being tested by Cypress using TypeScript Tests. see commit C in repo.

Code Coverage

Now we have our tests running we would like to be able to identify areas of code that are either not being tested or not being sufficiently tested. In order to support this we need to add an instrumentation package that will track which lines of code has been executed that is Istanbul and a package that is capable of producing reports based on the instrumentation data, that is provided by nyc.

Install our code coverage related packages.

yarn add -D nyc istanbul-lib-coverage

Now we need to configure our scaffolding to serve instrumented code during development.

Configuring Code Coverage

This proved to be the most time consuming part. The difficulty is due to CRA projects not exposing webpack or babel for configuration. But some tooling such as Istanbul / nyc require access to webpack. So what are our choices, from my investigations I came across the following :

  • eject from cra (Not an option as we want to stay within cra scaffolding).
  • customise cra's react-scripts
  • customise cra using a configuration override package

Customise React-Scripts

Chose not to pursue this option, from initial investigation I felt the learning curve would be steeper, would result in having multiple scripts to manage (already using --typescript), what if we needed to support another package would mean another script etc.

customise cra using a configuration override package

Appears to be a long standing approach to this issue with offerings like react-app-rewired being around since 2016. From that repo I looked at rescripts , react-scripts-rewired and craco .

I chose craco because I felt I would be able to get something working without as much in-depth understanding. Not sure I was correct as I did find the documentation somewhat terse. I intend return to rescripts when I get the time.

Installing craco

yarn add @craco/craco

edit package.json to use craco rather than react-scripts

 "scripts": {
    "start": "craco start",
    "debug": "craco --inspect-brk start",
    "build": "craco build",
    "test": "craco test",
    "cypress:open": "cypress open"
  },

I found the debug command very useful to allow me to inspect various objects and to confirm what was going on. To use the visual code debugger create the launch.json via the debugger and edit it to mirror the following :

    "configurations": [
        {
          "name": "Attach To npm",
          "type": "node",
          "request": "attach",
          "port": 9229,
          "address": "localhost",
          "restart": false,
          "sourceMaps": false,
          "localRoot": "${workspaceRoot}",
          "remoteRoot": "${workspaceRoot}"
        }
      ]

Configuring craco

The reason we installed craco is to allow us to add some istanbul related configuration to the unavailable webpack configuration. The technique I eventually got working was to add a craco.config.js file in the project root and add the following to it:

const {
  when,
  whenDev,
  whenProd,
  whenCI,
  whenTest,
  ESLINT_MODES,
  POSTCSS_MODES
} = require("@craco/craco");


// todo
module.exports = {
  reactScriptsVersion: "react-scripts" /* (default value) */,
  webpack: {
    alias: {},
    plugins: [],
    configure: {
      /* Any webpack configuration options: https://webpack.js.org/configuration */
    },
    configure: (webpackConfig, { env, paths }) => {
      webpackConfig.module.rules.push({
        test: /\.(ts|tsx)$/,
        exclude: /node_modules/,
        use: {
          loader: "babel-loader",
          options: {
            presets: ["@babel/preset-typescript"],
            plugins: ["istanbul"]
          }
        }
      });
      return webpackConfig;
    }
  }
};

This is basically injecting a rule to instrument TypeScript files with ts or tsx extensions using istanbul when loading those file via babel-loader. The @babel/preset-typescript and babel-loader packages are required so we need to explicitly add them too

yarn add -D @babel/preset-typescript babel-loader

The const at head of file is for later use not used in this demo.

if we run cypress tests or load the app in a browser and use the developer tools to inspect the source code we should see instrumented code in the file main.chunk.js generated by webpack. The cov_ldyy6m5t16 function and coverageData object in the following screenshot are instrumentation related

02-cypress_with_instrumented_code-fs8.png

see commit D in repo.

Capture instrumentation statistics and create code coverage report

While running our instrumented code via our typescript tests, we now need to collect the run time statistics and generate a human readable report from those statistics when the tests have completed. The Cypress team has created a package that does just that for us and we install that next

yarn add -D @cypress/code-coverage

This provides us with cypress tasks that can record the instrumentation statistics for each test run, collate them (in .nyc_output/out.json) and when the test run is completed generate a html based report from the collation into the coverage folder. This html report (coverage/index.html) can be viewed in a browser and allows drilling down into individual modules.

We need to import these tasks into our configuration and make them available to the testing engine.

Add the following to the cypress/support/index.js file

import '@cypress/code-coverage/support'

Register with the engine by adding the following to the cypress/plugins/index.js file

module.exports = (on, config) => {
  on('task', require('@cypress/code-coverage/task'))
}

Version 14 of nyc requires the following to be added to your package.json* this will be fixed in a later major release.

  "nyc": {
    "extension": [
      ".ts",
      ".tsx",
      ".js",
      ".jsx"
    ]
  }

run

yarn start

I suffered from inconsistent behaviour during this investigation where dependency failures might or might not occur, the best I can offer is if you get an error similar to

The react-scripts package provided by Create React App requires a dependency:

  "babel-loader": "8.0.5"

you can remove the babel-loader entry from your dev dependencies (sometimes I have seen this dependency disappear), remove node_modules folder and yarn.lock file and issue

yarn

to reinstall dependencies, that has, so far, always worked for me.

Running the tests

see commit E in repo.

We should now be in a position to run our TypeScript tests and on completion open the index.html file in the coverage directory. The Cypress runner should show some addition tasks being run during test execution. These are the various instrumentation collation and report publication tasks being performed.

03-cypress_with_code_coverage_tasks_colored-fs8.png

In our editor / file manager we should see a .nyc_output and coverage folder containing our instrumentation detail.

04-VScode_with_coverage-fs8.png

Finally, what all this was leading up to, opening that index.html file should reveal our code coverage in a human readable and hopefully revealing way.

05-code_coverage-fs8.png

and we can drill down into various modules

06-Code_coverage-fs8.png

Conclusion

The motivation for writing all of this was firstly to record it for my sake, in case, in a couple of weeks (or days) I need to modify or debug something, this is too much to expect to remember in detail. Second, given the popularity of the tooling involved someone else, hopefully, will avoid the hours (actually days) that it took me to digest all of this and get a working environment configured.

Useful links

I found the following links of Gleb Bahmutov (VP of Engineering at @cypress-io) particularly useful

  • his blog site
  • code coverage issue his comment on 6 May is full of very useful links explaining instrumentation, the integration with Cypress, adding instrumentation to tests etc.