Escaping Regressions With Chai OpenAPI Response Validator

Wrong Perception of Sufficient Documentation

The backend behavior was different the frontend expected. Response from endpoint has changed.

Like many growing teams, we were shipping new features bi/biweekly, but paid the price in regressions, leading to steps back. A change of endpoint’s behavior worked in isolation, only to break an existing flow during integration. Frontend developers only had a Postman request collection in their hands  -  it wasn’t the right pillar for cross-team cooperation:

project.postman_collection.json
dev.postman_environment.json

The root cause: absence of a single source of truth and no automated validation between backend changes and consumers.

New Rule That Saved Many Hours

We began mandating OpenAPI specifications for all new backend services and enforcing them through automated integration API tests.

At first, we needed to start openapi.yaml(or JSON). Whether autogenerated or written manually, the primary goal was to achieve a 100% accurate representation of API behavior. Unreliable tools for automatic generation from code and tons of potential annotations in code didn’t amaze us, so step-by-step manual creation was our choice. IDE plugins and https://editor.swagger.io/ helped with visualization/rendering on the go.

Once ready, npm install chai packages and setup:

import chai from 'chai';
import chaiResponseValidator from 'chai-openapi-response-validator';

const {expect} = chai;
chai.use(chaiResponseValidator('/docs/openapi.yaml'));

Note: There is also a more independent, newer, non-chai alternative openapi-response-validator .

Then the one line with satisfyApiSpec that does the hard work was placed after every axios call in our native Node.js Test Runner suite.

it('Anonymous user can log in with email and password', async () => {
  const response = await axios.post(url, data);
  expect(response).to.satisfyApiSpec;

  // Some additional assertions handling endpoint specifics
  assert.strictEqual(.....);

For local development, it url is the developer machine’s localhost. For the remote deployment pipeline, it’s the dev environment.

This validates whether the response body’s structure matches the one defined in OpenAPI (it supports an axios response from which it takes the path and matches it to the OpenAPI specs), even taking into account multiple variants defined with oneOf. It’s crucial to have required parameters correctly defined; also additionalProperties is important to define the strictness of the schema.

A perfect sentence can be found on the chai plugin’s webpage https://www.chaijs.com/plugins/chai-openapi-response-validator/.

If your server’s behavior doesn’t match your API documentation, then you need to correct either your server or your documentation, or both.

Define npm so tests can be run quickly anytime during the development:

"scripts": {
    "test": "node --test test/integration/**/*.test.js",

Why It Works So Well

1. Fail Fast: Backend developers validate the solution locally against localhost; most malfunctions are resolved before the code reaches frontend developers in the test environment.
2. Single Source of Truth: OpenAPI is both documentation and part of the test suite.
3. Enforces Discipline, Saves Time in the Long Run: Writing spec-first takes ~10% more time upfront but saves 10 times in debugging. Even utilizing manual creation isn’t that effort-heavy. After a slower beginning, the first endpoints serve as copy-pastable boilerplates for new endpoints.
4. Return on Investment: New team members can read OpenAPI YAML, eliminating the need for endless onboarding sessions.

Challenges Exists

In the JavaScript world, it’s easy to fall into a more open style of implementation. Sometimes, the backend handles various scenarios under a single endpoint, based on a combination of input parameters from the frontend, which may result in different responses. This can’t be reliably represented in OpenAPI, so we use description field for such exceptions, where one sentence can explain how and when something happens.

The Results Speak For Themselves

The backend bug rate dropped by 75% even in development and test environments. In production, we are almost regression-free, covering most scenarios of most endpoints. I never had more peace during deployments.

Frontends integrate faster, and less communication overhead is needed between frontend and backend teams.