7 API conformance: schema testing
This chapter covers
- Writing code-based schema tests
- Auto-generating API schema tests
- Using a validation proxy for schema tests
In the last chapter, I looked at how generating the OpenAPI definition (and the associated API reference) from the API service helps ensure that the OpenAPI definition conforms to the behavior of the service. But it may not be possible to generate the OpenAPI definition due to technical limitations in legacy API services. And even if you can generate the OpenAPI definition from the API service, the behavior of the service under exploratory testing may reveal surprising behavior from the application, which may not be captured by the OpenAPI definition. For example, a bad request with illegal characters can cause the backend application server to return an error before the request is processed by your controller's error-handling logic. And so, an OpenAPI definition generated based on the controller code may not cover this case.
So, it is still valuable to have an additional layer of verification to check that the published API reference matches the API behavior. One way to do this is with schema testing using the OpenAPI definition as the definition of the API schema. In this chapter, I discuss three techniques for writing OpenAPI schema tests. These are code-based schema tests, generated schema tests, and using a validation proxy.