Containerization and microservices bring new challenges to software testers, and especially to integration testers. Software systems are getting more and more complex with an overload of interdependencies between microservices and containers. In addition, each instance of a service can have its own independent lifecycle and versions, which adds to the difficulty. This new application paradigm can’t be tested effectively by only using integration testing.
Are there solutions for tackling these challenges? Yes, and one of them is consumer-driven contract testing.
What is contract testing?
Contract testing is a way to ensure that services (an API provider, a client etc.) can communicate with each other.
The contract is between a consumer (for example, a client who wants to receive some data from an app) and a provider (for example, an API on a server that provides the data the client needs). In a microservice architecture, the terms client and server are not always suitable to encapsulate the connectivity between services. Communication methods are different, and the hierarchy of roles is not as clear. For this reason, we stick to consumer and provider in this article.
What is consumer-driven contract testing?
With contract testing, we want to ensure that a service works between a consumer (same as the client) and an API provider (the provider).
What happens when the provider changes a part of the service? You will need to know if the consumer can still use the service and whether the changes still meet the demands of the consumer’s contract. This is when you need to perform consumer-driven contract testing.
Since these contracts are written and managed exclusively by consumers, providers must first obtain permission from consumers before they can make any changes. If providers need to implement new features, they can see which consumers would be affected by the change(s) by looking at the status of the consumer’s contract tests.
Are there contract testing tools?
Contract testing is relatively new, but luckily there are several tools available, such as API Blueprint, RAML, OpenAPI, Pact or Postman Collections. These tools help to build specifications for APIs versus publishing current API behavior in a static way with documentation. Of the available tools, Pact and Postman are the real specialists, but whereas Pact focuses exclusively on contract testing, Postman has a large community which can make their platform more appealing.
Postman for consumer-driven contract testing
When implementing contract testing in your organization, Postman is the place to start.
It is easy to use alongside your other test automation tools such as the Sauce Labs software suite.
How does it work?
Postman Collections are executable specifications of an API. There are several different ways of running these Collections:
on your local machine using the Postman app,
in the command line,
and on CI systems with the help of newman or on the cloud using Monitors.
You can also add dynamic behavior to your requests with pre-request scripts and execute response assertions in your tests. You can find detailed instructions for documenting this behavior in Postman in Kaustav Das Modak’s article on transitioning from manual to automated API testing. These steps are all combined in the Workspaces in Postman. Now, you can share your contract collections with your team automatically rather than manually.
A consumer-driven contract test with Postman
As a software testing and data science enthusiast, I like logs – especially error logs.
I will show you how to develop a service that returns a list of log entries from a central log store. Let’s say that we want to have one endpoint that returns the last three log entries, including the name of the service that created the entry, the timestamp of the entry, and a string with the description of the entry.
You can find the endpoint at /api/v1/logs.
When you send a GET request to this endpoint, it should return JSON data in the following format:
{
"count": Number,
"entries": Array[Object]
}
The Object represents a log entry and is structured in the following format :
{
"serviceName": String,
"timestamp": Number,
"description": String
}
Develop the blueprint
As with every Postman API test, we need to create a collection to visualize the API structure. The details of the collection come from the provider of the service being visualized.
This collection is called the blueprint.
Figure 1
Figure 1 shows the blueprint belonging to the above example.
As we have seen, it has one endpoint and one GET-request.
We expect a JSON-response with the latest three log entries (the objects) from this GET-request. Figure 2 shows an example of this:
Figure 2
You can generate your API documentation automatically with Postman. See this tutorial to learn how to publish your document. Figure 3 shows this blueprint:
Figure 3
This blueprint can be shared with all members of your workspace so that everybody can get the latest version.
We still have to develop a mock server to mimic the behavior of the logging service in production. This tutorial shows you how to create a mock server, and Figure 4 shows the results of our test case:
Figure 4
Creating contract collections
The contract collections can be made from the blueprint and the mock server.
Postman has a lot of other interesting features too. For example, you can make sure that you get the correct response headers, body, and time based on the assertions that you created in the test.
This is illustrated in Figure 5, where I placed an assertion in the body to verify whether the response is in the correct JSON format.
{
"count": 3,
"entries": [
{
"serviceName": "foo",
"timestamp": 1540206226229,
"description": "Received foo request from user 100"
},
{
"serviceName": "bar",
"timestamp": 1540206226121,
"description": "Sent email to user 99"
},
{
"serviceName": "foo",
"timestamp": 154020622502,
"description": "Received foo request from user 10"
}
]
}
# test if response data is JSON
var jsonData = pm.response.json();
Figure 5
With the help of Postman, you will have an excellent basis for implementing your consumer-driven contract testing.
Conclusion
Consumer-driven contract testing is a way to tackle the challenges of testing microservices. It is a method to ensure that the service between the consumer and the API provider functions correctly. With consumer-driven contract testing, the consumer is in control. While there are several tools available for automating consumer-driven contract testing, Postman is an excellent option because it is both intuitive and versatile.
