Calculating the code coverage of integration tests
Door Stefan Jansen / sep 2012 / 1 Min
Door Niek Knuiman / / 3 min
Versioning. It's a word feared by many. When working on a project, versioning often doesn't get the attention it deserves until it's too late. It has also become increasingly harder now that microservices are popping up like bugs in a PHP script ...
When creating a microservice architecture, all interaction between the services is documented, discussed or - if you're lucky - both. But as time goes on, programmers lose sight of the complete landscape or the documented knowledge becomes outdated. On top of this, every newly added microservice will create a multitude of new interaction points that need to be managed and documented.
We can use integration tests to check whether a newly changed service is ready to be deployed. It can take hours to run those tests and if they fail, parts of the system need to be changed. As these integration tests often test a chain of services, it can be difficult to figure out which services are responsible for the failed test. So what can we do instead?
In this blog, I propose a possible solution: contract testing. And to be more specific, consumer-driven contract testing. Fancy words, right? The definition is actually quite simple. It means that the service that is requesting resources (the consumer) creates a contract with its demands for the supplying service (the provider).
With this type of testing, providers can test if the changes made match the demands in the contract set up by the consumer. This action removes the need for a real or mock service to be built. Instead, the test can just check if the changes are in accordance with the contract.
.png?width=884&name=Pact%20demand%20flow%20(1).png)
What if the consumer deploys an incompatible version?
This can easily be avoided. With every new build, the consumer uploads a new contract. This new version will need to go past the provider first. Nothing should be deployed until it's checked. Following these steps will come naturally with a great tool. I'll get back to this in a bit!
Does this mean that providers have to implement every consumer demand whenever they make a change?
Contracts are by no means an excuse for not communicating with the developers of different services. In fact, working with contracts demands for more communication. If a consumer wants a new feature, this should be discussed with the provider before uploading a contract.
The consumer will create a contract only after reaching an agreement with the provider. The provider will use this contract to test their assumptions against those of the consumer. If it turns out that a provider cannot meet certain demands, or if the consumer requests something unexpected, then contracts should be reevaluated in collaboration.
Okay cool, but what does it look like in practice?
This is where I'll introduce Pact, an open-source contract testing tool! We sure live in a wonderful age of open-source & collaboration.
Pact provides us with all the tools necessary to generate, distribute and validate contracts within a system. It supports most common languages and the use of multiple languages within one architecture. Pact takes the consumer-driven contract scenario and automates most of the process.
The flow starts off with the consumer definition of the unit tests: what response does the consumer expect in what state? It's important to note that the consumer should test the interaction with and not the functionality of the provider. If the consumer can process a response but the test fails, you're likely testing the functionality of the provider. This should be tested with provider unit tests.
For example, take a validation service that returns true or false based on input. A consumer could test if the validation service returns false when the input exceeds a certain number of characters. But the validation service is responsible for that character limit. It should be checked in its unit tests and not in the contract.
With every change a consumer makes, a contract will be created and sent to the Pact broker. When the provider makes a change, all consumer contracts will be retrieved from the broker. A mock server will playback the requests according to the contracts. If the responses are in accordance with the contract, the provider can deploy the changes.
However, the Pact broker does more than just storing and sending contracts. After the compatibility with a contract is verified, the results are sent back to the broker. The screenshot below shows the useful insights that this provides.
This information can also be displayed as a version matrix between two services. As you can see, Pact automatically generates documentation based on the contracts!
 |
 |
That looks great, but we already have an environment setup that uses integration tests. We don't have the resources to switch to contract testing!
Just take it one step at a time. The contract tests can run perfectly alongside your existing test suite. Maybe use it as an excuse to increase your test coverage or start using contract testing for new services. With microservices, you add a lot of communication management. Consumer-driven testing helps to make good contracts while also holding developers to the terms in that contract.
Resources
Pact.io, for documentation, examples & features that were not covered here (like the broker matrix view!)
Pact broker, get your Pact broker container or request a free hosted broker here!
| Testing
Door Niek Knuiman / okt 2024