Have you ever pondered how to write integration tests for third party API? The following post might show you a new approach. We will be using Consumer Driven Contract Testing (CDCT).
Let’s start with what a Third Party API is supposed to be. We regard it as API which is not under our control. We cannot change it ourselves and asking for a change might take weeks or months. Examples range from the Google Maps APIs to some Legacy CRM system internal to our company.
Until the last year I have experienced three ways of dealing with these APIs from a testing perspective.
.jsonfiles) into source control
The first one is by far the easiest. We make sure the application still works by manually checking if the application does what its supposed to do and whether the correct values from the APIs are used.
With the second approach we make sure the API is properly integrated without the need to manually checking it. The downside is that these APIs might be slow to respond, leading to slow tests and tests which will fail if our CI tool loses the network connection. One might also run into rate limiting issues when the API is called too often in a short amount of time. We might even incur a cost for every request since not every API is free of charge.
The third approach is the one I usually encounter. With it we are neither subject of additional costs per request, slow tests or rate limiting issues. The real API is only ever hit once during the creation of the initial
.json file. There are even tools which help us with the workflow such as Ruby VCR.
Still, one issue arises. We would not notice right away if there is a mismatch between the response in the
.json file and the actual response. Aside from sloppiness and not checking in the correct response, e.g. with missing structure, we would also not notice directly whether the API producer breaks the API by removing fields. The latter might be improbable when working with Google APIs but is certainly possible with company internal systems or the API of smaller vendors.
To mitigate the issues above a colleague introduced the idea of using Consumer Driven Contract Testing (CDCT) for third party API. Since we’re already using it for API we do have control over the tooling was already in place and the developers were versed in its usage.
CDCT is often called TDD for microservices and used as such. The API consumers (e.g. an app or another microservice) writes contracts on how they require the API to behave and thus which fields of a response is used.
One consumer might be interested in the age field of the user, another might only care about the address.
On the provider side (a service) there will be a test which makes sure the contract is never violated. Thus if the provider removes a field from a response, which is not mentioned in any contract, all tests will still pass. The field was save to be removed. No consumer was using or expecting it. Thus for the example above one would not be able to remove the age or the address. Though removing the phone-number would be possible.
Of course we cannot give these contracts to the Google Maps API or to our legacy CRM system. These providers presumably neither care nor have the tooling in place to support CDCT. Thus on first glance using CDCT here seems odd. What we can do is create another service which acts as a replacement for the actual service during testing. The service will hold the contracts defining the required fields from the actual API. We call these services
proxies. They never proxy requests but act as an intermediate between the API and the implementation during automated testing. The proxy project will have two jobs.
And our result
It takes 6 hours and 18 minutes to drive from Berlin to Stuttgart.
At first glance the only piece of interest to us is the value
22651. It is the amount of seconds it takes. Our consumers, e.g. an Android App, will decide how they want to format the value for the user.
We should make sure the
value field is included in the response. Of course the documentation says it’s included but we want to be extra careful. Using a Groovy DSL of Spring Cloud Contract we can define the following contract.
It only contains the parts of the response we care about and which request would create the expected response. As a result the following test, asserting on the relevant fields, will be generated automatically.
If we provide the specified parameters in the request (
when:), we expect the specified response (
The DSL in the assertions is JSON Assert, a fluent interface for JsonPath. A side-effect is how we have now documented in code which fields we use. Maybe later during development we find a cheaper or faster API filling our requirement to have these durations calculated. We would only have to compare if they offer the same functionality we already require.
The generated contract tests pass without us writing any implementation code. Google already did all the work for us. We’re done with the first step. Verifying the API works as expected.
Aside from the generated tests, which some might regard as useless in as “Why would you want to test API you dont own?”, we get some stubs. These stubs, which are mostly the response with some metadata as json file, can be used by our consumers to develop against. Its using WireMock, a HTTP mock service, which returns the response defined in the contract when hit with the request defined in the contract. After all it was one of our initial goals to not permanentely call the real API and to have stubs which respond as the real API does.
These stubs are made available to other projects via the local
.m2 repository, a git repository or from our nexus or artifactory instance. An example using the local
.m2 repository can be found in the sources at the end.
In our consumer project we will need a
Client to make the appropriate HTTP calls to get our desired duration. Using the stubs we can test drive its functionality.
Pretty straight forward. With the actual implementation, using Feign, we should be able to make the test pass. (JsonPath is used for brevity)
To tell the test to hit the stub instead of the real API we need a service mapping configured below. In production you will want to point
Our tests are passing. Nice. Let’s ship it. We schedule a build to run the tests of the proxy project in an interval of our liking. For example once a day. The tests for the consumer projects can be run as often as we like. They never call the real API.
Are we done?
Not really. All seems fine until a road gets blocked and suddenly the duration takes one second more and we receive
22652 instead of
22651. Subsequently leading to a failing test. Instead of just changing the value to
22652 and waiting for it to change again on a future date we should improve the resilience of the test.
The updated syntax of the value part would be
stub (WireMock) will always provide the value
22651 to a consumer. The actual tests (the assertions) against the real API will only require the API to respond with at least one digit. That’s what the
regex('\\d+') is for. Thus our generated assertion will change to
The real API might return
12345 and the test would not care. Nor should it. Subsequently the test on the consumer side, which is expecting
22651 would not break.
Seems were done…
Not really, try to create the following request
The API does not always return the duration. Definitely not when using a fictional city. We could, and should, create a second contract covering the error case and make sure our
Client can deal with the error. Currently it can’t. It would run straight into
NullPointerException. Having contracts, and thus stubs, for these edge cases is a great way to harden the
Client to deal with more than the happy path.
For the examples above everything looks great. Until we get to a requirement of preparing the API to respond with something specific. Say to fetch something via
GET request we might need to first
POST something. We also did not take into account adding a personal API key to the request. A way to address these issues will be part of a future post.
To summarize, using the above technique, we made sure
Hopefully this might be of use to others too. It surely is for us. We use the approach for every integrated third party API.