API Testing: The Definitive Guide


API Testing: The Definitive Guide

Continuing our mini-series on backend testing , we’re diving deep into API testing to discover how it can increase apps’ quality without significantly raising the QA costs.

Almost any application relies on third-party APIs for some of its functionality, from signing in with Facebook to finding a location with Google Maps. But few teams spend enough efforts on API testing unless their APIs are public.

This can have disastrous consequences.

But no worries, this guide will teach you everything you need to know about API testing:

What is API testing and why it’s important

Application Programming Interfaces (API) is what allows developers to integrate features from other apps into their own products without having to build them from scratch.

This means users can log into your event management app via Facebook, find a venue they want on Google Maps, and buy a ticket via a booking service. All without leaving your app.

APIs enhance user experience and make developers’ life easier.

They can also connect different components of an app. In a microservices architecture, APIs glue together different modules, each having a separate database and interface. This means they can function independently and provide a higher uptime.

In a traditional three-tier architecture, APIs represent the middle layer of an application. They govern the business logic and connect the other two layers (GUI and Database).

In the past, testing was done mostly at the GUI layer. By using the app’s interface, testers could find bugs in its business logic. But to start the testing, they had to wait for developers to finish the UI.

This slowed the testing process and increased the costs of fixing the bugs.

Agile development and test automation make this approach increasingly outdated.

Now developers often create their apps by writing the APIs first. And once the business logic is ready, they design a UI around it creating the user experience.

By testing at the API layer, you can find logic-related bugs much earlier and fix them for a fraction of costs.

Test automation allows you to increase the test coverage and keep the QA costs low. But automating at the GUI layer is expensive and time-consuming. Any changes to the app’s interface can easily break the testing scripts.

Automated API scripts require less maintenance than interface tests and can uncover logic-related bugs that can’t be detected with unit tests.

Of all the testable pieces of your product, APIs present the most security risks. Attackers can exploit vulnerabilities in them to access user data or infect your servers with ransomware.

The cost of such vulnerability is much higher than a buggy button or a form.

To sum it up, the main benefits of API testing :

  • Start your testing before the front-end is ready and fix logic-related bugs much cheaper.
  • Boost the security and performance of your application.
  • Discover bugs that can damage other products and business processes if your API is public.
  • Re-use the testing data generated during API tests in GUI tests.
  • Uncover performance issues.
  • Increase the test coverage and its accuracy.
  • Run regression tests to ensure that the recent changes didn’t break your APIs and the systems using them.
  • Make it easier to scale up your system (if all the business rules are implemented via APIs).
  • Work with JavaScript, Python, PHP or any other language supporting JSON/XML.

How is API testing done?

In its simplest form, API testing involves sending API calls and checking their responses.

An API can return data in different formats of which JSON and XML are the most popular, a Pass/Fail status, or a call to other APIs.

Before you begin testing your APIs, you should know what kinds of requests they can make as well as their limitations.

You can find this information in API documentation or get it directly from the developers. Ask them about the available endpoints, HTTP verbs that you can use and their authorization limits, the required fields and their validation limits, response codes that come with Failed/Passed requests and the error messages displayed with failed requests.

Only in this way you can be sure that the API behaves as expected.

Now you can set up an environment that would allow you to make API calls and validate the responses. As APIs don’t a GUI, testing them has to be done at the messaging level. You send the requests directly, but there are a number of great tools that make API testing much easier (more on that later).

After you input the required API parameters and configure the DB and server, check if everything works as it should by calling an API function.

After this, you’re ready to start the API testing.

From the very beginning, you should clearly understand the API functionality and have a well-defined scope of testing.

To learn whether your API is up to the task, you’ll have to write the test cases that include:

  • Input parameters
  • Expected responses
  • Max response time
  • Parsing inputs
  • Error handling
  • Correct formatting of the response
  • Add each test case to your test script and execute it after each new build.

Types of API tests

With these questions answered, you can start writing the test cases.

Depending on the scope of your tests, you might want to run different kinds of tests. In addition to Functional tests, it might be worth to run Security, Regression, Documentation, Reliability,

Creativity, Negative, and API Integration testing.

#1 Functional testing

These tests ensure that the API is operational and can be easily integrated into other systems.

In functional testing, you’ll have to look at separate user stories (features from the user point of view) and check if they satisfy the requirements. The aim is to ensure that the system can handle all the possible scenarios and errors gracefully.

First, you’ll have to go through the requests that users are likely to make while navigating your app as usual (the so-called “happy path”).

At this point, remember to add assertions (e.g. the API should return a correct response code).

After exhausting the “happy path” tests, you can move on to the negative tests. Their aim is to test how the API reacts to all possible incorrect user inputs and ensure that any error is handled gracefully.

They ensure that hackers can’t inject malicious scripts into the database and that system doesn’t crash when somebody makes an invalid input. These can involve sending requests with the wrong endpoints/HTTP verbs/headers, making API calls without authorization, or requesting the non-existing data.

Functional testing can be done on three levels:

  1. Contract tests

API contracts specify how to communicate with an API, what services you can request and how. If your contract isn’t up to snuff, your API is basically useless.

Contract tests make sure that the contract is properly written and can be used by the developers.

2. Component tests

Now you’ll have to validate each method available in your API separately. To write component tests, consume the API contract and generate the clients. Go through your test cases one by one making requests with positive and negative data.

Next, validate that:

  • The requests and responses are well-structured.
  • The API returns an appropriate response code (including an error if you’re going for it).
  • The error messages contained in the body of a response are correct
  • The output meets the expected baseline (by making assertions or using regression/diff).
  • The APIs have an expected response time.

Component tests are, perhaps, the most important part of testing your API as they can lay the groundwork for all subsequent tests.

3. Scenario tests

Now you can combine individual component tests into various scenarios.

You can either look at the user stories to understand what API requests are made in each particular scenario. Or you can perform the required actions via your UI and capture the traffic that goes to the APIs.

Either way, scenario tests can reveal any unexpected behavior when combining different API calls.

After your API is released, users and developers might start using it in unintended ways, which can lead to unexpected bugs. Therefore, it’s important to write as many scenarios as possible with

unexpected combinations of API calls.

When new functionality is added to the system, you can re-use your component tests to write scenarios for the added features.

#2 Security testing

Testing security is crucial to the success of any product as an exploited vulnerability can easily destroy your reputation.

In August 2017 Panera Bread’s webapp has started leaking the personal data of its users . An unauthorized API endpoint allowed anyone to access the customers’ names, birthdates, addresses, phone numbers, and even credit card data.

At least 37 million people were affected by this incident.

To secure your app, you’ll have to validate access controls, user authorization, encryption of sensitive data and other security requirements.

You can use the existing test scenarios to discover the attack vectors and simulate hacker attacks in penetration tests.

You can, for example, combine your scenario tests with SQL injection attempts or parameter fuzzing. As the result, your security tests will reveal any changes caused by these attacks.

Test Automation

One of the reasons so few teams run through API tests is that they simply lack time. With Agile development, there’s always pressure to deliver your product faster.

The only way you can buy enough time for deep API testing is by investing in automation.

You can write test scripts once and have them run automatically every time developers make changes to the API code. Automation testing process can eliminate the repeated tasks, speed up testing, and increase its coverage.

Smartbear predicts that by 2020, the number of teams automating >50% of their API tests is going to increase by 30% (from 59% to 77%).

API testing is one of the areas where automation is most valuable.

Testing your API on a continuous basis is essential for maintaining the security of your app.

Automated tests make it easier to cover the full spectrum of API tests: functional, security, positive, negative, edge cases, etc.

Unfortunately, 100% API test automation isn’t cost-effective.

Not all parts of your system have an equal priority when it comes to test automation. It’s important to develop a strategy that would maximize the development speed and ROI.

You can add the most value to your product by automating:

  • Regression testing.
  • Functional testing.
  • Performance testing.
  • Multilingual testing.
  • Error testing.
  • Data-driven testing.

There are, of course, some limits to automated API testing. You still need manual testers to account for the multitude of use cases, security vulnerabilities, and possible integrations.

For this reason, usability, exploratory, and ad-hoc testing are better left to humans.

API testing tools

There are many tools that can help you test your APIs. At MindK, we prefer Postman for manual testing and SoapUI for automated.

Postman is one of the most popular tools for ad-hoc and exploratory testing with more than 5 million testers using it regularly.

It’s a simple REST client that comes in the form of a Chrome extension and desktop apps for Windows, Linux, or Mac. You can download them for free or subscribe to Pro/Enterprise plans ($10/18 per user per month).

Unlike most similar tools, Postman has an excellent (if somewhat slow) UI. This makes it an excellent choice for testers who don’t want to get bogged down in coding.

You can run API tests in isolation using Postman’s request builder or create collections of multiple tests.

Postman supports easy knowledge sharing and collaboration. You can, for example, upload the test cases to your knowledge base or package API calls and the expected responses into a collection and send it to your colleagues.

This is a powerful, yet sometimes confusing feature.

The tool also allows you to create Boolean tests, extract data from most modern web APIs and even automate your tests. Among its neat features are support for multiple environments (e.g. development, staging, etc.) and REST requests.

At the same time, it’s impossible to build a chain of actions in Postman and its WYSIWYG editor is far from perfect.

If you need advanced automation or complex test scenarios, nothing beats SoapUI . It’s the most popular tool on the market that supports both RESTful and SOAP API testing.

It comes in two formats: a free open-source version with the full access to the source code and a Pro version (€595/year) with some extra features (e.g. an SQL query builder, a form editor, and XPath assertion wizard).

SoapUI is easy to learn (you can get up to speed in just three days). It has a convenient drag-and-drop interface that allows you to quickly create complex asynchronous tests and simplifies working with XML/JSON.

And if you need to go beyond the standard functionality provided by the tool, like assertions or workflows configuration, you can these features them with your own code in Groovy.

You can load data from databases, files, or Excel and simulate the API interactions in data-driven tests. SoapUI allows you to repurpose performance and security tests into functional test cases with just a couple of clicks.

Unfortunately, the free version lacks version control for the test scripts along with some other limitations.

Wrapping up

And that pretty much it for API testing.

It’s time to consider the benefits it can provide your organization against its costs.

Can the potential savings from fixing bugs early make up for the larger QA budget? Can the increased test coverage and accuracy add enough value to your product? Can you risk releasing an insecure app?

Whatever your answers are, I hope you’ve found everything you need to make an informed decision.

And now I’d like to hear what you think about API testing.

Do you find it beneficial to your product? Do you plan to implement it in the future? Or, maybe, you consider the costs too high?


API Testing: The Definitive Guide

Jameson Lopp’s Not so Secret Bitcoin Superlist


Check out some of the new features coming to the iOS and Android mobile Office apps



API Testing: The Definitive Guide