How to Test API Security: A Guide and Checklist

APIs are the pipes that connect various applications and (micro)services. As data flows through them, security is of utmost importance to prevent data leakage. Also, since APIs are like doors into your application, they're the obvious entry point for attackers who want to break your system. How do you test your API's security to ensure your data is safe, your user's privacy is protected, and your system remains healthy?

OWASP, the Open Web Application Security Project, has created a list of the top ten security issues applications typically face. They've also created a specific version for APIs because while some security concerns affect all kinds of apps, there are also API-specific issues. This article will discuss testing APIs for security in general and then will look at each specific problem. We'll end with a checklist that you can use to walk through your test setup.

Security Testing as Part of API Testing

First, let's zoom out a little and look at API testing to see where security testing fits in. Generally speaking, API testing starts with functional testing of individual API calls. You can think of them as unit tests. There's a valid input and an anticipated response for each test, and running the test confirms that the response matches expectations.

Then you create tests covering a chain of API calls for expected use cases. Finally, you can set up performance testing to ensure that your API remains functional and reliable under higher load.

You can also set up monitoring to make sure your API remains available and reliable over time. These are generally positive tests for the happy path, which means you define a desired input and outcome and check that the API works as expected.

Security testing mostly comes in after the first level of individual API tests. In addition to the valid inputs, you also create test cases with invalid requests. These so-called negative tests help you figure out if your API error handling is working as expected. You can also use these negative tests to confirm your API security through the creative design of invalid inputs that could break your API or leak data. Getting insights from tracing data through tools like Traceable AI can help you discover API usage and potential edge cases worth testing.

You should also include negative tests in your performance and API monitoring, especially when running stress tests. Some security issues may manifest themselves only under these circumstances.

Tools For API Testing

As we've established, security tests are similar to functional tests. It's just that they're negative tests, which means you don't test for desired outcomes but for undesired outcomes. Hence, you don't necessarily need specific tools but can use the same tools as with functional testing. A common choice is using the Postman HTTP client to design tests and then automate their execution with its command-line companion Newman. Another interesting tool is Taurus, an automation framework for multiple test runners.

If you have a machine-readable definition of your API in OpenAPI format, you can use that definition to help design tests. You can import OpenAPI files in Postman, and Taurus also supports converting OpenAPI into test definitions. There are some cloud-based testing solutions as well.

You can run some tests on your development machine or during continuous integration and deployment (CI/CD). Of course, you can test in production, but that may mean polluting your production system with test data, and it's already a bit late (although better late than never) to find out what's wrong when it already affects users. Ideally, you'll have a staging or test environment to spin up on demand or to run continuously. As authentication and authorization are at the heart of many security-related API problems, a test environment with a good amount of users and data with realistic permission settings is helpful.

Creating Test Cases

In software testing, we generally differentiate between black-box testing, where the tester doesn't know the system's internals, and white-box testing, where they do. You can create most security tests as black-box tests by going beyond the documented API's confines and seeing what happens. On the other hand, knowing something about the API and the underlying database helps find edge cases that could cause problems, such as fields that exist as database columns but not in the API. White-box and black-box testers, or security experts and software developers, can collaborate and contribute test cases to the pool of security test cases.

Now you should have a general idea about testing and tools. For the remainder of this article, we'll look at the most critical OWASP security problems for APIs and potential test cases to find them.

Authentication and Authorization

We commonly group these two "auths," but it's crucial to understand their differences. Authentication is about establishing the identity of a user, i.e., who they are and if they've adequately identified themselves. Authorization is about asserting the permissions of the authenticated user. Three items on the OWASP API Top 10 deal with the two "auths." They are API1:2019 Broken Object Level Authorization, API2:2019 Broken User Authentication and API5:2019 Broken Function Level Authorization. API3:2019 Excessive Data Exposure is also related.

Authentication

APIs can have different authentication mechanisms just like doors have various kinds of locks. Standard mechanisms are HTTP Basic Authentication with a username and password, API keys passed as headers or query parameters, and OAuth 2.0 Bearer Access Tokens. The first straightforward test case is accessing API endpoints that require such a credential with no credential or an invalid one. Make sure to test all HTTP methods, including those probably absent from the API definition, like HEAD or OPTIONS. If your server returns anything other than 401 Unauthorized, make sure to fix that.

Authorization

Authorization is a more complex beast. On this blog, we previously covered an authentication-related issue that Uber faced. Don't let that discourage you. It's vital to test authorization after authentication as well.

First, to test role-based access control (RBAC), you need a list of different roles for your API users. There is typically a user and an administrator role, but you could have more than that. Write down what they're supposed to do in the API and, even more important, what they are not supposed to do. This information belongs in your API documentation, by the way, to raise awareness of each user's capabilities. Sadly, OpenAPI support for expressing security is limited.

Now, for testing, create a user for each role with an appropriately scoped credential. Then design positive tests and negative tests to ensure that users can do what they're allowed to do and are 403 Forbidden from doing other things.

Resource-Level Access Control

Independent of role-based access control, you probably have private resources in your API that are only available to their creators or share targets. Make sure that only those with the proper permissions can access them.

As part of your API design, you have to decide how to handle requests to resources that aren't allowed. For example, if /objects/{id} points to a private entity and an unauthorized user calls GET on this URL, you can answer with 403 Forbidden. However, it may be more secure to answer with 404 Not Found instead because the mere existence of a URL could reveal unwanted information and invites malicious API consumers to attempt to brute-force URLs.

You must also check collection endpoints. Even if /objects/{id} rejects access, is the resource listed in endpoints like /objects or /users/{id}/objects? It shouldn't be.

There are use cases for long, unguessable URLs, but you generally shouldn't rely on them as a security measure in APIs. Even if the ID in /object/{id} is something like a UUID, your API should block access to unauthorized users. Make sure to test that.

Field-Level Access Control

We've looked at authorization on a resource level, but what about the field level? There may be some response fields that your API only wants to reveal to users with a particular role. Hence, your test cases should not just check for HTTP status codes or other shallow response information. Also inspect the JSON structure of the response.

Does your API feature a selection filter, such as a query parameter named "fields" to select the fields included in a response? If so, make sure to test this parameter as well. A common issue is that role-based permissions remove an unauthorized field from the JSON object but including that field's name in the parameter reveals the field again.

If your API implementation is close to its database schema, a field filter parameter might provide access to database columns that you never want in your API.

Input Validation

Related to the issue of field-level access control is an issue that OWASP lists as API6:2019 Mass Assignment. This time we're not looking at retrieving data but rather at sending data to the API. What does your API do when the user sends additional fields in request bodies besides those you included in your API definition? Does the API break? Are those stored in the database as well, or could this override internal defaults, including those relevant for security? That's one of the cases where white-box testing is crucial.

APIs should define constraints for inputs such as data types and ranges (for example, a particular parameter could only be valid as an integer between 1 and 100). With JSON Schema in OpenAPI, you can encode these constraints for both documentation and implementation of validation. Always create test cases outside the limitations and let your test runner confirm that they result in a 400 Bad Request error.

If you have string inputs and an SQL database in the back end, create negative tests with queries or commands. OWASP lists API8:2019 Injection as an issue for APIs just as it is for web applications.

Other API Security Issues

Another one is API4:2019 Lack of Resources & Rate Limiting. Your API should include rate limits to prevent overloads and brute-force attacks, such as continually trying random keys until one works. That's one of the things you can examine as part of performance testing. If you have documented rate limits, go beyond them to ensure that all endpoints reject additional requests with an appropriate error response.

Also, try to break your API with a stress test. Then look at the 500 Internal Server Error (or other errors from the 500 range). Do your API server, API gateway, or reverse proxy produce these errors? What do they contain? Make sure they don't include things like stack traces, internal network topology, or other private information. Debug information should be only for your developers and network admins, not the API consumers (this relates to API7:2019 Security Misconfiguration).

One last thing I want to highlight is a case of API9:2019 Improper Assets Management: older versions of your API. If you use API versioning and keep multiple versions of your API deployed, your ongoing testing strategy must cover all of them. If you only look at the most recent version, you may overlook older issues that someone might exploit.

Checklist

We've already covered some ground for security testing. Let's put it in a checklist that you can use to design and execute your testing strategy:

• Whenever possible, launch a separate test environment of your API so you can test without breaking production.
• Set up functional tests for the happy path first and automate them with a toolchain of your choice.
• Design negative tests for edge cases that could result in security-related issues using the same tools. For a first quick win, start by testing authentication.
• Create thorough documentation of all access control mechanisms in your APIs, such as roles. Create test users with possible sets of different permissions as well as private resources. Then design test cases where these users attempt to access unauthorized resources. Remember, authorization is as essential as authentication!
• Don't treat your API as a black box. Understand your back-end architecture and find the kind of issues it's vulnerable to (such as mass assignments, SQL injections, etc.).
• Create test cases with input beyond limits. Examples are entering additional properties, going past defined constraints, and command or SQL injections (if necessary).
• Observe all error responses for leakage of internal information.
• Include security tests during performance testing to ensure that any odd behavior under stress doesn't break security.

The list should be an excellent starting point to ensure your API's security. Note that this list and any single blog article cannot cover the subject in depth. Testing is an ongoing process, and you can always improve your test coverage. Most important, however, is to get started now and find your security issues before someone else does!

This post was written by Lukas Rosenstock. Lukas is an independent PHP and web developer turned API consultant and technical writer. He works with clients of all sizes to design, implement, and document great APIs. His focus is on creating integrations and mashups that highlight the values of APIs, and educate developers about their potential.