Runscope API Monitoring    Learn More →

Posts filtered by tag: api testing

How to Debug Common API Errors

By Heitor Tashiro Sergent on .


Debugging and troubleshooting APIs is something that any developer that works with APIs has to go through at some point. In an ideal world, APIs would always return a 200 OK with just the right data that we need, or in case of a failure, it would return us the perfect status code and error message allowing us to easily understand what went wrong.

In reality, APIs don't always work like that. API developers might have constraints that do not allow them to implement the most informative status code or error message, API errors can be triggered by real-world conditions that are hard to account or test for, and sometimes ourselves, as API users, can make requests with typos or mistakes that APIs just don't know how to handle.

In this post, we're going to focus on API users and what they can do to debug common API errors they might encounter when testing and working with APIs, whether these APIs are public or private […]

Read More →

Categories: testing, debugging, apis

Using Google Sheets and Runscope to Run API Tests with Multiple Variable Sets

By Sam Aybar on .

We frequently run into users who are looking to run an API test multiple times, using a variety of initial variables. For example, the user might be looking to:

  • Test the same request against a list of user ids, where user_id is the only query string parameter that changes.
  • Test edge cases for an API call. Passing a string to an endpoint that expects an int, or passing special characters to it, and making sure that nothing breaks.
    Share tests and results for multiple variable sets with other teammates for debugging.

We have been able to accommodate this feature for a long time using our Batch Trigger URLs. With a GET request to a Batch Trigger URL, you can pass in an array of objects to specify initial variables for the test, and kick off up to 50 test runs at one time using different sets of variables.

But what if you want an easier way to do this than to try to format an API request with a bunch of variables and build a JSON array with the different sets? And how do you match up your responses easily to the different sets of variables? What if we could make this easier?

Read More →

Categories: debugging, integrations, testing

How Trustpilot Tests and Monitors 200 Microservices

How Trustpilot Tests and Monitors 200 Microservices

By Heitor Tashiro Sergent on .

Trustpilot's journey with Runscope began back in 2015. They started off with our free trial plan, and have since grown to almost 5 million test runs a month. They were kind enough to chat with us about their experience, and how they transitioned from a single monolithic application, to a more efficient microservice architecture with over 200 microservices and hundreds of endpoints. 

Here's Dumitru Zavrotschi sharing their story...

Read More →

Categories: customers, microservices, monitoring, testing

6 Common API Errors

6 Common API Errors

By Heitor Tashiro Sergent on .

Have you ever used an API that returned an HTML error page instead of the JSON you expected, causing your code to blow up? What about receiving a 200 OK status code with a cryptic error message in your response?

Building an API can be as quick as serving fast food. Frameworks like Express, Flask, and Sinatra combined with Heroku or zeit's now help any developer have an API up and running in a few minutes.

However, building a truly secure, sturdy, hearty API, can take a little more work, just as a chef takes more time when crafting a great meal. You need great docs, clear and concise error messages, and to meet developers' expectations of...

Read More →

Categories: apis, debugging, microservices, monitoring, integrations, testing

3 Benefits to Including API Testing in Your Development Process

By Ashley Waxman on .

One of the most common ways we see our customers benefiting from API monitoring is in production—making sure those live API endpoints are up, fast and returning the data that’s expected. By monitoring production endpoints, you’re in the loop as soon as anything breaks, giving you a critical head start to fix the problem before customers, partners or end-users notice.

However, we’re starting to see more and more customers create those API tests during the build process in staging and dev environments. As Matt Bernier, Developer Experience Product Manager at SendGrid, says, “We can actually begin testing API endpoints before they’re deployed to our customers, which means...

Read More →

Categories: events, testing, product

6 Common API Testing Mistakes (and How to Avoid Them)

By Neil Mansilla on .

If you work with APIs in your day-to-day, then you’ve probably experienced this scenario: you put in several hours to create tests for all your services, and then plug them into your workflow, but somehow you still aren’t aware that your APIs are failing. Either a customer tells you before you find out, your team can’t triage an API issue fast enough, or your tests are passing but your apps still break. If you’ve encountered any of these headaches, the problem may be your process around building and maintaining your API tests. We’ve identified six API testing mistakes that engineers, API developers, QA testers and DevOps teams alike have made so we can help you solve those problems and build better APIs together.

1. Building tests that don’t represent real functional use

It’s easy to set up tests that verify independent services and endpoints, and then call it a day when they all pass. Test Inventory API, check; test Shopping Cart API, check; and so on. However, your end-users are likely consuming these methods in conjunction with one another, not independently. Building tests without considering how the APIs will be consumed may be quicker in the short-term. However, in doing so, you won’t be testing across concerns, which could prevent you from uncovering and debugging potentially serious API issues.

In traditional software testing, tests built across multiple units or functions are called integration tests. Building integration tests for APIs can be easier than you think. Take a typical retail API scenario. We’d test some inventory/SKU management resource methods along with a shopping cart resource method, like this:

  1. GET /items (fetch a list of items)

  2. POST /items (create a new item)

  3. GET /items/{itemId} (verify existence of newly created item)

  4. POST /cart (add this new item to the shopping cart)

  5. DELETE /items (remove the item)

  6. GET /items (verify that the item has been removed)

Each of the APIs used in the above scenario may work when tested independently, but without testing the entire flow, you cannot be sure that they are working together, as intended.

2. Leaving out response time assertions

API tests can be built to check for any number of variables, like status codes and response content. Those pieces may be vital for checking method correctness, but what if an API request is taking 10 seconds to respond? Does this test still sound like it should pass? While often overlooked, response time assertions are a quick but necessary addition to any API test to make sure all your boxes are checked when it comes to a complete end-user experience.

Set up response time assertions that are reasonable and represent how long you or your developers think it should take. If you start at a high threshold, you can easily scale down and see what works for that particular request. A high threshold response time assertion is far better than nothing, particularly when testing production endpoints. An app that takes too long to load can send your consumer off to the next app.

3. Not including API dependencies

Traditional software integration testing involves testing separate units of code together, ensuring that they operate consistently and reliably, together. With modern applications that depend heavily on web services, it is commonplace to rely on web services that live outside of your four walls. Therefore, testing only your own APIs doesn’t give you the whole picture of how your app will operate in the real world. Your API is a product that depends on partner services, and if any of those services fail, your API could also be failing to your customers.

A good rule of thumb: If your product relies on it, then you should test and monitor it. Third-party integrations can be just as valuable as your own APIs, and if your app or service is broken, your consumers won’t know (or care) about whose service is failing.

4. Testing APIs in a vacuum

Building API tests can be a bit of a solo act, but the minute a test is in your workflow, you need to bring in the appropriate teams for when issues do occur. API tests don’t always fail for the same reasons and can impact a variety of stakeholders. Therefore, test failures require the attention of different teams in your organization. If you set up test failure notifications to go to just you, you’re adding time, effort and headaches to your workflow.

The minute an API test is added to your development or operational workflow, involve the right people via the notification channels they use most. Add notifications for the team responsible for remediating API issues by integrating your API tests with Slack, PagerDuty, HipChat and other tools to empower your whole team to solve API problems fast.

5. Ignoring intermittent problems

When your tests come back with 99.99% success, that measly 0.01% failure rate is easy to cop to minor blips that will recover on their own. Errors that occur infrequently and recover quickly become easy to ignore. However, by not digging into the root cause of intermittent problems, or worse, not realizing an increase in their frequency, you could be missing an opportunity to catch a systemic problem early on that may manifest as a much bigger failure later.

Going into traffic logs to debug these issues is an extra step, but it’s a step worth taking. Runscope logs all the API traffic that runs through your tests and allows you to compare requests and results side by side so you can debug intermittent problems quickly, and avoid potentially serious future problems.

6. Managing everything by hand

With so many dev tools out there helping you automate your workflow, there’s no reason to keep creating API tests manually. Doing this work by hand can be cumbersome and take time away from other important work. One way to streamline your test creation process is by importing the work you’ve done in tools like Swagger or importing HAR files. In Runscope, these definitions and files can instantly be turned into API tests.

If you manage large test suites or have complex multi-step tests, break free of the UI and launch your IDE. The Runscope API allows you to programmatically read, create, modify and schedule large numbers of complex API tests and puts test automation and management in your hands.

Start Testing the APIs You Depend On

With these best practices in your arsenal, you’re set to begin testing the APIs that power your business. Sign up for Runscope to start testing your APIs today. You can also join us this morning at 10 a.m. PT for our free live webinar, Getting Started with API Monitoring. [UPDATE: This webinar has passed, but you can watch the recording.] We’ll show you how to create your first API test, notify your team when tests fail and more. If you can’t make it, stay tuned for the recording!

Categories: apis, howto, testing

Everything is going to be 200 OK®