Solve API problems fast with Runscope. Learn More →

Running a single API test plan against multiple environments

By John Sheehan on .

Every day we hear from Runscope customers who are looking to run the same Radar test against multiple environments. Sometimes it's "staging" and "production" environments, other times it's for a fleet of servers behind a load balancer that all serve the same API.

In this post I'll show you how to use a few of Radar's features in order to create a single test plan that can be applied to different environments.

Creating the Test Plan

Let's start by creating a simple test plan using the Runscope API. In this case we'll just hit a set of endpoints in our 'prod' environment directly:

This is a simple test that checks that our credentials are correct, creates a new bucket, then retrieves the newly-created bucket (the second request stores the new bucket key in a variable if you were curious).

You can see that all of the requests are hard-coded to use our production host name in the URL. We also have an "Authorization" header (not shown) with a access token that will only work in our production environment. To make this test work with our staging environment we need to start by abstracting out the environment-specific data into variables.

Using Initial Variables for Environment-specific Settings

I'll define two Initial Variables for our baseUrl and accessToken values, using the same values I previously hardcoded into the request editor.

I've included the the scheme in our baseUrl value since our staging environment uses plain http instead of https. I also left out the trailing slash since it makes tests easier to read later on when we use that variable.

With our initial variables defined, update the requests to use the variables:

When I run the test from the dashboard, it will use the values I've entered in the Initial Variable editor and dynamically insert them into the requests:

With our test plan defined, we can now use Trigger URLs to initiate test runs with custom sets of initial variables.

Dynamically Specifying Environment Variables with a Trigger URL

Trigger URLs let you kick off test runs from other tools that are part of your build or deploy process, like GitHub, Jenkins, Heroku and more. Every test has a unique Trigger URL that accepts a single GET or POST request to start a test run. These URLs also accept parameters to allow you to specify custom initial variables to use for a given test run.

For this test, I've put together a Trigger URL that sets the baseUrl and accessToken values to match those required by our staging environment. It looks like this:

Requesting that URL via curl or the browser will start a new test run with those values, overriding the default values and dynamically inserting them into the request data:

For our case, we make a request to the Trigger URL at the completion of a deploy for a service to any given environment. We can also run these tests against local or private endpoints using the Runscope Radar agent (in private beta, contact us to learn more).

Need help with your own API tests?

If you need help getting your API tests configured to run in multiple environments, drop us a note, we're standing by ready to help.


New and improved options for programmatically interacting with Runscope Radar

By John Sheehan on .

Every team works with their APIs a little differently. Because of the variety of environments our customers have, it's important that we provide flexible extensibility options to make sure we allow developers to build integrations we may not be able to build ourselves. We offer three main ways to do that: the Runscope API, Radar webhook callbacks, and Radar Trigger URLs.

Over the past few months we've been improving these extensibility points based on your input and feedback. The backlog of undocumented features was getting a little long, so we've documented them all and have summarized the changes below (with a bonus Passageway feature).

Radar: Test Results API

We've added additional endpoints to the Runscope API (for interacting with your Runscope account data) for Radar tests and results. You can use this API to build your own test status dashboards or integrate with other services that don't accept webhooks. 

Learn more 

Radar: Trigger URL for All Tests in Bucket

We've added a Trigger URL that will queue a test run for all tests within a bucket. This is similar to the "Run All" button on the Radar test overview page. Bucket-Wide Trigger URLs support custom variables and can be found in the Bucket-wide Settings.

Learn more 

Radar: Batch Trigger URL

Sometimes you want to run a single test with multiple sets of initial variables in one go. The Batch Trigger URL supports just that, allowing you to start up to 50 test runs for a single test at once, each with a unique set of initial variables. 

Learn more 

Radar: Additional Trigger URL Response Data

All Trigger URLs have been updated to use a new response format that will detail the one or more runs that were initiated, including the resolved initial variables (the variable state after bucket-wide and test-specific initial variables and scripts have been processed). The response also includes the agent ID or region code that will execute the test run.

The new response is returned when using a Trigger URL ending in /trigger, bucket-wide Trigger URLs, or the Batch Trigger URL. For backwards compatibility, Trigger URLs using the previous format (URLs ending in /start or /run) will continue to function indefinitely. We recommend migrating to the latest version at your earliest convenience.

Learn more 

Radar: New Trigger URL Options for Locations

Two new parameters are now supported for choosing which locations to use for the test runs initiated by the Trigger URL. You can specify one more more runscope_region parameters to initiate test runs is specific geographic locations. If you're running the Radar agent in your infrastructure, you can specify one or more runscope_agent parameters to start a test run using the specific agents.

Learn more →

Radar: Additional Webhook Notification Payload Information

The payload delivered for webhook notifications has been updated to include the location (region or agent) the test run was executed in, more details about the requests executed (including response time, size and status), and the initial variable state before the first request was executed.

Learn more →

Radar: New Data Sent to New Relic Insights and Keen IO Integrations

Location (region or agent) for a test run is now included in the test run event data sent to New Relic Insights and Keen IO.

Radar: New Built-in Function

A new built-in function to generate a random string of up to 1000 characters is now available. Use {{random_string(10)}} (replacing 10 with your desired length) to use the new function in your request templates.

Radar: New Script Functions for URL-safe Base64 Encoding/Decoding

Radar Scripts can now use atob(input) and btoa(input) to do a URL-safe Base64 encoding or decoding of a string. This mirrors the functionality of window.atob() and window.btoa() in browsers.

API: Delete Bucket

The API now supports deleting buckets (except the default bucket on an account) by key.

API: Share Request

A new endpoint has been added to support creating a public share URL (similar to sharing a request in the dashboard) for a given message.

Passageway: Predictable URLs

Passageway generates a unique tunnel URL on each initialization of the agent. This can be annoying if you're using Passageway as part of a CI process. Using the new --fixed command line flag you can get a consistent URL between runs of the agent.

Keep up With the Latest Updates

Be sure to follow @Runscope and keep your eye on the Changelog for all the latest product updates new features.


Monitor and Test your APIs that use OAuth 1.0a with Runscope

By John Sheehan on .

API authentication schemes come in all shapes and sizes. Tokens, HTTP basic auth and OAuth are the most common that our customers use. Different versions of OAuth present different challenges though, with OAuth 1.0a being the most complicated of them all.

Today we're launching native support for OAuth 1.0a in your Runscope API tests. No more complicated scripts or request signing steps, just enter your keys and run your tests.

Simplifying Signatures

If you're unfamiliar with OAuth 1.0a, let's recap how it works. Every request you make to a protected resource on an API that uses OAuth 1.0a needs to be signed using a shared secret that only you and the API provider know, specific to your app. Request signing is a complicated, error-prone process; thankfully made easier by the plethora of libraries for various languages.

When testing or monitoring APIs with Runscope, these signatures were especially hard to generate. Now, paste in your credentials and the request signature will be automatically generated and added to the request:

As you can see in this example, Variables are fully-supported so you can store your keys once for a specific test or all tests within the same bucket.

If your API developer portal doesn't generate keys for you to test with, the Runscope OAuth 1.0a Token Generator can help you out.

More to Come

We'll continue to improve support for different authentication schemes, including looking into ways to address the UI-driven portion of OAuth 1.0a or 2 authorization flows. Help us out by letting us know what your API authentication requirements are.


API Craft 2014 Conference Recap

By Darrel Miller on .

The agenda.

The agenda.

Earlier this week I headed to Detroit for the second annual API Craft Conference. Being my first time I didn't know what to expect, but I knew many of my fellow API design aficionados would be on hand and I was eager to learn from the many smart attendees.

Day One: Hypermedia in Focus

This year's API craft started with a Hypermedia Panel. The creators of all the most popular hypermedia formats were on the panel and they all had an opportunity to do a short presentation. The event was streamed and recorded.

One of the most important takeaways for me from this panel, was that there is a growing appreciation for the fact that hypermedia is more than just defining passive relations between resources.  Abstracting behavior behind hypermedia links is an equally valuable scenario.

Days Two and Three: More than Just Hypermedia

As an open spaces conference, the second day started with people pitching ideas for sessions. I was particularly interested in the sessions relating to ways of communicating semantic meaning because I believe that is one of the biggest remaining challenges in API design. Some clear themes emerged from the session proposals: testing, measuring, monitoring and managing of APIs was well represented, as well as plenty of sessions around making the business case for APIs and especially hypermedia APIs.

The rest of the day was jam packed with valuable sessions and conversation. I learned about the variety of different ways people map their domain to resource models in Glenn Block's session on Web API architectures. In the API testing session run by Kevin Shamoun it was interesting to hear the tools developers were using and the challenges they were facing.  Kevin Swiber held a very well-attended session, amusingly named 'Why y'all hatin' on hypermedia?' where we discussed commonly heard objections to hypermedia and solutions to overcome those objections. 

During the closing discussion, I asked the attendees how many were already working on hypermedia based solutions, and I would estimate between 30-40% of hands went up.  This is a huge increase from what I have seen in the past.

Despite the abundance of hypermedia content, there was plenty of discussion on topics like security, testing, monitoring, management, design, diagramming, mobile, IoT, and many more topics.  The proceedings for all the sessions are being recorded in the Github wiki. 

Overall, my three big takeaways from the event were:

  • Hypermedia is finally getting mainstream attention.
  • The level of expertise in HTTP API design is significantly improved from a few years ago.
  • There continues to be a big demand for HTTP API related libraries and tooling, especially on the client side.

API Craft was a great conference and I'll definitely be reserving the dates on the calendar when next year's event is scheduled.


Everything is going to be 200 OK

Sign Up — Free!