Runscope API Monitoring    Learn More →

Posts filtered by category: code samples

Tutorial: Continuous Integration with Codeship and Runscope API Tests

Tutorial: Continuous Integration with Codeship and Runscope API Tests

By Heitor Tashiro Sergent on .

A part of an engineer's toolbox is a good automation workflow. One of the tools in that box is a Continuous Integration provider, which can help teams prevent integration problems and improve quality control.

Runscope can be used by itself on running and monitoring APIs, but it can also be used as part of your CI workflow. Our CEO, John Sheehan, created a sample Python script that can be used to trigger a set of tests in your Runscope account, and change the build status based on their results.

In this tutorial, we're going to show you how to use that script with Codeship. We'll cover how...

Read More →

Categories: code samples, tutorial, testing, integrations

Working with the Azure Storage REST API in automated testing

Working with the Azure Storage REST API in automated testing

By Chris Kirby on .

This is a guest post by Chris Kirby, Director of Technology at Authenticom. He's a big fan of automating processes, beer, and games. You can follow him on TwitterGitHub, and his blog.

Testing! Every developer's favorite topic :). For me, if I can save time through automation, them I'm interested. Automated testing for a developer typically starts with unit tests, which even if you don't subscribe to TDD, you've written at least one of them just to see what all the fuss was about. Like me, I'm sure you saw that testing complex logic at build time has huge advantages in terms of quality and taking risks. However, even with the most comprehensive tests at 100% coverage, you've still got more work to do on your journey towards a bug-free existence.

Given that most modern applications rely on a wide variety of cloud platform services, testing can't stop with the fakes and mocks...good integration testing is where it's at to get you the rest of the way. Integration testing is nothing new of course, it's just more...

Read More →

Categories: apis, code samples, customers, howto, testing

Using Runscope to Test APIs Protected with the Hawk Authentication Scheme

By Gustavo Straube on .

We’re excited to have Gustavo Straube, Software Engineer and Co-founder at both Creative Duo and All Day Use, show the Runscope community how to test APIs that use alternative or custom authentication schemes.

Nowadays, employing authentication protocols with your APIs is a necessity, but dealing with them can be taxing. At Creative Duo, we usually build APIs to act as the backend for mobile apps and modular systems. In both cases, even when using TLS protocol to protect the data, we have to guarantee that only authorized requests get a valid response from any service. In the past, we used a home-grown solution to avoid unwanted access, but it wasn't reliable and writing a trusty security protocol has never have been our focus.

After trying some alternatives, we ended up choosing Hawk authentication scheme for its simplicity and ease of integration. It provides enough safety for our applications and their consumers. The primary design goals of Hawk are to:

  • Simplify and improve HTTP authentication for services that are unwilling or unable to deploy TLS for all resources
  • Secure credentials against leakage
  • Avoid the exposure of credentials sent to a malicious server over an unauthenticated secure channel due to client failure to validate the server's identity as part of its TLS handshake

As we started to configure the first test we found a little problem in our way: the API we were testing uses Hawk for authentication, and we only found options for Basic Authentication and OAuth 1.0.

Checking the available configurations, there was an option to add static headers. However, as a replay protection procedure, authentication headers in Hawk are valid for only one minute. With that in mind, the static header was not an option since we must update the header within one minute. Furthermore, automatic tests would be impossible to run.

Recently we discovered Runscope and its awesome features to monitor and test APIs. After a tweet exchange a few emails with Runscope team, we got a script to start with. Yes! It is possible to write scripts to run before (initial scripts) and after tests. The scripts should be written in Javascript, which is a great choice since a lot of developers know at least a bit of JS. Also, it's possible to use a few common libraries with scripts, like Moment.js and CryptoJS.

Creating the Authentication Header

Before creating the dynamically generated authentication headers, we must set up some variables we'll use in the script.

Now we can start to build the header itself. Hawk protocol is simple: it basically uses an HMAC hash, created with the request info, using a secret key.

Knowing the protocol basics, the following code becomes pretty obvious.

Step 1: Set up the request data into variables.

var now = parseInt(moment() / 1000);
var method = "GET";
var path = "/api/path/to/resource";
var host = "";
var port = 80;
var ext = null;
var nonce = Math.random().toString(36).substring(6);

Step 2: Create the normalized request string, with each value followed by a newline, as required by the protocol.

var artifacts = "hawk.1.header\n" + now + "\n" +
     nonce + "\n" +
     method + "\n" +
     path + "\n" +
     host + "\n" +
     port + "\n\n";
if (ext) {
     artifacts += ext;
artifacts += "\n";

Step 3: Create the hash using the HMAC function from CryptoJS.

var mac = CryptoJS.HmacSHA256(artifacts, variables.get("hawkSecret")).toString(CryptoJS.enc.Base64);

Step 4: Build the authorization header contents.

var header = "Hawk id=\"" + variables.get("hawkKey") + "\", ts=\"" + now +
"\", nonce=\"" + nonce + "\"";
if (ext) {
       header += ", ext=\""+ ext + "\"";
header += ", mac=\"" + mac + "\"";

Step 5: Set the header contents to a variable to use in the request configuration.

variables.set("hawkHeader", header);

Now that we have an authorization header, which changes according to the time, we can add it to our request, using the variable from the script.

Finally, we can run our test, check the response and add some assertions.

Initial scripts run before the request, so we cannot retrieve data from the request to use when creating the header. That is why we set all request parameters in the script (method, host, path, etc.). To avoid modifying the script for each different test configuration, we can create variables for all those settings which change between tests so that we can simply copy and paste the script when needed.

At Ease with API Testing & Auth Tools

Now that we've found an auth solution that works well for us, we're put even more at ease having a tool like Runscope in our arsenal to monitor and test the health all of those APIs. If you have any questions or comments about using Hawk protocol or testing APIs using authentication schemes, feel free to leave a comment below or tweet at me!

You can start testing APIs protected by an authentication protocol by signing up for Runscope for free. 

Categories: code samples, howto, product, monitoring, security, testing, community

API Tests Level Up: Automated Browser Test Steps with Ghost Inspector

By John Sheehan on .

Ghost Inspector is a cloud-based browser testing service. In addition to letting you quickly create UI tests for your websites without writing any code, Ghost Inspector is also integrated with Runscope so you can add UI test steps to your automated API tests.

Browser Test, Meet API Test

So, what are some use cases that intersect UI and API testing? For an API that integrates into web applications, there are a multitude. Combining Runscope with Ghost Inspector can help create end-to-end test coverage, from the backend to the front end, and all of the interactions in between.

A great example at this intersection is OAuth. Runscope is already capable of testing OAuth protected services by fetching access tokens programmatically; however, for 3-legged flows, where the end-user is directed to the OAuth provider for credentials and scope/permission, a browser testing tool is best suited because that authorization interface is designed for humans, not machines.

Walkthrough: Testing an OAuth API From End-to-End

By combining the capabilities of Runscope and Ghost Inspector, we are now able to perform comprehensive end-to-end application and API integration testing and monitoring. Lets go ahead and create a sample test that will perform the following tasks:

1.  Test the UI of an OAuth sign in process using Ghost Inspector

2.  Using the authorization code from the previous step, test the OAuth API by exchanging the auth code for an access token using Runscope

3.  Using the access token from the previous step, test the actual API by fetching protected user resource information using Runscope

We’re going to use the GitHub API for this example. Many of you are familiar with GitHub, and their API is well documented. 

Creating a Ghost Inspector Test

Ghost Inspector supports two ways for creating tests:

  • Manually - you provide the starting URL and define each test step by hand
  • Automatically - using a Chrome extension, a live web session is recorded and all test steps are automatically imported into Ghost Inspector

For this example, we’ll build our UI test manually, which is much easier than it sounds. Our test will cover signing in to a web application using GitHub credentials. We’ve stood up a simple web application that handles the OAuth sign in process. Lets begin with the steps for creating our Ghost Inspector UI test:

1.  Sign up for Ghost Inspector. Create a free account and sign in.

2.  Create a Test Suite, which is similar to a Runscope bucket.

3.  Create a new test. We’ll name the test, “BabyUnicornMagic Sign In with GitHub”, set the start URL to the front page of the web app, and use a blank test template, because we’ll be adding our own steps. 

4.  Add the steps. After the test is created, click on the Add Steps button. We’re going to add five steps to our Ghost Inspector test. These steps will take care of singing into the web app with our GitHub credentials.

Ghost Inspector uses CSS selectors to identify elements to interact with on a page. Recent versions of Chrome or Firefox can help to extract the unique CSS selector values from the DOM. Simply right-click on the element and choose “Inspect Element”. From within the inspector, right mouse-click again on the highlighted element, and choose “Copy CSS Path” in Chrome, or “Copy Unique Selector” in Firefox.

Each test step in the screenshot above contains a CSS selector. Step #1 clicks on a sign in link, which initiates the authorization flow. By the time we reach Step #2, the Ghost Inspector browser will be on the GitHub login page -- this step, and Step #3 instruct the Ghost Inspector to populate the login form with our GitHub username and password. Step #4 submits the form. Step #5, our last step, will click on the “Authorize” button that is displayed the first time our GitHub account approves this app. On subsequent test runs, GitHub will skip this step -- thus, we’ll make this step optional.

5.  Save the authorization code by stopping the Ghost Inspector test. We’re going to add a Network Filter to our test, instructing the Ghost Inspector browser not to follow the auth code redirect URL. This will preserve the code for later use.

At the bottom of the Ghost Inspector test editor, click the Add Filter button. We paste the app’s OAuth redirect URL in this field.

Run the Ghost Inspector Test, Watch the Video

And that completes our first Ghost Inspector UI test! After it's saved, we run the test by clicking Execute Test. The test will take 5 to 10 seconds to complete. Every Ghost Inspector test run is video recorded, revealing every test step, from text inputs, mouse clicks to page transitions. Below is an actual video of our test run:

Integrate Ghost Inspector With Runscope

Now we’re going to set up the Runscope Radar API test that will integrate the Ghost Inspector test we just finished creating above:

1.  Connect Ghost Inspector to Runscope with an Integration. Adding the Ghost Inspector integration is simple -- just click on Integrations from the Radar menu, then click Connect Ghost Inspector from the list of services.

2.  Create a new Radar test; add the Ghost Inspector test. Now that we've connected Ghost Inspector to Runscope, a new button is available in our Runscope Radar test editor: Add a Ghost Inspector Test. When we click that button, our Ghost Inspector test suites and respective tests appear. Lets select the new UI test we've just created.

3.  Extract the authorization token from the Ghost Inspector test. Recall earlier that in Ghost Inspector, we configured the test to stop right before the auth token exchange. We use JavaScript to extract the authorization code and store it in the variable {{auth_code}}.

4.  Test the OAuth API access token generator. The next step in our test plan will exchange the authorization token for an access token. Below we configure the request to the GitHub OAuth API. Both {{client_id}} and {{client_secret}} are defined as Initial Variables. We also extract the access token into the variable {{access_token}}.


5.  Test a protected resource API method. With the newly minted access token, we'll test the API method to fetch the email addresses associated with the authenticated GitHub user account. Lets also set up an assertion that validates the account's email address.


Runscope + Ghost Inspector = Better Software

That's just one example of how Runscope and Ghost Inspector can help to build more comprehensive tests that span the API as well as the application that integrates with the API. On their own, both Runscope and Ghost Inspector are very poweful tools for automatically monitoring, testing and debugging APIs and web applications, respectively. However, these tools combined provide you with a new dimension of holistic test coverage, resulting in better software for your users.

If you have any questions or need any help, drop us a line. Our support team is always ready to help.

Categories: announcements, code samples, howto, ghost inspector, testing

Build a Health Check API for Services Underneath Your API

By Neil Mansilla on .

A  running  cat, in honor of National Cat Day. That's Oskar (click photo for original). Full attribution below.

running cat, in honor of National Cat Day. That's Oskar (click photo for original). Full attribution below.

Monitoring your API's health and performance is something Runscope can do very well. But what about monitoring the underlying services that your API relies on? Databases, like MySQL and Redis, or DNS services on BIND or Mara -- they don't come with RESTful APIs, and in fact, they communicate over lower-level protocols like TCP and UDP. 

When you're relying on services without monitoring, the only time you realize those services are down is when your API (or the apps that rely on your API) go down. Wouldn't it be great if those services had health check API endpoints?

Creating separate health check API endpoints for all services

One approach is to build simple health check API endpoints — a method similar to a ping that reports whether the service is up or down.  Consider building a health check endpoint for every service your backend relies on and using Runscope for testing, reporting and triggering notifications. If you use the same client library or SDK in the health check app that you use for your API backend, you'll have the additional benefit of dependency testing that client library.

That being said, include as few dependencies as possible, isolating the health check app to only what's necessary to connect to the service. I mean, you wouldn’t want a MySQL test to fail because Redis was down. The health check endpoint should do one thing, and one thing only — check that a single service is up and running. If so, return 200. If not, return 503.

MySQL service check example

Below is a pint-sized PHP sample app to test if a MySQL server is up. This app has no library dependencies. It's using PHP's integrated mysqli library. This app can be parked on any web server that can execute/interpret PHP files (e.g. Apache, Nginx, etc.). Ideally, it would be mapped to an endpoint route specifically for health check resources -- but in this example, I've just placed it in the home directory of a vanilla Apache instance. I have MySQL running on another box located on the local network ( and bound to the default port (3306).

  $config = [
     "server"     => “",
     "username"   => "runscope",
     "password"   => "testpass"
  header('Content-type: application/json');

  $link = new mysqli($config['server'],$config['username'],$config['password']);

  if ($link->connect_errno) {
    echo('{ "status": "Unable to connect" }');
  else {
    echo('{ "status": "Connection successful." }');

Ping! Test the endpoint with Runscope Radar Agent

Since my MySQL and Apache servers are behind the firewall, I'll be testing my new health check endpoint with Runscope Radar Agent. This agent will perform tests on the local network and report the results back to Runscope. The only requirement when using the remote Radar Agent is that the computer can make HTTPS requests. Read more about (and download) the Runscope Radar Agent here

After I ran the agent on my local machine (took only a minute to set up), it appeared as a testing location on my Runscope Radar test editor. The location titled Internal-SFO-Office in the screenshot below is my laptop that's running the Runscope Radar Agent.

Runscope Radar Test Editor view. After installing the Runscope Radar Agent on my local machine, it pops up in the list of testing locations.

Runscope Radar Test Editor view. After installing the Runscope Radar Agent on my local machine, it pops up in the list of testing locations.

Next, I clicked Run Now. The MySQL health check test ran from my local Radar Agent. Below were the results from Traffic Inspector.

Setting up notifications

Voilà! The next step would be adding notifications, so that if there is a failure, the correct individual or team could receive email notifications. Additionally, Runscope supports webhooks, so that after tests run, succeed, or fail, Runscope can POST the test result data to any URL. Lastly, if you use PagerDuty, HipChat, Slack, Zapier, etc., Runscope smoothly integrates with all of those services. Read more about notifications here.

Should these be public or private?

You might ask yourself whether these endpoints should be made public or private. The short answer is private. By private, I mean that these should be treated as internal to your company, and not shared with partners or third party consumers of your API. Why? Because the information is too granular and could reveal details about your infrastructure that shouldn't be made public. However, if you were to wrap up all of these micro health checks together, and return a more abstract status response, that would be appropriate as a publicly accessible endpoint.

More micro sample health check apps on GitHub

In addition to the PHP+MySQL sample above, we've built a couple of other examples: Python+Redis, .Net/C#+MS-SQL and Perl+DNS. You're more than welcome to contribute your own. The GitHub repo is located here.

If you need some tips on how to get these health check apps working, or help with setting up Runscope Radar tests against them, our support team is standing by, happy to help.

Cat photo attribution:  LInk to original photo here (Flickr user Tambako). Used under Creative Common license.

Categories: code samples, howto, apis, testing, monitoring

How to Auto-Generate Runscope-powered SDKs for RESTful APIs

By Neil Mansilla on .

The following is a guest post from Adeel Ali from APIMATIC, a company that offers an automatic SDK generator for REST APIs. APIMATIC can import schemas/descriptions of RESTful APIs (WADL, Swagger, I/O Docs, API BluePrint, Google Discovery) and generate SDKs (Android, Windows, iOS, Java, PHP, Python, Ruby, Node.js). Adeel shares an easy to follow guide to building an SDK with APIMATIC and integrating Runscope.


Toward Better SDKs - With APIMatic

SDKs or client libraries are an argumentative topic in the API ecosystem. They are rightly criticized being expensive to build as well as painful to update and maintain. On the other hand, they are considered as crucial, which is evident from the fact that almost all of the top APIs come with SDKs. A closer look at the SDK criticism reveals that the actual pain-point lies in the current development lifecycle of SDKs, and it’s no brainer that if SDK development can be made efficient then it will be a win-win for everyone.

A solution to the above pain-point is replacing the problematic manual development of SDKs by automatic SDK generation. Auto-generated SDKs are cost-effective, easy to update, and offer huge reduction in work-load. is an automatic SDK generator, based in the land of Hobbits with the exact vision of better SDKs. APIMatic was founded in May 2014 by me and Zeeshan, and currently is in public beta phase, that means you can get SDKs for your API in 7 languages for FREE. More about APIMatic can be read in a review by Kin Lane. 

APIMatic & Runscope – A Natural Alliance

We met John Sheehan for the first time last week at APIWorld 2014, where he hinted about embedding Runscope’s API monitoring capabilities in APIMatic SDKs. The idea looked very intuitive because APIs are mostly accessed via SDKs, and by using Runscope powered SDKs, API providers could inspect/debug all of the request and response data traveling back and forth from the clients using their SDKs. The amount of convenience could be assessed by the fact that setting a single flag would automatically enable API debugging capability in SDKs in 7 languages (to date), while no settings would be required at the client side.

The idea has now become a reality, and in the remaining of the post I am going to show how you can generate Runscope powered SDKs.

Runscope Powered SDKs – Step by Step

Step 1
Create a Runscope bucket, and copy the bucket key. 

Step 2
Specify an API description at APIMatic. The two ways to do it are: importing an API description, or describing an API from scratch. I am describing Vimeo’s public API from scratch here, and will only be specifying two end-points to keep the demo simple. 


Step 3
Go to CodeGen Settings and enter the obtained Bucket Key from Runscope. Save the settings.


Step 4
Once the API is defined, go back to the API list, and click the validate button to validate the API description.

Step 5
If all is well, you will be presented with icons of different languages to generate SDKs. Choose your favourite language, and click the icon to download the SDK in a ZIP file.

Step 6
Unzip the contents of the downloaded SDK and open the Configuration file to provide your credentials to access the API. The contents of a generated C# Portable Class Library can be seen in the following snippet. It can be noticed in the Configuartion.cs file that the BaseUri has been transformed to the Runscope-desired format. If you are not using an interpreted language, you’ll now need to compile the SDK to get a compiled binary. There is a ReadMe file in the root folder file if you need any assistance.


Step 7
Now you have a (compiled) library in your hands which you can reference in an application. Here I am referencing and using the .dll file obtained from the previous step in a simple application. This application searches for videos on Vimeo against a provided keyword, and shows the total number of videos found as the result. The code to access the API functions via SDK can be seen in the snippet below.

Step 8
Go back to the Runscope portal, and select your bucket from the top left corner. You can watch all of the API traffic initiated by your app. The summary of the API calls originated by my application can be seen in the following snippet. Click on any particular call to see a detailed record, compare requests or responses, or play around with the other available options at a very friendly portal.


That’s it!

Congratulations on generating your first Runscope powered SDK. I love hearing feedback, so please comment on the post, or send me an email. If you need any assistance, don’t hesitate to ask. 

Categories: code samples, howto, apis

Everything is going to be 200 OK®