Runscope API Monitoring    Learn More →

Tutorial: Continuous Integration with CircleCI and Runscope API Tests

By Heitor Tashiro Sergent on .

CircleCI is a popular Continuous Integration and Delivery solution (used by Facebook, Kickstarter, Spotify), and integrating it with Runscope Trigger URL tests so you can run your Runscope API tests takes only a few minutes!
Runscope can be used by itself to monitor and test APIs, but it can also be used as part of your CI workflow. We have a sample Python script in our GitHub that can be used to trigger a set of tests in your Runscope account, and change the build status based on their results.
In this tutorial, we're going to show you how to use that script with CircleCI. We'll cover how to:

  1. Generate a Runscope API access token
  2. Get your tests Trigger URL
  3. Set up your environment variables in CircleCI
  4. Run the Python script as part of your test commands

Let's get started!

Getting Your Runscope Variables

Trigger URL

When running this script from the command line you can pass one parameter to it, which is a Runscope Trigger URL. If you want to run a single test, you can find its trigger URL under your environment settings:

Runscope test environment settings with the Trigger URL option selected

If you want to run all the tests in a bucket, you can find a separate trigger URL in your bucket settings:

Runscope dashboard highlighting the Bucket Settings link
Runscope bucket settings page, highlighting Trigger URL section

Make sure to copy the Trigger URL you wish to use, so we can use it with our script in the next steps.

Generating Your Runscope API Key

We need a Runscope personal access token to interact with the Runscope API and retrieve the results from our test run.
To get your access token, head over to your account's application tab, and click on Create Application. In the next screen, give your application a name, website URL, and callback URL. You can use dummy URLs if you're just using this app for your CI integration (e.g.

Runscope applications page displaying the Create Application page with dummy values

Click on Create Application to finish the process. Then, scroll down to the bottom of your new application page and copy the personal access token value. We're going to use that in our next step:

Runscope application page detail, with the Personal Access Token highlighted

Setting up the Runscope Python Script

You can find the sample script in our GitHub:

The two files we're interested are requirements.txt and For this tutorial, I'm just going to work with the raw links from our GitHub repository.
If you're integrating this into your project, I highly recommend either forking it to your own repository, or adding these files to a separate folder. That way, you can prevent your build from breaking in case there's an update to the repository.

Integrating with CircleCI

In your CircleCI account, select the Build tab on the left-hand side menu, and click on the gear icon next to the project you want to integrate with Runscope to open its settings:

CircleCI Builds menu, highlighting the gear icon next to a project's under the project list

On the left-hand side Settings menu, click on Environment Variables under Build Settings, then click on Add Variable. We only need to add one environment variable here named RUNSCOPE_ACCESS_TOKEN. Paste the access token that you copied in our previous step under Value, and click on Add Variable:

CircleCI project settings page, showing the Environment Variables page with the Runscope access token variable set

Now, let's go to Dependency Commands under the Test Commands menu.

The CircleCI environment already comes with python and pip pre-installed. The first thing we need to do is make sure the necessary packages for the script are installed. Add the following command to your Pre-dependency commands window:

pip install -r

Note: Remember to change the requirements URL above to your fork or local file.

Next, I'm going to add another command just below it to download our file (you can skip this step if you copied the file to your project):

CircleCI project settings, showing the Dependency Commands page with the Pre-dependency commands window filled with the Runscope commands

For the final step, let's head to Test Commands under the Test Commands menu. In the Post-test commands window, we can run our script. It takes one parameter, which is the Trigger URL you copied at the beginning of this tutorial. So we can just run the command as:


Note: Remember to use the correct directory for if you copied it to a folder inside your project.

CircleCI project settings, showing the Test Commands page with the Post-test commands text window including the command to run the Runscope python script

Continuous Integration Complete

In your next build runs, you should be able to see an extra step running the Python script, and hopefully returning a green checkmark ✅:

CircleCI build detail page, showing the successful output of the Runscope Python script where all the API test runs have passed

With Runscope integrated into your CI process, we hope that you have even more confidence in your builds and that your APIs will be 200 OK.

We used CircleCI in this tutorial, but these instructions should also apply to other CI providers as well (check our Codeship tutorial and our Jenkins plugin). If you need any help with those, please reach out to our awesome support team.

Are you integrating Runscope in your build process? If so, we'd love to hear how you're doing it, just reach out to us via email or on Twitter.

Categories: code samples, tutorial, testing, integrations

Everything is going to be 200 OK®