Solve API problems fast with Runscope. Learn More →

This week in APIs: What we can learn from the Yo API

By Darrel Miller on .

We've teamed up with James Higginbotham of LaunchAny and Keith Casey of Casey Software to bring you their hand-curated weekly newsletter for API developers. You can subscribe to the API Developer Weekly email newsletter here

Also, be sure to check out James's and Keith's new book A Practical Approach to API Design on Leanpub. On to the links!

Hot Topics

What we can learn from the Yo API

"We started Yo’ed, a hub entirely written with Golang, which triggers actions when someone Yo’ed an account. While playing around with the API, we also wondered why we were attracted to it in the first place. Having a closer look, we discovered that it has great lessons to teach us!"

API Changelog Aims to Reduce Risk to API Consumer Business Models

"API Changelog provides up-to-the-minute tracking of APIs to ensure that API developer-consumers are alerted to any upgrades or changes to endpoints. Changes to an API can create business risks as the end customer’s experience may be affected if an API being used in an application or aggregation product suddenly becomes inaccessible."

Microsoft Tackles Internet-of-Things With New Data Stream Processing Service

"Microsoft added Event Hubs to its portfolio of messaging services called the Azure Service Bus. Event Hubs  is similar to Service Bus Queues and Topics in that it supports first-in-first-out messaging, competing consumer scenarios, and data retention policies. Its client-side cursor, partitioned consumer support, and significant time-based retention options are unique with the Microsoft portfolio."

An API Definition As The Truth In The API Contract

How the API definition, terms of service, and other aspects of an API combine to create a complete API contract and how a machine readable version of this contract is important to provide assurances to API consumers. 

Slideshare: Reusable APIs

Mike Amundsen's recent presentation on how to build APIs that are reusable and more machine readable. 

Facebook fixes huge API security vulnerability

"Security researcher Stephen Sclafani discovered a massive API-based security vulnerability at Facebook. Facebook's security mistake is one from which all API developers can learn. Most Facebook users were vulnerable to spying, spamming and hacking because of lax security on an old API."

The Business of APIs

The Benefits Of APIs For Transport And Logistics

A look at the recent API momentum in the transportation and logistics sector.

The Real World Cup Winner: APIs

"What was really interesting was how this was truly a, “World Cup powered by APIs.” World Cup organizer FIFA has stated that more than a billion fans worldwide accessed information about the tournament through its digital platforms."

To fight Google and Foursquare, Yelp frees its data

"Get ready to see Yelp reviews all over the web. The local business discovery and review site recently announced a revamped API, which increases the number of calls a third-party app can make to Yelp’s data to 25,000 calls per day, compared to an earlier 100 call limit."

API Development

Hypermedia as the engine of application state, the client-server dance

"We are currently seeing a significant amount of discussion about building hypermedia APIs.  However, the server side only plays part of the role in a hypermedia driven system.  To take full advantage of the benefits of hypermedia, the client must allow the server to take the lead and drive the state of the client.  As I like to say, it takes two to Tango."

Video: Adrian Cockcroft: Migrating to Microservices

A video presentation from Adrian Cockcroft on the strategies, patterns and pathways to perform a gradual migration from monolithic applications towards cloud-based REST microservices.

Create and Host your First Web API with APISpark

A look at the APISpark platform and how to build your first web API in 15 minutes or less.

CORS Makes Your API Portable And Remix-able

"CORS being enabled, is the difference between an API being portable, and remix-able, and it being locked down to its original developer portal."

Cloud Watch (from Cloud Developer Weekly Newsletter)

Amazon's Cloud Is One of the Fastest-Growing Software Businesses in History

"Five billion dollars. That’s how much Amazon.com (AMZN) will rake in from its cloud computing business this year, according to a new estimate from Pacific Crest Securities. If true, it’s an incredible figure. It would mean that Amazon’s cloud revenue shot up 58 percent in a single year, from $3.1 billion in 2013."

Rackspace, battling Amazon, tiers cloud services to show off its strengths

"Facing off against cloud giants AWS, Microsoft and Google, Rackspace regroups in a way it hopes will show off its service and support advantages."

10 Docker Tips and Tricks

"As a Solutions Engineer at Docker Inc., I’ve been able to accumulate all sorts of good Docker tips and tricks.  The sheer quantity of information available in the community is pretty overwhelming, and there are a lot of good tips and tricks that can make your workflow easier (or provide a little fun) which you could easily miss."

When do bash init scripts run?

A nice diagram posted to Twitter show when various bash init scripts run. 

Recommended Security Setup for AWS

"While we’ve never heard of a security breach inside of AWS itself, there have been numerous incidents of Amazon accounts getting illegally accessed and damaged. When thinking about application and hosting security, it’s important to frame the questions in the right light. Specifically, always use the word when."

Why a Media Giant Sold Its Data Center and Headed to the Cloud

"Condé Nast's CTO says the media company is no longer in the business of maintaining a data center. It has gone 'all-in' on the cloud with Amazon Web Services."

Want to share something?

As always, if you want to chat, share a link, or make a suggestion, feel free to drop us a quick note using Twitter (@launchany and @caseysoftware) or by emailing us at: james@launchany.com.

Categories:

Golang at Runscope

By Frank Stratton on .

Wake me up before you go Go...
— Wham!

When we started Runscope, we used Python to power all our services.  Python’s flexibility, utility, and maturity has allowed us to move quickly from idea to prototype to production. Consequently, we’ve invested in building a lot of tools and processes for developing and deploying Python services within our infrastructure.

With our recent announcement of Runscope Enterprise, the story of Runscope expands beyond just deploying code to servers that we manage to having code run in a variety of different customer environments.  This was a perfect time to step back and ask ourselves if Python was the best language for the job.

 

Distribution and Python

Runscope Passageway is a client application  that provides HTTP tunneling to your local machine from the public internet. These HTTP requests are automatically fed into your Runscope stream so you can easily debug a local web service the same way you would a public API. During the initial beta period, we distributed the client via the Python package manager pip.

In theory, this seems like a perfectly reasonable, single line, method for downloading a package.  In practice, it's much more complicated.   For starters, not everyone is a Python developer and not everyone is running a *nix environment.  There are a lot of prerequisites that you need to get to the point where you can even attempt to install from pip.  Once you’ve cleared those hurdles and have pip installed, you’re not done yet.  For instance, one of the most common issues we’ve seen on Windows platforms was gevent/greenlet support.  Not to mention that the most recent version of pip no longer supports dependency_links by default.   Python packaging is great for Python libraries, but for general purpose utilities it requires you to write a lot of supporting scripts to check and fix the environment you’re going to be running in.

Our next foray into Python client distribution was an early alpha of an agent for Runscope Radar that you can run from within your own infrastructure.  For this project we decided to look into various Python binary generation frameworks. We tried py2exe, cx_Freeze, and PyInstaller.  Each of these fell short when trying to build binaries for multiple platforms.  While it was possible to generate binaries, the configuration and build process was time consuming and poorly documented.  Furthermore, it required each build to happen on target platform.  For us, maintaining three separate build servers running three different operating systems was less than ideal.  Even then, many of our early test builds failed because of one or more modules not building correctly on the target platform or some dynamic library not being present.

There are ways around a lot of the issues we saw, but it largely required rewriting, rearchitecting, and simplifying the code and its dependencies.  If we were going to spend a large amount of effort starting from scratch, we had the opportunity to try another approach.

 

Adventures in Go

One of the requirements for Runscope Enterprise agents is a truly effortless install, no matter what platform the agent is running on.  Go’s ability to easily cross-compile to multiple architectures made it a top candidate.  And based on the experiences and recommendations of those who’ve worked with go in these situations, we decided to try it out.

First Impressions

Go is fun.  Go has an interesting feature set that makes the language feel both familiar and new at the same time.  Using goroutines and channels for the first time makes you go “wow, that was neat!”

Learning by example is by far the easiest way to learn go.  While godoc.org has a nice complete reference for go packages, as a beginner the most useful documentation are the short Example usages in the godocs.  The Go Blog (http://blog.golang.org/) is also filled with a wide variety of detailed examples and solutions.  Seeing how functions behave in context helps cement a lot of the new language idioms.

The concepts of go are simple enough to understand (especially coming from a C/C++/Java/Python background).   Most importantly, we were able to be productive in the first week without any issue.  After getting our feet wet, we were able to learn and use Go-specific features as we went along.

Essential Tools & Resources

Go Fmt - Auto formatting of go code makes a bigger difference than you’d imagine.  Having a language that can self enforce a style guide lets you write code your way but be readable for everyone.  When coupled with the go vim/emacs plugins, you spend less time organizing code and more time getting stuff done.

Go Vet - "Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string. Vet uses heuristics that do not guarantee all reports are genuine problems, but it can find errors not caught by the compilers."

The Go Blog  - Filled with great detailed posts about individual langauge features or patterns.  Usually when we were stuck on something a google search ended up on the this blog.  It’s nice having an authoritative source for information when you’re learning the tricks of a new language.

Go By Example  - A great reference that gets stright to the point.  Go by Example gives you complete working examples to help you understand Go’s finer points.  Each example comes complete with imports, a main function, and the command to run, and example output.  By giving you the full picture, it helps cement your understanding of how thing actually work, rather than giving you snippets of code out of context.

Build & Deployment

Our builds are done by a Jenkins server.  We use the project’s workspace as our GOPATH to isolate each go project from each other.  Each build runs a go get -u and a go build task.  For our cross-compiled libraries we have the build task repeat the go build for each target OS + Architecture combination.   Once a successful build happens, we publish the built binaries into an AWS S3 bucket.  These binaries are downloaded to our cluster when the appropriate service gets deployed.

Shortcomings

With any young language, there will be some rough edges.  We did encounter, from time to time, some small issues as we explored Go.  Here’s a brief overview of those issues:

Repositories Everywhere

Most of the code we write as internal libraries are in private GitHub repositories.  Unfortunately ‘go get’  doesn’t clone a private repo using your ssh keys.  To bootstrap our build system, we had to manually clone those dependencies into our workspace.

hg, bzr, git, …  Public Go libraries are hosted all over the place, and each repository has it’s own SCM dependency.  If your package import is github.com then go get will use git.  If it’s code.google.com you need to install Mercurial.  If you import from launchpad.net, you need Bazaar.   And it’s not just your direct dependencies, if one of your dependencies has it’s own dependency on launchpad, then you might need to install a new source control tool.  There are a lot more moving parts to Go under the hood than it seems at first glance.

Things that do not cross-compile.

There are a handful of functions that do not play well when you cross-compile and that panic at runtime.  For example, user.Current() cannot be called in a cross-compiled binary (aka CGO_ENABLED=0), and it seems like they’re going to stay that way.  These functions that aren’t available when you cross-compile aren’t documented well and end up dying at runtime instead of compile time, it’s also unclear if a library or dependency of a library will have issues until you run it and find out the hard way.

Another shortcoming of cross-compiling is that you cannot use system CA certs for SSL connections.  You now have to accept a path to the cacert files from the user, or bundle your own.

Final Thoughts

Adding Go to our repertoire has enabled to do do things we couldn’t easily do in Python alone. It has allowed us to quickly build stable, high-performance systems which we can deploy to a variety of environments with ease. While still rough around the edges, with each new release we’ve seen some great improvements to the language.  When we started our Go experiment, our focus was on easy to distribute client binaries.  Since then, we’ve found more uses for Go in our internal services, and our use continues to expand as the language matures and our overall go skill level improves.  

If you’d like to work on high performance Go services at Runscope, check out our jobs page and drop us an email!

Categories:

Introducing Global API Monitoring with Runscope Radar Locations

By John Sheehan on .

Runscope Radar is a powerful tool for teams to test their APIs automatically on every commit, build or deploy. In addition to testing, scheduling your API tests to run as often as every minute gives you ongoing API monitoring as well. This unique combination of testing and monitoring has been popular with both developers and non-developers alike.

Today we're fulfilling one of the most popular feature requests by our customers monitoring their APIs with Runscope Radar: geographically-distributed tests.

Monitor Your APIs from Around the Globe

Runscope operates in eight service regions. Previously, these regions were available for Runscope URLs but starting today they are also available for Radar tests. With locations across the globe (US East/West/Midwest/Texas, Australia, Brazil, Ireland and Singapore) and across two service providers (AWS and Rackspace), your tests will give you a complete picture of how your API is performing.

Tests with multiple locations work just like any other test. Response time assertions are particularly useful when combined with locations. You can also use all of our popular 3rd-party service integrations like notifications with PagerDuty, HipChat, or Slack and analytics with Keen IO or New Relic Insights.

Monitor Internal or Private APIs Too!

Radar Locations aren't just limited to the service regions we offer. Runscope Enterprise customers can also use Locations to run their API tests behind the firewall. After installing a simple agent that runs on your infrastructure, the agent will appear in the list of locations in the test editor. To get started testing or monitoring your internal APIs, request access to Runscope Enterprise.

Available Now

Multiple locations can be enabled from any of your tests from the test editor. You can also enable locations for all tests within a bucket using Bucket-wide Settings.

If you haven't yet tried Radar for API testing or monitoring, be sure to sign up for your free 30-day trial and get started today.

 

Categories:

This week in APIs : 5 Ways an API is More Than An "API"

By Darrel Miller on .

We've teamed up with James Higginbotham of LaunchAny and Keith Casey of Casey Software to bring you their hand-curated weekly newsletter for API developers. You can subscribe to the API Developer Weekly email newsletter here

Also, be sure to check out James's and Keith's new book A Practical Approach to API Design on Leanpub. On to the links!

Hot Topics

5 Ways an API is More Than An "API"

"If we are trying to speak to that CEO or manager rather than the taxi driver, we could be underselling how transformative an API truly is for anyone thinking about how they can use APIs in business."

Awesome API User Experience Should Include The Operations Users

"Awesome API user experience should cover more than end users and developers.  It needs to cover operations and dev-ops.  Here are a few reasons why"

Making Your API a Valuable Product

"Much in the same way you do with your core products, you need to invest some time and energy into every stage of your API quality life cycle – from development and testing to post-production monitoring. Let’s focus some attention on how you can elevate the quality of your API to make it a valuable (and valued) asset in your organization."

Low Hanging Fruit For API Discovery In The Federal Government

"I looked through 77 of the developer areas for federal agencies, resulting in reviewing approximately 190 APIs. While the presentation of 95% of the federal government developer portals are crap, it makes me happy that about 120 of the 190 APIs (over 60%) are actually consumable web APIs, that didn't make me hold my nose and run out of the API area. "

The Business of APIs

Twilio Helps Nonprofits Enter the API Economy With Social Good Venture

"Twilio’s new social good venture, Twilio.org, is bringing API-enabled communications to nonprofits around the world."

APIs are coming. Welcome to the B2B Sharing Economy

Why APIs are lowering market transactions costs and are reshaping of the economy we know for a new generation of entrepreneurs.

Survey: CA Finds API Security, Usability Taking Center Stage

"As APIs capture the attention and imagination of more devs, a survey of API professionals finds security and usability now rank as top areas of focus. The survey, from CA Technologies’ Layer 7, polled approximately 180 API enterprise stakeholders, from front-line devs to IT architects."

The Business of APIs: What Product Managers Need to Plan For

"In today’s cloud computing world, many Product Managers will be faced with the decision of whether to open an API to their users.  Open APIs have many advantages, but they also bring significant business challenges.  In this post, I describe key benefits and challenges of open APIs, and why Product Managers should treat an open API as its own product instead of as a feature."

The 'Internet Of Things' Will Soon Be A Truly Huge Market, Dwarfing All Other Consumer Electronics Categories

There are already clear signs that the biggest tech companies — and even smaller players — are trying to get out front of the race to dominate the IOT. Google has acquired Nest. Apple has unveiled its HomeKit platform. Even Staples and Honeywell — not typically companies thought of as tech leaders — are putting out new IOT-related products.

API Development

Soundcloud API usability review

A look at the steps required to build an API usability review using the Soundcloud API. Submitted by @apiusabilitytst.

Say 'Hi' to mrJSON

A project that enables you to define your resources, launch a hosted API, and even generate test data to help teams quickly focus on the client side. 

Slideshare: OAuth you said

An introduction to OAuth, a brief history, what to avoid, and links to further reading.

Slideshare: Moving from API Design to Deployment

A look at one of our recent projects, how we go through the API design process, and how we used various cloud, tools, and strategies to easily support messaging and ease of deploying upgrades for our API.

Cloud Watch (from Cloud Developer Weekly Newsletter)

Are Docker Users Migrating to Ansible and Away from Puppet and Chef?

The Ansible automation technology is different from Chef and Puppet. It operates strictly over SSH and requires no server side daemon process.

Dockercon video: AWS Elastic Beanstalk and Docker

In this video, Evan Brown from AWS explains how to deploy your Containers via a Dockerfile, public repository, or private repository. Evan also highlights the best practices for security and secret management, logging, and scaling and monitoring your Docker Containers on Elastic Beanstalk.

Softlayer: The bare metal IaaS provider

Softlayer is an IBM company that provides IaaS. What makes them different from Amazon?

Red Hat Enterprise Linux Atomic Host expands to Google Compute Engine

Red Hat Enterprise Linux Atomic Host is now available as a technology preview on Google Compute Engine for customers participating in the Red Hat Enterprise Linux 7 Atomic special interest group (SIG). The inaugural SIG is focused on application containers and encompasses the technologies that are required to create, deploy and manage application containers.

Amazon Picks Mobile Development As Its New Product Category To Dominate

Amazon Web Services (AWS) seems to have the Midas touch. Seemingly everything the Seattle bookseller’s cloud infrastructure division introduces ends up being category-leading. That’s awesome for Amazon but pretty horrid for all those left in the wake of the juggernaut. The latest (and, admittedly, entirely natural) area for AWS to introduce product in is the burgeoning Mobile Backend as a Service (MBaaS) area.

Revisiting "What is DevOps"

If all companies are software companies, then all companies must learn to manage their online operations.

Want to share something?

As always, if you want to chat, share a link, or make a suggestion, feel free to drop us a quick note using Twitter (@launchany and @caseysoftware) or by emailing us at: james@launchany.com

Categories:

Everything is going to be 200 OK

Sign Up — Free!