Runscope API Monitoring    Learn More →

Posts filtered by category: ghost inspector

Democratizing QA: How Automated Testing Tools Empower Teams

By Patrick McKenna on .

We’re excited to have Patrick McKenna, Global Head of Product Engineering at Kurtosys, discuss the evolution of QA with the Runscope community.

Sometimes I wonder how quality assurance (QA) got this bad. Humanity's engineering achievements are nothing short of mind blowing. Buildings that reach the stars, tunnels bored under cities, bridges that span seas. But when it comes to software that doesn't break, quite frankly, we suck.

If you agree with me, the chances are you feel the situation is not getting any better. If you're an engineer, you probably feel like you don't test your software well enough. If you are a product manager, you likely feel like in the time it takes for you to explain how to test your software, you could have just done it yourself. If you run QA, you probably feel that people don't understand what you do or really value it, but gosh, is it hard! And as a CEO, you probably feel like your team doesn’t do enough QA because the last release had an embarrassing bug that showed up during your VC pitch.

If any of that rings true, then welcome to the club. The bottom line: QA is hard, and it is mostly because, up until now, much of the tooling out there has sucked. However, things changed drastically for us when we began relying on both API testing and UI testing solutions that are so easy to use, anyone from a developer to a business person can take them for a spin.

QA-ing Our QA Processes

Not long ago, the best QA solution was to throw people at the problem; pay an offshore outfit bags of cash to write a custom Selenium framework for your app, which unfortunately ceases to be useful the moment you discontinue the contract because, guess what? Maintaining 1,100 Selenium tests is impossible for anyone but a gang of offshore QA guys.

Two years ago, we replaced our monolith system with a microservices stack. We got into trouble pretty fast with integration issues—services at the core would change and everything else would break because of response differences and so forth. The unit tests on the core would pass, but those tests didn't cover all the ways that other services were interacting with them. As a result, sometimes the platform remained broken until someone manually tested it or another service was rebuilt. Sometimes defects went undetected for longer than you would typically like them to, making pinpointing those issues time-consuming.

We realized pretty quickly that the best people to contribute integration tests for a service are the people who depend on them—which isn’t necessarily the people who historically write the tests. We also understood that integration tests need to run constantly, not just at build time in the dev environment, but also at runtime in prod. As the platform grew and was used in ways we didn’t anticipate, defects would appear. Our users became the one mutagenic factor that the build server couldn't truly account for.

Decentralizing QA with Runscope and Ghost Inspector

This notion of democratizing the test capability started to germinate with us. We wanted to empower anyone in our company to write tests, especially the people on our team who maybe didn’t know how to code, but worked very closely with our clients. We started using Selenium and Postman, but the learning curve was too great and they lacked an easy framework for continually running and contributing to the corpus of services we have in our stack.

Once we signed up for Runscope for API testing and Ghost Inspector for browser and UI testing, they quickly bridged this gap for us. At the danger of this sounding liking a puff piece, I have to say that they are incredibly usable pieces of software that let you write functional tests and integration tests for your web app in an absolutely delightful way. I honestly love their software.

However, the victory for us though is how both Runscope and Ghost Inspector have changed the face of QA in our company. We are beginning to grow what I believe is a QA culture, in which people's feelings of dread and malaise around QA are now replaced with feelings of empowerment—even our chairman writes Ghost Inspector tests now!

Ghost Inspector is instrumental here because it allows us to observe the exact experience that a user would have visiting a site or app because of its video capability. Knowing something failed is one thing, but having a recorded video of the failure case, plus browser logs, is quite another. We also use Runscope heavily to monitor our APIs because it gives us probes that we couldn’t easily achieve in any other way.

Cutting through Complexity

When I think about what this all means to me, I'm reminded of something Kent Beck once said, something along the lines of, “Writing software without tests makes me anxious ... and I don't think software developers deserve to feel anxious about their jobs." When you boil down what we do as technologists, we try to make cool stuff that makes people’s lives better and absolutely, positively, never ever breaks. Trouble is, sometimes that last part is a whole lot more consuming than the first, and that's a bummer.

So I guess I'm here to report that QA might be easier than you think nowadays. With tools like Runscope and Ghost Inspector, you can all do it. While you are on the couch, on the way to work, in the shower (well, maybe not in the shower).

Categories: testing, microservices, customers, ghost inspector


Integrate Runscope and Ghost Inspector Tests with AWS CodePipeline

By Ryan Park on .

Continuous Integration (CI) and Continuous Deployment (CD) are software development practices that can help to detect problems early and improve productivity. In a typical CI/CD workflow, each time a developer checks in code, the project is automatically built, tested and deployed. If your project relies on APIs, or is an API itself, integrating functional API tests as part of your CI/CD workflow can insure that your web services are fully functional with your latest code push. Likewise, if your project is a web application, including browser tests as a step in your CI/CD pipeline can help to detect UI problems introduced with your last commit.

AWS CI/CD Solution: CodePipeline

We’re excited to announce that we’re integrated into the new AWS CodePipeline product which was made available to customers today at AWS Summit NYC. AWS CodePipeline is a continuous delivery service that enables you to model, visualize and automate the steps required to release your software. Customers of CodePipeline can include Runscope API tests and Ghost Inspector browser tests in their continuous delivery pipelines to catch faults and regressions early in their development processes.

With CodePipeline, you model the full release process for building your code, deploying to pre-production environments, testing your application, and releasing it to production. CodePipeline then builds, tests, and deploys your code as modeled every time there is a code change. You can integrate partner tools and your own custom systems into any stage of the release process to form an end-to-end solution. CodePipeline enables you to rapidly deliver features and updates with high quality through the automation of your build, test, and release process.

Many of you with CI/CD workflows in place have already integrated Runscope tests into their pipeline, like uShip and SendGrid. For those users looking for a CI/CD solution, CodePipeline can streamline those processes even more with direct pipeline integration of your Runscope and Ghost inspector tests.

Easy Test Integration from the AWS Console

The CodePipeline visual configuration makes it easy to visualize your build, test and deploy flow. In the screenshot above, the pipeline has three steps. In the first step (Source), the source code is obtained from an AWS S3 bucket. In the second step (API Testing), a Runscope API test is initiated. In the last step (Browser/UI Testing), a Ghost Inspector browser test runs. Adding a Runscope or Ghost Inspector test to your pipeline is as simple as few clicks.

For more information about AWS CodePipeline, click here. If you have any other questions about Runscope or Ghost Inspector, or how to integrate your tests into your CI/CD workflow, contact our support team. Check out the full announcement for more information about the integration. 

Categories: announcements, ghost inspector, integrations, product, testing, monitoring


5 Post-Launch Steps to Make Your App Successful

By Neil Mansilla on .

This post originally appeared on the Build Blog by ThinkApps

So you’ve designed, built and launched your app. Before you pat yourself on the back and move on to the next project, your work isn’t quite done. In order for your app to gain adoption, consistently delight users and be successful for the long haul, you need to put certain systems and workflows in place. Great apps are successful for many reasons—they evolve, exceed customer expectations and function reliably. Making your app great doesn’t have to be difficult, if you have the right tools. We’ve compiled a list of five tool-tips to help you—and your app—be awesome.

1. Tune in to your community channels

When your users encounter a problem or have a question, they won’t necessarily reach out to you directly. Users often post issues to Twitter and Stack Overflow when they’re looking for answers, or sometimes vent about unaddressed issues on a blog. We use tools like IFTTT, Zapier and Google Alerts to monitor the web and social media for certain keywords related to our product. We pipe these results into Slack to get instant notification in the channel we use most. By taking the time to set up these systems outright, we have confidence that we’re in tune with our community and can stay on top of what’s being said about our product so we can engage accordingly.

2. Ask for feedback, not just ratings

Most people believe that users love their app. Do you? Have you ever asked? It can be difficult to engage your users without the proper tools, and feedback is key for building on success. For native apps (mobile and desktop), tools like Apptentive make it super easy to understand customer sentiment. All you have to do is drop in the SDK and insert a few lines of code, and you’ll be able to survey your users to find out how they really feel about your app in no time. Just make sure you make that information actionable by using it to help inform future iterations and updates.

3. Use notifications wisely

Notifications, be they mobile push notifications or emails, are a great way to remind users that they should be using your app. But it’s important not to spam your users with every event that you believe is valuable. So where do you draw the line?

For business applications that have a large number of notifications, provide a simple interface that allows your user to toggle each type of message on or off. For lighter weight apps, do research and experiments to find out which events mean the most to your users, and at what time they are most contextual or valuable. 

For mobile apps, platforms like Urban Airship and Parse make it very easy to send notifications to just a few or a large number of users. For sending email, use a trusted provider for high reliability and deliverability, such as SendGrid or MailChimp. These tools have analytics and recommendations baked in to help guide you in the right direction and leverage the data accordingly.

4. Test your UI

Bad experiences can happen to any website for various reasons—a server was unavailable, a network connection was lost or maybe someone really did forget to bundle in a library or file on the last code commit. A new generation of tools make web UI testing a lot easier for non-testing professionals, like Ghost Inspector. You don’t have to write any scripts or a single line of code.

The Ghost Inspector browser extension can record clicks, form submissions and more, allowing you to create tests simply by using your own web app in the browser. Once a test is created, you can run it continuously on a set interval, so that if your web app isn’t rendering and operating exactly as you expected, you’ll receive notifications via email, Slack or PagerDuty. Additionally, Ghost Inspector automatically records video and screenshot of every test run, so you can see exactly what your users experience, have a record of it and debug the problem.

5. Monitor the APIs on which your app relies

More and more, apps rely on APIs to integrate data and functionality into one unified interface. Whether the APIs your apps are consuming are home-grown or from another service provider, it’s important to know that all of them are not only up, but also operating as expected. Runscope keeps you informed of any API issues you deem important, including issues involving uptime, latency and correctness (that the API is returning the right data).

Like Ghost Inspector, you don’t need to know any code to use Runscope. API tests run in the cloud from up to 12 locations around the world, so you know how your API is performing everywhere your users are. With integrations into some of the most popular channels, including Slack, PagerDuty, HipChat and more, you and your team can get real-time notifications via the tools you use most. Once you’re ready to dive into an issue, Runscope provides unparalleled visibility into the traffic running through your API, providing the entire response body data and ability to share test results with partners or teammates in one click.  

Coming up with a great idea and building your app is just the beginning. To make and keep your app successful, you must always be conscious about how you can improve upon it—and you can only do that if you’re measuring and monitoring key metrics. Luckily, there are many tools available for app developers like you to monitor customer sentiment all the way to monitoring APIs. Putting those monitoring systems in place allows you to keep a pulse on your app to keep it successful in the future. Many of them offer free trials to get started—sign up for Ghost Inspector and Runscope today to ensure that your app's success lasts. Or sign up for both and save!

Categories: api ecosystem, apis, ghost inspector, community, howto


API Tests Level Up: Automated Browser Test Steps with Ghost Inspector

By John Sheehan on .

Ghost Inspector is a cloud-based browser testing service. In addition to letting you quickly create UI tests for your websites without writing any code, Ghost Inspector is also integrated with Runscope so you can add UI test steps to your automated API tests.

Browser Test, Meet API Test

So, what are some use cases that intersect UI and API testing? For an API that integrates into web applications, there are a multitude. Combining Runscope with Ghost Inspector can help create end-to-end test coverage, from the backend to the front end, and all of the interactions in between.

A great example at this intersection is OAuth. Runscope is already capable of testing OAuth protected services by fetching access tokens programmatically; however, for 3-legged flows, where the end-user is directed to the OAuth provider for credentials and scope/permission, a browser testing tool is best suited because that authorization interface is designed for humans, not machines.

Walkthrough: Testing an OAuth API From End-to-End

By combining the capabilities of Runscope and Ghost Inspector, we are now able to perform comprehensive end-to-end application and API integration testing and monitoring. Lets go ahead and create a sample test that will perform the following tasks:

1.  Test the UI of an OAuth sign in process using Ghost Inspector

2.  Using the authorization code from the previous step, test the OAuth API by exchanging the auth code for an access token using Runscope

3.  Using the access token from the previous step, test the actual API by fetching protected user resource information using Runscope

We’re going to use the GitHub API for this example. Many of you are familiar with GitHub, and their API is well documented. 

Creating a Ghost Inspector Test

Ghost Inspector supports two ways for creating tests:

  • Manually - you provide the starting URL and define each test step by hand
  • Automatically - using a Chrome extension, a live web session is recorded and all test steps are automatically imported into Ghost Inspector

For this example, we’ll build our UI test manually, which is much easier than it sounds. Our test will cover signing in to a web application using GitHub credentials. We’ve stood up a simple web application that handles the OAuth sign in process. Lets begin with the steps for creating our Ghost Inspector UI test:

1.  Sign up for Ghost Inspector. Create a free account and sign in.

2.  Create a Test Suite, which is similar to a Runscope bucket.

3.  Create a new test. We’ll name the test, “BabyUnicornMagic Sign In with GitHub”, set the start URL to the front page of the web app, and use a blank test template, because we’ll be adding our own steps. 

4.  Add the steps. After the test is created, click on the Add Steps button. We’re going to add five steps to our Ghost Inspector test. These steps will take care of singing into the web app with our GitHub credentials.

Ghost Inspector uses CSS selectors to identify elements to interact with on a page. Recent versions of Chrome or Firefox can help to extract the unique CSS selector values from the DOM. Simply right-click on the element and choose “Inspect Element”. From within the inspector, right mouse-click again on the highlighted element, and choose “Copy CSS Path” in Chrome, or “Copy Unique Selector” in Firefox.

Each test step in the screenshot above contains a CSS selector. Step #1 clicks on a sign in link, which initiates the authorization flow. By the time we reach Step #2, the Ghost Inspector browser will be on the GitHub login page -- this step, and Step #3 instruct the Ghost Inspector to populate the login form with our GitHub username and password. Step #4 submits the form. Step #5, our last step, will click on the “Authorize” button that is displayed the first time our GitHub account approves this app. On subsequent test runs, GitHub will skip this step -- thus, we’ll make this step optional.

5.  Save the authorization code by stopping the Ghost Inspector test. We’re going to add a Network Filter to our test, instructing the Ghost Inspector browser not to follow the auth code redirect URL. This will preserve the code for later use.

At the bottom of the Ghost Inspector test editor, click the Add Filter button. We paste the app’s OAuth redirect URL in this field.

Run the Ghost Inspector Test, Watch the Video

And that completes our first Ghost Inspector UI test! After it's saved, we run the test by clicking Execute Test. The test will take 5 to 10 seconds to complete. Every Ghost Inspector test run is video recorded, revealing every test step, from text inputs, mouse clicks to page transitions. Below is an actual video of our test run:

Integrate Ghost Inspector With Runscope

Now we’re going to set up the Runscope Radar API test that will integrate the Ghost Inspector test we just finished creating above:

1.  Connect Ghost Inspector to Runscope with an Integration. Adding the Ghost Inspector integration is simple -- just click on Integrations from the Radar menu, then click Connect Ghost Inspector from the list of services.

2.  Create a new Radar test; add the Ghost Inspector test. Now that we've connected Ghost Inspector to Runscope, a new button is available in our Runscope Radar test editor: Add a Ghost Inspector Test. When we click that button, our Ghost Inspector test suites and respective tests appear. Lets select the new UI test we've just created.

3.  Extract the authorization token from the Ghost Inspector test. Recall earlier that in Ghost Inspector, we configured the test to stop right before the auth token exchange. We use JavaScript to extract the authorization code and store it in the variable {{auth_code}}.

4.  Test the OAuth API access token generator. The next step in our test plan will exchange the authorization token for an access token. Below we configure the request to the GitHub OAuth API. Both {{client_id}} and {{client_secret}} are defined as Initial Variables. We also extract the access token into the variable {{access_token}}.

babyunicorn_access_token.png

5.  Test a protected resource API method. With the newly minted access token, we'll test the API method to fetch the email addresses associated with the authenticated GitHub user account. Lets also set up an assertion that validates the account's email address.

babyunicorn_testemail.png

Runscope + Ghost Inspector = Better Software

That's just one example of how Runscope and Ghost Inspector can help to build more comprehensive tests that span the API as well as the application that integrates with the API. On their own, both Runscope and Ghost Inspector are very poweful tools for automatically monitoring, testing and debugging APIs and web applications, respectively. However, these tools combined provide you with a new dimension of holistic test coverage, resulting in better software for your users.

If you have any questions or need any help, drop us a line. Our support team is always ready to help.



Categories: announcements, code samples, howto, ghost inspector, testing


Everything is going to be 200 OK®