Quality Assurance Operations, or QAOps, is one of the latest buzzwords in the software testing industry. A previous Sauce Labs blog post explained what QAOps means at a high level. In this post, we’ll dive deeper by explaining how to operationalize QAOps — or, in other words, how to implement QAOps in a real-world setting.
WHAT IS QAOPS?
Put simply, QAOps means integrating software testing into the CI/CD pipeline, rather than performing software tests on a one-off basis. The core idea behind QAOps is to increase direct collaboration between developers and testing engineers instead of having them work in isolation.
PUTTING QAOPS INTO ACTION
In practice, QAOps seeks to provide value by simplifying the process of testing and making it more efficient. This can be done in a variety of ways, including automated testing, parallel testing, and scalability testing. When looking to provide value, there are some issues that need to be overcome in order to “implement” QAOps.
Let’s walk through each of these strategies and explain how they can help create a QAOps practice.
Automated testing is an essential ingredient for QAOps. Automated testing refers to using technology to perform tests—without the need for human intervention. Automated testing reduces the effort required to run tests; however, it is not a silver bullet that can simplify the work for testing engineers, because not all tests can be automated successfully. There is, therefore, the need to consider which tests should be automated to deliver the most value.
The obvious culprits for automated testing are tests that are easy to automate, thus taking away the need to repeatedly apply tedious manual effort. Additionally, any tedious test cases that require complex interactions should also (ideally) be automated. This reduces the chance of diluting the quality of your tests with human error. Frequently used functionality should also be automated. Appropriate automation in conjunction with manual tests will increase insight into possible issues with highly used functions, which is important for ensuring that your most-used functions do not fail (thus, you avoid creating a bad experience for your users).
As mentioned above, implementing QAOps should make your testing more efficient instead of slowing down your CI/CD. Applying automated tests can speed up your process, but it can create bottlenecks if all tests are run serially.
One effective way of overcoming this is to use parallel testing. Parallel testing refers to running multiple tests at the same time rather than running them one after the other. In practice, this might require more hardware to be effective, because you will need more infrastructure in order to run more tests simultaneously. However, the benefits of quicker test results (helping you push to production) will outweigh the cost of a few extra CPUs in the long run.
Of course, you can also take advantage of cloud-based test infrastructure, allowing virtually unlimited capacity to run parallel tests—without having to purchase more hardware of your own.
Scalability testing, another component of QAOps, refers to tests that allow the measurement of an application’s response to increasing or decreasing load, making it possible to determine your application’s performance under different conditions, and allowing you to optimize for desired characteristics.
Performing these tests may require access to specific resources to put your application in production, which could mean extra computing costs if you need to test under real-life conditions. As an alternative, you could test the scalability of your application on a smaller scale (which will suffice as long as the behavior of your application is predictable under larger loads based on your small-scale tests).
GETTING BUY-IN FOR QAOPS
Beyond the technical keys to QAOps described above, keep in mind, too, that a successful QAOps operation requires getting buy-in and support for QAOps from your organization — which starts with communicating the value of QAOps to decision-makers and other members of the technical team. You’ll want to lay out a clear path for achieving QAOps so that the rest of the organization understands what’s feasible.
In short, operationalizing QAOps requires both a specific approach to software testing and a campaign to convince stakeholders of the value of QAOps. Clearing these hurdles may take time, but once you are past them, you’ll begin enjoying faster, more efficient software testing.