In a certain sense, there is a paradox inherent in software testing within a DevOps organization. On the one hand, DevOps emphasizes the importance of continuous testing starting early in the software delivery cycle (which means shifting testing to the “left”). On the other hand, DevOps also encourages continuous feedback loops so that developers learn quickly about problems that occur post-deployment, and can react to them. In this respect, DevOps encourages teams to shift testing “to the right,” in order to collect data about software quality post-deployment and use it to drive continuous feedback cycles.
This paradox may leave developers and QA engineers wondering: Are we supposed to shift testing to the left, or to the right? The short answer to that question is that you are supposed to do both. For the longer answer, keep reading.
SHIFT-LEFT TESTING
If you are familiar with the DevOps and software testing conversation, you are probably aware of the shift-left testing concept. Shift-left testing means performing tests as early as possible in the software delivery pipeline, with the goal of discovering issues when they are easier to fix.
SHIFT-RIGHT TESTING
Shift-right testing, meanwhile, refers to the idea of performing tests later in the delivery pipeline than you normally would (in other words, to the “right”). Whereas you would typically perform most tests just before deployment, shift-right testing involves moving tests to post-deployment, production environments.
In some respects, shift-right testing is just another term for software monitoring of production environments. However, it’s a bit different in that it emphasizes the importance of involving QA teams in production monitoring and using their insights to help improve overall application quality. That is different from traditional production monitoring, which is performed only by IT engineers, who typically respond to problems but (in a pre-DevOps world) didn’t feed information about production issues back to the rest of the software delivery team in order to improve the software quality overall.
The concept of shift-right testing has received less attention than shift-left testing, but it is a thing that people talk about.
HOW TO SHIFT LEFT AND SHIFT RIGHT
It can be tempting to think that if you do either shift-left testing or shift-right testing effectively enough, then you don’t need to do the other type of testing. If you catch all software quality issues early on through shift-left testing, then you need not worry about problems in production, right? And similarly, if you do shift-right testing really well, you’ll find your application issues and address their root cause quickly, obviating the need for thorough pre-production testing. Or so you might assume.
In reality, of course, the most effective QA strategy involves both shift-left and shift-right testing. You’ll never catch every application problem through pre-production testing, so shifting left on its own is not enough. Likewise, if you rely solely on shift-right testing, then you won’t find out about problems until software is in production, which means you expose your end-users to software quality issues unnecessarily.
For QA teams with limited resources, the question then becomes determining how to shift left and shift right at the same time. In other words, how can QA teams beef up testing on both ends of the software delivery pipeline without doubling the time and effort they spend on testing?
Automated testing is one big part of the answer. Through automated testing, you can run more tests with fewer engineers. You can thus test on both the left and the right effectively, without driving yourself crazy. (Keep in mind that in the context of shift-right testing, test automation might include not just traditional scripted software tests that execute automatically, but also the use of automated monitoring tools.)
Parallel testing (which tends to go hand-in-hand with test automation) is another useful strategy. By running tests in parallel, your QA team can run more tests at once, which helps tests be performed on both ends of the pipeline.
Finally — and this is perhaps the most important point — in order to shift left and shift right at the same time, your QA engineers need to be in close conversation with other members of the software delivery team, on both ends of the pipeline. For shift-left testing, this means that QA engineers should be working closely with developers so that issues that are discovered through shift-left tests can be fixed within application code quickly. For shift-right testing, the QA team must collaborate closely with IT staff so that data about software quality or performance issues from production environments can be used to drive an improvement in software quality. (It also helps in this respect to have a strong relationship between the QA team and developers, since it’s developers who will need to take information from shift-right testing and use it to write better code.)
CONCLUSION
You can have it all. You can shift your software tests to the left and to the right at the same time. And as long as you leverage the right tools, such as test automation and parallel testing, you don’t need to double the size of your QA team, or have your QA engineers work overtime to do it.
