“It Works for me.”Those are the three most frustrating words a bug reporter can hear.
I know I’ve sent and received that same message at least a couple dozen times in my career. While not every bug report was legitimate, there were still a handful that could be reproduced only in a production environment. It’s a tough situation for everyone, but it’s not insurmountable. The problem, unsurprisingly enough, is the lack of parity between the development and production environments.
So why, given an unreproducible bug, is environmental parity the solution? While it may not be a magic bullet, having two disparate setups adds a host of unintended consequences that can hinder, if not completely block, your ability to reproduce issues.
The most important step toward mirroring your environments is data parity. How you define data parity is going to be determined by your organization’s security policies, but the bigger issue is going to lie in the amount of data, rather than the content of that data. I can’t even count the number of projects I’ve worked on that didn’t manage any seed data for development database tables. Unit tests have (finally) become a standard on most projects, but managing seed data is still something that takes a backseat to productivity.
Seed data generators, when written properly, should be given the same forethought as unit tests. This is especially important when the project has methods that might not hold up so well when presented with a large amount of data. I recently inherited a project from another agency that could have benefited greatly from some well-written seed data generators. The database queries ran fine on manually created local test data, but they were immediately problematic when presented with a couple million rows from a production database dump.
Data parity is the easy part, though. The next step is server parity. There’s nothing worse than getting onboarded as a developer on a new project, and being given a 10-page document outlining all the irreversible changes you have to make to your machine just to run the project. Unless the production server is a mid-2012 MacBook Pro, this is a horrible way to set up and manage a development environment. This is why I am a big proponent of Vagrant. Vagrant was designed and built for this exact problem. Literally nothing good can come from half a dozen developers with half a dozen different environments.
While it may be difficult to mirror your server architecture exactly, Vagrant can get you 90% of the way there. It even has the ability to configure and manage multiple virtual machines, so you can mimic external database and caching servers. Unfortunately, as far as Vagrant can get you, perfect parity isn’t always reasonable. Your server architecture can be replicated pretty closely, but server traffic and integrations that rely on external services are much more difficult to match without some work.
This is where we cheat a little bit.
We may not be able to perfectly mimic the traffic, but applications like Reimann can be used to simulate a high-traffic load on your application, while Riak CS can be used to locally test your Amazon S3 integration. Just a little bit of research can turn up a host of other applications that are compatible with your third-party services. It’s not perfect, by any means, but it gives you the ability to easily and cheaply test the load of these services, as well as the actual integrations.
Taking the time to properly setup and manage a true parity solution between your production and development environments shouldn’t be optional. While it’s easy to take the “startup” route and rush development in order to achieve a solution, a well-designed development environment will pay both your team and your users back tenfold.
