To seasoned IT engineers, DataOps may not sound very difficult. After all, if you work in IT, you know all about managing databases, provisioning data storage, moving data around and so on. How hard could DataOps be, really?
The answer is, “Harder than you might think.” Even if you work frequently with data in the context of ITOps, DataOps probably requires you to learn some new skills.
Those skills are not impossibly hard to master; any competent IT engineer can learn to do DataOps. But it requires learning some new tricks and—most important—some new ways of thinking about data and how it is processed and managed.
What is DataOps?
For the purposes of this article, DataOps refers to data operations in general. It encompasses all the processes an organization must perform to make use of data—from collecting and storing data to managing and transforming it, to analyzing and archiving it, to optimizing the performance of the data that drives applications.
ITOps and Data
ITOps engineers certainly interact with data on a regular basis. However, the type of data-related work they do is limited to specific types of tasks, such as the following:
- Setting up and managing databases.
- Provisioning data storage infrastructure.
- Load-balancing and monitoring data transfers over the network.
- Configuring access control policies for data.
- Managing data backup and recovery routines.
- Managing and aggregating log files.
ITOps might occasionally do other things with data, too; the above is not an exhaustive list. But in general, the work that ITOps does in relation to data falls short of the broader set of tasks that DataOps entails. ITOps does not typically play a major role in work-related data analysis, data transformation, data quality control or data monitoring and optimization.
Extending IT Skill Sets for DataOps
We live in a world where data is becoming increasingly more important to running businesses and applications. Making sense of data is critical for making informed business decisions. At the same time, data is critical for powering applications that rely on machine learning and artificial intelligence. More generally, it’s essential for monitoring and managing application environments.
That is why it is more important than ever for ITOps engineers to extend their work into the realm of DataOps.
As noted above, although making that jump requires gaining skills that IT engineers don’t typically use in their IT work, it’s not as hard as you might think. Consider the following strategies for helping to extend ITOps skills into DataOps work.
Think in Terms of Data Performance, Not Just Data Availability
ITOps engineers are used to making sure that data remains available. They troubleshoot disk failures, network connection failures and other issues that could make data unavailable.
It’s easy to extend these skills into DataOps if IT engineers focus not just on data availability, but also on data performance more broadly. Instead of thinking just in terms of whether data can be accessed as intended, think about whether data can be accessed quickly enough, whether it meets data quality requirements and if there are any bottlenecks that threaten the ability of data to do its job.
Systematize Data Operations
Because managing data is not IT engineers’ main job, it’s easy for them to think of data-related tasks as one-off or ad hoc requirements. They might check databases for corruption only when an issue arises, or wait until a storage system starts running out of space to think about how to optimize it.
DataOps requires a more systematic and consistent approach to data management. Data performance optimization and management should be performed continuously, as part of the daily work that IT engineers do to support application delivery in general.
Optimize Infrastructure for Data Operations
When it comes to planning and setting up infrastructure, IT engineers usually hold the keys to the kingdom. They decide which infrastructure to stand up and how to architect it.
Traditionally, data operations were not a primary consideration when planning new infrastructure or optimizing existing infrastructure; instead, the focus was on application deployment. But by making data operations a key focus as well, IT engineers can ensure that the infrastructure they control is ideally suited to meet their company’s data operations needs, too.
Make Data Monitoring a Part of APM
IT engineers usually own application performance management, or APM. But traditionally, their focus on APM workflows is about optimizing the performance of software, not the data that software depends on.
Given the inseparability of application performance from data performance today, that approach no longer works. Consequently, IT engineers need to think about data performance management as an integral part of APM. In addition to identifying software and infrastructure problems, their APM tools and workflows should alert them to data quality and performance issues.
Conclusion
ITOps and DataOps are distinct disciplines, but they increasingly depend on each other. For that reason, it’s important for IT engineers in many cases to extend their skill sets beyond traditional ITOps and contribute to data operations, too.
Fortunately, doing so doesn’t require going back to school for a degree in data engineering. In many ways, IT engineers already have the foundational skills for data operations. They simply need to extend their thinking and toolsets to support data operations.
