Getting Value Out of the IT Operations Big Data Gold Mine



Detect and isolate operational issues, then prescribe and predict with the help of CA Digital Operational Intelligence

“How much data does your business generate?” “Way too much, several Terabytes per day.” “What do you do with that data?”  “Some we discard, some we store in case it’s useful.”

This is a common conversation I have with customers that have embraced the application economy. Microservices, containers, cloud services, SDN, social media, the list goes on… are just some examples of these non-stop data sources of metrics, alarms and logs.

Yes, the amount of data is overwhelming, but it is also a real Gold Mine for Data Scientists and Artificial Intelligence. Some of the big players of the market have already realized its value and how to monetize it. All organizations know that this Big Data hides evidences that could prevent most of their operational issues, they just need to smell the smoke before seeing the fire.

How to Get Value Out of the Chaos

The silo approach has been a major issue in most organizations. I have heard this topic since the beginning of my career, “Data correlation is hard to achieve when data is spread in isolated repositories.”. Hence it is important to leverage a Data Lake repository where data is stored in its natural format. The Data Lake will be the gold mine where all the raw data lives, ready to be processed by AI algorithms. Data Lakes are the beef of Data Scientist and Data Developers.

Some of the key characteristics of Data Lakes are:

  • Easy to import/export data: Real time ingestion/extraction APIs should be available.
  • Secure: Data must be secure at rest and in transient (to be compliant with GDPR).
  • Accessibility: Machine Learning and analytics needs easy access to raw data to product Insights.

A basic classification of the IT operational data that can be stored in a Data Lake is:

  • Metrics
  • Logs
  • Alarms
  • Events
  • Inventory
  • Groups
  • Topology


This data can come from several domains: User Experience monitoringApplication Performance ManagementInfrastructure Management or Network Operations (NetOps).

big data

The area in the middle is the “gold” we extract from its “ores” or the business Insights produced by intelligent algorithms from the raw data in the Data Lake.

CA Technologies has more than 20 years of experience in the APM, Infrastructure Management and NetOps domains, so we are in an outstanding position to deliver an advanced analytics solution which provides comprehensive insights by automatically ingesting structured and unstructured data across your performance monitoring tools.

Gain Comprehensive Insight Using Analytics

But how do we convert raw data into Business Insights? The answer follows…

CA Digital Operational Intelligence is a machine learning–driven, advanced analytics solution designed to help IT operations teams deliver a phenomenal user experience, improve service quality and drive operational efficiencies.

The built-in machine learning algorithms can detect anomalies, determine root-cause, prescribe actions and even predict situations to watch.

The moment you login, you land on a Business Service View that provides end-to-end visibility and enables you to perform root-cause analysis rapidly.

big data

CA Digital Operational Intelligence Service Analytics


This service-centric analytics view (above) is automatically created by correlating services and mappings with the IT assets that support them. A service can be impacted by IT assets issues or even by other business metrics such as a negative customer sentiment or a decrease in the revenue.

The solution also embeds operational intelligence for noise reduction; it suppresses redundant and duplicate alerts and correlates them into “situations to watch” by using data mining algorithms that look at the “what”, “when” and “where” of each alarm –  helping you to proactively detect issues for faster resolution.

big data

CA Digital Operational Intelligence Alarm Analytics

Do I need to worry about alarm management in CA Digital Operational Intelligence? Not at all, alarms are smart and dynamic. The Machine Learning algorithms will determine the typical values for ANY metric ingested in our Data Lake, based on historical trends and generates alerts for abnormal scenarios while reducing false ones. Basically, CA Digital Operational Intelligence learns what is the “normal” behavior for every metric.


big data

CA Digital Operational Intelligence Anomaly Detection

CA Digital Operational intelligence is built on top of a powerful, open and easy-to-scale engine based on open sources technologies ElasticSearch, Kafka, and Spark. The intelligence is provided by CA’s Analytics platform, CA Jarvis, which is the “brain” that processes the raw data generating Service, Alarm, Performance and Capacity Analytics.

The cross-domain data complexity is encapsulated by an intuitive visualization GUI that correlates and unifies the vast underlying data helping you to deliver a superior user experience.

Sr. Engineering Services Architect at CA Technologies. He is an IT professional with +10 years of work experience and has a Master Degree in Telecommunication Engineering.


Click on a tab to select how you'd like to leave your comment

Leave a Comment

Your email address will not be published. Required fields are marked *

Skip to toolbar