WHITE PAPER

Building Analytics Now and for the Future

Sponsored by: IBM
Jill Feblowitz
September 2015

IDC OPINION
Utilities continue to be held accountable for maintaining reliability; protecting the health, safety, and
environment of the employee and the community; keeping customers satisfied; and enhancing
shareholder value in the face of rapidly changing conditions. Today, utilities are actively seeking to
address most, if not all, of these challenges by deploying analytics. The most forward-looking utilities
are seeking to understand how new insights can be gained through application of analytics to data
from throughout the organization. The first steps have been to understand the potential use cases of
analytics, but ultimately, utilities want to understand how to most cost effectively enable analytics
across the enterprise. Companies want to be able to assemble an approach that will work for them
now and that will be "future proof." IDC Energy Insights' findings are as follows:

Regardless of the business unit (generation, transmission, gas power and/or water
distribution, or retail) and the function (planning, operations, maintenance, marketing, or
customer service), utilities must be able to use analytics to provide the critical link between
strategic, tactical, and operational decision making.

Confidence in results is improved by applying advanced analytics (simulation, optimization,

clustering, and cognitive) to a large pool of data throughout the organization.

The ideal analytics platform will support streaming data, allow quick access to multiple sources
and types of data, deliver quality data, analyze the data in context, and provide an open
system for development.

IN THIS WHITE PAPER

This IDC Energy Insights white paper discusses the current and future challenges that utilities face in
meeting their business objectives and the role analytics will play. It calls out the use of analytics for
strategic asset management as a concrete example of an approach that is already proving to be
successful. Guidance is provided to utilities about how to evaluate analytics options.

SITUATION OVERVIEW
The past several years have seen some major changes in the conditions that utilities are facing. There has
been a global redistribution of demand growth in Asia and the Middle East and changes in supply with
unconventional oil and gas. Then too, the mandate to decarbonize has utilities reassessing their asset
portfolios. At the same time, distributed energy resources are going mainstream as consumers "power their
own." In addition, extreme weather events are occurring with greater frequency, and aging assets may still
operate beyond their useful life. Utilities battle other industries for the best talent as their mature talent is
retiring as well. Finally, today's consumer expects to shop, drive, and network anytime and anywhere.

September 2015, IDC Energy Insights #EI258958

Twenty years ago, utilities faced no competition. Now competition comes not only from the unbundling of
the markets but also from nonutility companies. In the face of these conditions, utilities continue to be held
accountable for maintaining reliability; protecting the health, safety, and environment of the employee and
the community; keeping customers satisfied; and enhancing shareholder value.
Today, utilities are seeking to address most, if not all, of the aforementioned challenges by deploying
analytics. Utilities now have access to more information than ever before and have come to recognize
the need to make the best use of available data by applying analytics. In fact, investment in analytics is
increasing. Recent research shows the deployment of Big Data and analytics in production in business
units and enterprisewide has increased from 10.2% in 2013 to 30.3% in 2015 (see Figure 1).

FIGURE 1
Stages in Deployment of Big Data and Analytics in Utilities, 20132015
Q.

At what stage is your organization today in the deployment of Big Data and analytics?

In production, enterprisewide
In production, business units
Pilot/proof of concept
Researching

Considered, not yet pursuing

Not considering
0

10

20

30
40
50
(% of respondents)

60

70

2015 (n = 89)
2014 (n = 84)
2013 (n = 125)
Source: IDC's Vertical IT and Communications Survey, 2013, 2014, and 2015

2015 IDC Energy Insights

#EI258958

Regardless of the business unit (generation, transmission, gas power and/or water distribution, or
retail) and the function (planning, operations, maintenance, marketing, or customer service), the
business must be able to do four types of analysis to provide the critical link between strategic, tactical,
and operational decision making (see Figure 2):

Portfolio analysis. In this form of analysis, decisions are being made at the highest level of the
organization (e.g., Does the company have the right mix of products, customers, and supply?).
The intent of this decision making is to allocate resources over the longer term.

Scenario analysis. The transition from strategic decision making to tactical decision making
involves the ability to vet the portfolio and value chain structure by performing "what if"
simulations. The output of this analysis is an understanding of the company's vulnerabilities
and the ability to mitigate (or, in some cases, exploit) risks.

Value analysis. This form of analysis seeks to balance outcomes that can be in conflict.
For example, a company may improve its order fulfillment reliability, but if it is achieved
through higher inventory investment, it may not be in the best interest of the firm. The objective
of this analysis is to create an optimized tactical plan one that balances countervailing
measures for the best possible outcome.

Situational analysis. At an operational level, this analysis ensures that actions are compliant
with the business policy formulated by the higher-level decision making.

FIGURE 2
Analytics to Inform Business Decisions
Do we have the
right set of
assets?
Do we have the
right product
offerings?

What are our

vulnerabilities?
Are we able to
mitigate risk?

What trade-offs
do we need to
make?
What are our
long-term and
tactical plans?

Are our actions

compliant with
business policy?
How do we
respond right
now?

Portfolio
analysis

Scenario
analysis

Value
analysis

Situational
analysis

Source: IDC Energy Insights, 2015

For the utility, portfolio analysis is about utility assets generation, pipelines, wires, and treatment
plants. It is also about product offerings that are standard (water, gas, and electric) but also could be
related to new products (HVAC, ancillary, and energy management). Scenario analysis is about
understanding the company's vulnerabilities and risks (security, emissions, and extreme weather) and
the ways to mitigate the risk of environmental impacts, exposure to commodity prices, and safety
incidents. Value analysis helps a utility make trade-offs, such as between customer satisfaction and
cost to serve. This analysis also supports long-term planning such as integrated resource planning
(IRP) and tactical approaches such as zero tolerance to safety incidents. Situational analysis which
is different from situational awareness is intended to help a utility ensure that short-term actions
comply with business policy. An example of situational analysis might be having standard operating
procedures in place or being able to respond quickly to unplanned outages or security breaches.

2015 IDC Energy Insights

#EI258958

Today, utilities are looking for answers to very practical questions, some of which are listed in Table 1.

TABLE 1
Questions to be Answered by Analytics
Objective

Questions

Reliability

How can we reduce outages and shorten outage restoration time?

Can we predict when the next storm will threaten a specific overhead distribution line?
When should we repair an asset that is potentially failing?

Protecting health,
safety, and the
environment

Can we anticipate an incident about to happen and prevent it?

When should we repair an asset that is potentially failing?
How can we detect potential pipeline failures in time to prevent accidents?

Capital effectiveness

How can we reduce the cost of replacing equipment (poles, pipelines, transformers, cables)?

Operational efficiency

How can we reduce theft and loss?

Where is there potential to optimize power flows across transmission lines?

Customer relationships

How can we respond more quickly to customer sentiment?

Are there customers who we should be targeting for demand response in constrained areas?
How can we drive customers to digital channels?
What approaches are likely to be successful in strengthening relationships with individual
customers?

Source: IDC Energy Insights, 2015

Of course, utilities can use basic analytics such as querying and reporting that come with business
applications to address these questions, but this approach will be limited. Some analytics may be
embedded in existing applications but would be more effective if extended across more information
sources than what is contained in the application. This approach does not take advantage of data from
multiple sources structured and unstructured, internal and external, time series and transactional
that is now available to utilities.
Applying advanced analytics (simulation, optimization, clustering, and cognitive) to a larger pool of
data improves the confidence interval of the results. IDC Energy Insights recently conducted a survey
of utilities in the United States and Europe to understand how mature utilities are in their approach to
analytics. Utilities that stated that they had achieved their objectives with their Big Data and analytics
initiatives high achievers were more likely to have invested in structured reporting tools and
multidimensional analysis tools than those that had not achieved their expected benefits.
High achievers also rated advanced analytic tools for predictive analysis or data mining, dashboards,
and analytics on mobile apps.
To illustrate, let's look at the area of asset management in the sections that follow.

2015 IDC Energy Insights

#EI258958

A Closer Look at Asset Management

Addressing the Risk
As an asset-intensive industry, the utility industry has always been interested in managing assets to
ensure reliability, safety, and efficiency. Utilities have largely moved beyond a break-fix approach and
are increasingly under pressure to become more strategic about how they manage their assets using a
risk-based approach. They want to:

Mitigate economic and health, safety, and environment risks associated with asset failure,
such as unplanned outages to generation plants or pipeline failure.

Ensure reliability of assets related to market demand, delivering effective capital spending for
high-volume/low-cost equipment (cable, transformers, poles, etc.) and high-cost/low-volume
equipment (turbines).

Reduce the costs of maintenance, improve operational performance, and extend the life of
the asset.

Eliminate unnecessary inventory and improve relationships with suppliers.

Strategic asset management takes into account asset management, workforce management, and the
supply chain. From the asset management perspective, companies seek to understand the health of
assets and take the optimal approach to address the asset based on business objectives. For
workforce management, the objective is to effectively, efficiently, and intelligently deploy a network of
operations and maintenance workers to mitigate risk. Supply chain management ensures that
replacement parts and equipment are up to specifications and arrive when the crews are scheduled for
work without maintaining excess inventory.
Taking a closer look at asset management, we note that the typical process involves:

Alerting to imminent asset failure or, better yet, predicting when an asset is likely to fail

Understanding the cause of the potential failure and possible remediation approaches

Deciding what to do when and whether to repair, replace, or take no action based on asset
criticality, cost, and resource availability (In ideal circumstances, the way forward will take into
account the criticality of the asset, level of risk, and market conditions such as price of
electricity or gas.)

The Role of Technology

Knowledge of an asset and how it performs under a given circumstance is tribal, passed from engineer
to engineer, as well as systematic, established through historical measurements that involve formal
benchmarking. Where applied human knowledge falls short or retires, computing-based systems are
increasingly able to bring clarity and consistency to asset management objectives.
For the most part, utilities are already set up to collect, monitor, and receive alerts on equipment in
generation, gas processing, and water treatment plants where equipment is highly instrumented and
operated through control systems. Alerts on highly distributed equipment, such as water and gas
pipelines and transmission and distribution lines, are available if there are control systems, but these
distributed assets tend to have fewer sensors and less connectivity than in plants.

2015 IDC Energy Insights

#EI258958

That said, alerting and alarming may either be excessive and ignored or not be delivered in time
to assemble the resources and equipment to address imminent failure. Predictive analytics are
used to identify potential failure or indications of asset health. Again, predictive analytics are in use
in many plant situations in the industry. Typically, analytics that are mathematically based detect
anomalies to normal operating conditions through the analysis of condition data (temperature,
vibration, power usage, etc.).
Engineering models based on degradation models or simulations under various conditions improve the
degree of certainty. With predictive analytics alone, utilities have been able to identify and address
single instances that have resulted in substantial avoided costs. For example, in one case, a large
North American power utility was able to save an estimated $4 million in avoided unplanned downtime
and repair costs in one "catch." Data sciencebased predictive analytics showed a pattern in vibration
at a steam generation plant that would not have been identified with standard monitoring. A root cause
analysis and inspection showed that the cause was loss of shroud material. The analytics applied did
not include prescriptive (identifying the repair alternatives) or optimization (weighing alternative action
paths), which likely would have yielded more efficiencies.
For the most part, identification of the root cause of failure relies on onsite inspection. However,
remote monitoring centers staffed by performance engineers are using drill-down techniques and
visualization and collaboration with onsite personnel. Still to come are prescriptive and cognitive
analytics to recommend actions based on learning the best practice approaches from similar
situations. The extra power of cognitive analytics helps in dynamically changing complex situations
where computing can quickly identify repeatable and effective solutions.
Optimization analytics are currently available in recommending what to do with limited resources, and
integration plays a role in executing on the decision. Advances in processing allow for utilities to
prioritize based on their objectives and quickly access optimization results. Integration with work and
asset management and procurement supports timely completion of the task, coordinating the
workforce and supply chain.
From another perspective, the strategic asset management process involves the ability to access data,
analyze, and execute and learn (see Figure 3).

2015 IDC Energy Insights

#EI258958

FIGURE 3
Strategic Asset Management Process

Analyze

SCADA
Condition sensor
Equipment specs and
exploded view
Repair history
Video and photo from
operations, maintenance
rounds
Equipment and project costs

Predict potential failure

based on mathematics
and/or engineering models
Perform root cause analysis
based on conditions and like
equipment in like conditions
Prescribe best practice
options to solve situation
Recommend based on
market conditions and
business objectives

Initiate alerts and job orders

to repair, replace, substitute,
and defer
Present data and
instructions to operations
and maintenance personnel
in the field
Recommend refined
operating procedures,
equipment design, and
optimization

Execute and
Learn

Access Data

Source: IDC Energy Insights, 2015

The Platform
It is clear that the technology to support strategic management is available today, but the challenge is in
assembling it in a way that delivers the most value for the investment. There is no shortage of vendors in
the market that are offering their particular take on the solution, architecture, or platform. Frameworks
and architectures assume what is being purchased is primarily services to build a solution. Platforms
assume a solution that is "ready for use," requiring fewer services to implement and maintain. There are
platforms being offered to the market that support device connectivity to support the Internet of Things.
There are platforms that are data and analytics oriented that support decision making.
Regardless of the label, utility decision makers should be able to quickly access and analyze quality
data. The ideal approach is a platform that can:

Support streaming data from equipment and sensors. The asset base of utilities will remain
heterogeneous with equipment, control systems, and communication protocols from multiple
manufacturers as a result of replacement cycles, vendor preferences, legacy processes,
and mergers.

Quickly access multiple types and sources of data. Data ingested may come from internal sources
(equipment sensors, historians, control systems, business applications EAM, ERP, GIS) and/or
external sources (manufacturer specifications, exploded view diagrams, localized weather data,
etc.). It may be both structured and unstructured, time series and transactional. For example, utility
customers may upload pictures of downed lines to social media that may be helpful to the utility in
dispatching crews with appropriate restoration materials. The platform should enable analytics to
be applied to data from across the organization. For example, new insights into equipment health

2015 IDC Energy Insights

#EI258958

may be found in analyzing data from distribution operations, maintenance, and customer meters
that tell the story of how the asset is operated, how it has been maintained, and how load is
expected to change. This will require a utility data model built on industry-recognized standards
such as the Common Information Model (CIM) with an understanding that standards are a work
in progress.

Deliver quality data. One of the chief complaints of utility decision makers is the quality of data
in existing applications. There are automated ways to ensure data quality through the use of
analytics in the data processing stage.

Analyze the data in context. Analytics on individual assets is the first level, but a nextgeneration approach involves viewing the asset in context. Whether dealing with a plant or a
pipeline, a utility needs to understand what equipment is critical to the process or what
connected asset is upstream or downstream from the asset in question. While a compressor
may not be considered critical to operations, it may supply a piece of equipment that is critical
to the process. From a power perspective, analytics enable whole-network reliability
maintenance and network-driven criticality and consequence analysis. A utility should be able
to manage any asset (unlike some that focus on only high-value assets) and be able to model
the interconnection of assets.

Accommodate multiple analytics. Rather than be limited to a prescribed set of analytics, the
platform should be able to accommodate multiple analytical tools and applications as
described previously. The particular methodology will likely vary based on the questions to be
answered. For example, Monte Carlo simulation might be appropriate in some circumstances,
while linear programming would be useful in others, or a combination of these two
methodologies, plus other analytics.
Another example is asset management. The models may be strictly mathematical
(i.e., indicating a potential problem when parameters being measured deviate from normal
operating characteristics) or an engineering-based model of the asset (asset failure predicted
under prolonged operating conditions), such as for cables, motors, compressors, and
transformers. In addition, the analysis can be conducted on both instrumented and
noninstrumented assets, such as utility poles. Better yet, a platform that is prepopulated with
relevant industry-specific analytics supports quick start-up.

Provide an open system for development. The most creative and innovative technology is
open source. It is being conceived and hatched by individuals or groups of developers.
Regardless of whether companies want to deploy open source technology, they will want to
implement an open system that allows applications to be built to answer questions beyond
asset management as the need arises. This enables utilities to implement in-house models
with their team of data scientists or to work with partners of choice while retaining ownership of
their intellectual property.

2015 IDC Energy Insights

#EI258958

Figure 4 displays what an ideal platform would look like for strategic asset management.
Figure 4 can be extended to help address other business objectives that the future holds, especially for
the distribution grid of the future.

FIGURE 4
Utility Strategic Asset Management System Mapping
= Data source, store
= Data source, real time

Business
applications

Geospatial

Operational
applications

Ecosystem partner analytic apps

Native
analytics

Native
analytics

Analytics platform

Integration

Tools

Control systems
Connectivity
Equipment

Devices

Note: Business applications include ERP, EAM, mobile workforce, and procurement. Operational applications include distribution
management systems, energy management systems, and generation management systems. Control systems are those associated
with PLCs and DCSs, as well as SCADA.
Source: IDC Energy Insights, 2015

FUTURE OUTLOOK
There will be even more information sources on the grid in the future. With replacements and new
construction, there will be a higher penetration of equipment with embedded sensing. Device
instrumentation costs continue to drop, which opens up the prospects for adding intelligence to
unconnected equipment such as transformers, relays, switches, and fault indicators on the distribution
grid. Even if a utility is not ready to connect equipment immediately, OEMs are instrumenting
equipment with communication capabilities for future use. Price declines are also expected in
communication networks, sensors, and data storage.
Research and development efforts are under way to assess the potential for edge or distributed
computing that will need to interoperate with central decision making. Capabilities at the edge such as
computing capacity in smart meters or distribution line sensors could improve latency, decrease the
need for bandwidth, and support automation. Centralized platforms will need to be able to coordinate
with computing platforms.

2015 IDC Energy Insights

#EI258958

Most importantly, the rapid increase in the penetration of distributed energy resources distributed
generation, demand response, and storage poses challenges and opportunities for the utility. Both
generation and demand will become more variable and be subjected to new dynamics outside of the
control of the utility. One of the challenges is loss of revenue at a time when utilities need to make
investments in infrastructure to support a two-way distribution grid. An opportunity comes with new
business models such as new market structures for distribution service providers to offer grid operation
services. Other opportunities come with delivery of energy services on the other side of the meter.

CONCLUSION
The platform of today should be able to support the needs of the future, with a minimum of additional
investment. Asset management is just one example. It will need to:

Accommodate multiple communications protocols and multiple manufacturers of standard and

new equipment (solar inverters and wind generators)

Help decision makers understand the quality of data sourced from heterogeneous systems

Handle scale of available data and facilitate both local and centralized processing

Work across business functions For example, it will be important for utilities to understand
which customers are most likely to respond to repeated calls for demand response in a
constrained neighborhood to understand how to prioritize the demand response calls to that
customer over time and eventually plan asset replacement or network growth using customer
related data.

Utilities seeking to advance their capabilities to make the best business decisions will need to:

Start understanding how the organization utilizes information to gain insight. Most organizations
are missing out on opportunities to use data from many parts of the organization.

Explore how platforms deliver on the asset management insight or other insights that platforms
promise. The data needs to be organized to ensure the costs do not outweigh the benefits.

Evaluate existing technology and its strengths and shortcomings and what role the technology
has in vendor solutions. The scope needs to include a determination of how disparate
systems, equipment, and analytics will be harmonized.

Experiment with proof-of-concept projects that are quick to establish, low in cost, and high in
transferrable learning.

Identify relevant technology and analytics skills among existing staff, peers, and vendors.
Assess skill gaps and plan to hire and/or externally source professional services to fill
those gaps, at least initially. Develop data experts with the operational knowledge and
cross-functional skills to analyze vast data sources with analytics tools.

Adopt a nimble, continuous improvement perspective to respond to rapidly emerging business

challenges while continuing to deliver high levels of reliability and cost control.

Use early quantifiable wins to demonstrate potential and justify budget allocations.

Take a look at changes in business processes that could create efficiencies as analytics are
implemented. Do not overlook change management as workers learn to work with analytics
tools and roles evolve.

2015 IDC Energy Insights

#EI258958

10

About IDC
International Data Corporation (IDC) is the premier global provider of market intelligence, advisory
services, and events for the information technology, telecommunications and consumer technology
markets. IDC helps IT professionals, business executives, and the investment community make factbased decisions on technology purchases and business strategy. More than 1,100 IDC analysts
provide global, regional, and local expertise on technology and industry opportunities and trends in
over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients
achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology
media, research, and events company.

Global Headquarters
5 Speen Street
Framingham, MA 01701
USA
508.935.4400
Twitter: @IDC
idc-insights-community.com
www.idc.com
Copyright Notice
Copyright 2015 IDC Energy Insights. Reproduction without written permission is completely forbidden. External
Publication of IDC Energy Insights Information and Data: Any IDC Energy Insights information that is to be used in
advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC
Energy Insights Vice President. A draft of the proposed document should accompany any such request. IDC
Energy Insights reserves the right to deny approval of external usage for any reason.