You are on page 1of 11

White Paper

Execute the Vision for Analytics in Oil and Gas: Delivering

Sponsored by: TCS
Chris Niven
March 2017


Big data and analytics are top IT initiatives for oil and gas companies in 2017. These initiatives are
especially important as oil and gas companies strive to further reduce their operating costs through
interconnecting systems for access and to manage the vast amounts of data generated in the oil field.
Companies must exploit this data to gain deeper insight into their operations and performance.

There are key advantages to leveraging analytics that will benefit C-level executives. By giving the
CEO, COO, CFO, and CIO immediate, accurate, and relevant information about operations
performance, these executives gain the ability to make informed decisions.

There are multiple types of analytics including BI, descriptive, predictive, and more advanced analytics
such as cognitive/augmented reality-based human-assisted remote sensing and inspection. The
objectives of augmented reality analytics are to assist humans in the field remotely to perform data
collection and inspection tasks using multimodal augmented reality and gamification.

This IDC Energy Insights White Paper explores best practices, approaches, challenges, and expected
benefits of using analytics to tap big data to solve complicated problems. Analytics can be used in a
variety of ways to improve efficiencies; for the oil and gas industry today, upstream operations and
production are two high-priority use cases for analytics.


The advent of low energy prices worldwide over the past couple of years has been a catalyst for oil
and gas companies to reinvent themselves to become operationally efficient and effective. When oil
prices started to plummet, energy companies could no longer just drill another well and count on the
profits to support the high costs of E&P. Initially, oil and gas companies responded by cutting staff,
delaying projects, and renegotiating contracts with vendors to rapidly lower costs.

Few oil companies can maintain profitability given today's low energy prices. Most companies must be
more efficient, particularly if they have already made cuts and renegotiated agreements. To
compensate for staff reductions, oil and gas companies are looking to quickly automate processes
wherever it makes good sense.

Companies are adding new devices and software in the oil field to facilitate automating some activities,
but value will not be realized until the associated data is collected and combined with other relevant

March 2017, IDC #US42374517

data to expand the reach of analytics. Many companies have indicated that their current IT
environment first requires an integrated data access solution to successfully deploy analytics.

While only a small number of companies have demonstrated maturity with their use of analytics, IDC
Energy Insights believes that big data and analytics are now mainstream in oil and gas: 16% reported
enterprise installations, and others are testing high-potential pilots for impact and returns. This is fairly
consistent with other industry averages, and the number of enterprise installation is growing rapidly.
For the most part, companies are using analytics on a limited basis, but the use is growing fast as the
benefits of analytics are realized.

As oil and gas companies seriously invest in analytics solutions, many need a strategy and an
architecture to help them make the transformation to the 3rd Platform. Companies often require the
expertise of third parties, especially during times of low oil prices and staff reductions. It is important to
work with professionals that possess deep oil and gas domain knowledge and technical expertise to
help energy companies identify and develop top analytics opportunities and solutions.


Ongoing business conditions in the energy industry are pushing companies to more rapidly adopt
innovative cloud and analytics technologies. Adopting these technologies often requires help from
trusted partners to deploy innovative solutions so companies can become more efficient, agile, and
capable of responding to change.

One of the main goals of analytics is to help companies leverage the vast amounts of data that are
continuously generated in the oil field. This data is critical to gain deeper insight into optimizing
operations performance as well as improve productivity and achieve other positive outcomes such as
adhering to compliance requirements and minimizing risk.

Big data and analytics encompass a broad set of initiatives — it is important therefore to first determine
and prioritize what areas of the business will benefit the most from analytics. It is also important to
identify which initiatives can be quickly deployed to achieve immediate value.

Oil and gas companies plan to increase IT spending in 2017 — most notably in upstream operations in
both production and drilling. As companies digitally transform their businesses and cloud, big data, and
analytics all come together with mobility applications, innovative technology initiatives are made
possible. The resulting integrated IT environment will be developed incrementally over time and will
connect internal and external partners, customers, mineral rights owners, vendors, suppliers, and
others into an ecosystem to provide a holistic view of upstream operations.

It is this kind of environment in which analytics can have the most impact. Oil and gas companies
reported that they have three main purposes for enabling analytics: to help optimize processes, to
allocate resources more effectively, and to predict outcomes. When selecting business objectives for
analytics, it is helpful to prioritize each objective in terms of how they will be impacted by analytics.

For example, if a company selects production optimization as the top objective, it is important to
understand how analytics will impact the processes and resource allocations involved. It is also
necessary to predict the expected outcomes to determine the potential value. It is advisable for
companies to study their IT environments for how and what type of analytics can be applied to realize
the optimal results.

©2017 IDC #US42374517 2

It is encouraging that oil and gas companies are investing money in drilling again, yet it is the
production environment that usually presents the best starting point for a big data and analytics
initiative. Existing wells that are proven and producing (PDP wells) can leverage analytics to establish
benchmarks for performance that can be monitored and managed and thereby realize the best results.
To determine best practices and predict future well performance, key metrics are helpful to understand
what worked and what didn't — metrics that are available via analytics.

Keep in mind, however, that the decision of where to begin with a big data and analytics initiative
depends on each company's strategy; drilling or another function may be more important than
production at some companies.

It is imperative to first understand the purpose of the analytics and the expected outcomes to be able
to identify the requisite data and determine the integration requirements — integration is often the
biggest challenge. The technologies and approach will depend on the type of data to be analyzed, as
well as the volumes, variety, velocities, reporting tools, and the accuracy required.

Solutions can range from using a relational database for managing critical structured data for analytics
to using newer and more open technologies like Hadoop and other similar technologies for managing
high volumes of unstructured data. The implications of each approach can be significant as techniques
will vary — for example, using master data management for detailed accuracy or, alternatively,
sampling large amounts of data if that is more appropriate. The most effective design, software, and
road map are best determined by a DX or data architect and a competent team of IT and oil and gas
business professionals.

Some of the promising areas where analytics can benefit oil and gas companies achieve operations'
optimization include drilling and production as the highest priorities. Production is the top LOB in terms
of budget increases for 2017 followed by drilling. Analytics is also useful in finding oil and in horizontal
ways like improving processes, supply chain, and other functional ways, but most of the IT budget will
be for improving production efficiencies and outcomes.

Most companies reported they expect that analytics will continuously help improve processes and
optimize output performance somewhere around 8-30% according to some studies. Analytics will also
be applied to help solve safety issues and mitigate risk as well as improving production and reservoir

Well Production Analytics

Oil and gas production is a large and complex activity with many moving parts. The basic concept of
analytics for production is applicable to all hydrocarbon plays in any geography, including shale and
offshore. All hydrocarbon plays involve hydrocarbon accounting, and allocations, and all wells have
equipment, devices, sensors, and other sources with meters that provide valuable data that can be
included in analytics.

Each company has its own approach for producing hydrocarbons, and analytics is helpful for solving
problems, determining best practices, and predicting failures before they occur. Several oil and gas
companies are leveraging analytics to monitor the health of equipment and devices. This use of
analytics is valuable for predicting potential downtime and even predicting an impending event to take
remedial action before it occurs. Upstream asset management is based on this concept of using
analytics to monitor the health of the devices in the oil field; some companies have integrated AI for
deeper learning and better decision making.

©2017 IDC #US42374517 3

Because of the volumes, velocity, and varieties of data in the oil field, an architecture may be deployed
that involves taking samples of unstructured data from the meter for continuous evaluation of
weaknesses or potential well failure to take remedial action to avoid downtime and potential disasters.
A different data management architecture might be developed to understand and manage production
performance using a master data management approach if the objective is to leverage accurate
allocations data down to the individual well level.

Integrating the requisite data is the most difficult problem for most operators. There are different
architectures to be explored as to which one is best suited for the objectives in consideration. This is
an arduous task and why it is important to set achievable metrics, as data from multiple systems must
be cleansed, formatted, and aligned with workflows to be meaningful.

As an example, consider optimizing the production of 3,500 wells, all of which are in the same
geographic region. Some wells are larger, some are deeper, and usually there are multiple different
systems involved with various formats that must be addressed to manage and provide meaningful data
to apply descriptive and predictive analytics about production performance.

Collecting near-real-time time series data is complicated and is usually collected in the operations’
data historian software, and this same data can even be taken directly from the device, sensors, and
other instruments and equipment with meters. This data is used to take a snapshot for a current status
of well performance, which is valuable by itself for making decisions. In addition to meter data, other
systems data can also be included for different perspectives such as financial, maintenance,
accounting, land, and GIS systems — all of which provide a more holistic view of production operations.

There are a host of automated interfaces and visualization tools and applications that companies can
use to monitor, manage, and apply logic, algorithms, and techniques to convert and present data in the
desired format for interpretation and understanding. For example, analytics can be applied ad hoc or in
the form of KPI-driven dashboards. Many companies like the dashboard approach for running
upstream operations as dashboard data can be color coded to easily depict potential and urgent
problems that are violating KPI thresholds. Some operations dashboards also have drilldown
capabilities to quickly assess the underlying problems, thereby enabling companies to take some
positive action to resolve them.

Some important KPIs for production include:

 Top-producing well
 Least producing well
 AFE budget costs by well and exception-based reporting
 Number of failures of each well
 Types of failures ranked by dollar impact or downtime
 Mean time between failures and resolution by well
 Equipment and device reliability for each well
Aggregate information about each well should also be an option as analytics results can be rolled up
by field, location, and project. Individual wells can be rolled up with other wells to be optionally
presented by day, week, month, and other time frames.

©2017 IDC #US42374517 4

In addition to using KPIs and dashboards to run the business, companies can use analytics to predict
well performance by comparing the results of other similar wells to forecast outcomes. This predictive
capability can be useful for determining what works and what doesn't as analytics helps shed light on
the interrelationships between wells and how each one will react to new techniques and other stimulus.

The kind of analytics tools companies use depends on the desired results, but it is always critical to
first make sure there is access to accurate data. Some other analytical tools and presentation
applications include ad hoc queries, report cards, advanced modeling, simulation, 3D virtual reality,
cognitive, and human language processing, and even AI is being added for rapid analysis of trends
and patterns.

Analytics is a powerful tool for engineers to examine on a regular basis and collaborate about any
changes in the IT environment that need to be addressed to optimize operations performance.
One major oil company deploys analytics every morning to look at activity performance and determine
if wells are performing to expectations. In this manner, multiple professionals can be included in the
discussions to decide what the problems are, what action should be taken, and the predicted outcomes.

In the future, the COO may want to expand analytics to include data from other systems to improve
operations performance. For this reason, a data management framework approach is ideal for
establishing standards and governance and developing an easy-to-use interface for adding new
applications data, including data from third parties.

Drilling Analytics
According to IDC Energy Insights research, companies are planning to increase IT spending related to
drilling significantly in 2017. Analytics can play an important role in drilling-related activities.

Companies primarily use drilling analytics to monitor, manage, and analyze critical drilling data to help
set benchmarks for gauging performance as well as test the impact of using different technologies and
methods. Benchmarks can be set that meet expectations about length of time to drill a well on one well
pad and time to move to the next site to spud.

Other metrics can also be set to help companies manage expectations and determine best practices.
Analytics for drilling will play a large part in helping companies select the ideal drilling equipment,
resources, services partners, and other conditions to improve operations performance.

Expanding Analytics Beyond Core Processes

Analytics is especially valuable when companies use it in conjunction with other complementary
initiatives such as process and workflow improvement efforts, as well as collaboration, scheduling
logistics, and communications projects. As companies strive to improve drilling and production
performance, analytics is also helpful for improving supply chain performance by highlighting ways to
eliminate inefficiencies and to better coordinate equipment, supplies, and people. The end result is to
improve delivery to deploy whatever equipment is needed.

Information from other applications can also add value to analytics. For example, HR data about
employees can be combined with analytics to help select those workers with the most appropriate
skills and experience for specific drilling projects. Including financial data with drilling or production
operations provides an immediate view of the AFE budget status.

©2017 IDC #US42374517 5

Based on past experience, IDC Energy Insights believes that, once the company is committed to
analytics, departments will usually compete to be included with other departments to also realize the
value and want to include their systems data in the data platform for a deeper view and management
of their own performance. Figure 1 illustrates some of the various business activities and opportunities
for upstream analytics. It is important for companies to determine high-impact areas where analytics
can help reduce costs and improve efficiencies.


Areas of Opportunity for Upstream Oil and Gas Analytics

Source: IDC's Oil and Gas Flash Poll Study, November 2016


There are multiple benefits for implementing analytics to help run upstream oil and gas operations. An
integrated IT production framework will help achieve a company's vision of operations optimization by
connecting all the requisite systems and managing the critical data.

Analytics will provide the flexibility to transform organizational behavior and help identify and enable
the best resources to improve activity performance across the ecosystem from drilling and production
to distribution and beyond. With a comprehensive analytics strategy, companies can help to reduce
operations costs and improve performance as well as ultimately improve productivity and profitability.

Some key areas of impact for upstream analytics include:

 Applying analytics to real-time SCADA and process control systems to help operators optimize
resource allocation and prices by making more real-time decisions with fewer engineers
 Being able to predict well performance based on historical results
 Determining sub-par production zones to shift assets to more productive areas
 Providing guidance for fracking and making changes in lifting methods

©2017 IDC #US42374517 6

 Identifying aging wells where the forecast does not meet a predetermined production threshold
to be flagged for immediate remediation
 Detecting well problems before they occur
 Understanding how maintenance is affected by pressure, temperature, volume, shock, and
vibration to prevent failure and avoid downtime
 Predicting equipment failures and enabling teams to more effectively schedule equipment
maintenance in the field
Analytics helps to find the otherwise hidden insights and engineering tweaks that drive more value from of
an oil field and as such can change the way companies approach operations. The new drilling and
production environments are all about having immediate, accurate data that can be shared with team
members for better collaboration and decision making. Successful operations performance improvements
require getting analytics experts, IT staff, and engineering leaders from different departments to emerge
from their respective silos so they can share data and develop new solutions together.


Advancements in technologies in the past few years have exceeded most energy companies' ability to
keep up. Companies are now faced with prioritizing budgets based on keeping the old systems running
while investing in innovation. For example, new applications, devices, and equipment continue to
grow, as do the number of sensors. The CIO must keep up with technology while transforming IT to the
3rd Platform at the same time.

Today, companies face several challenges: They strive to become tightly integrated, to implement
automated processes, and to rapidly adapt to change — all while maintaining operational excellence.
Developing an IT environment to support these goals requires a business plan with clear objectives
and metrics.

Key areas of consideration include:

 Support the initiative and create a team of experienced professionals

 Make oil and gas and technical knowledge and skills to deploy analytics mandatory
 Establish clear objectives and strategies for analytics to improve operations performance
 Start with a question or a problem and determine if there is sufficient data to solve it
 Evaluate existing technologies and deploy best-fit technology architecture and capabilities
 Identify, connect, combine, and manage multiple sources of data
 Take an integrated approach to data sourcing, model building, and organizational
 Take an iterative approach to develop the architecture and data management framework for
Building an integrated data management platform for analytics is needed, and each company must
decide the approach that works best given its current position, capabilities, and goals. Having the right
professionals with the technical skills and industry-domain knowledge is a significant challenge forcing
companies to evaluate their current team's capabilities and look for options outside of the organization
to fill any needs.

©2017 IDC #US42374517 7


TCS works with each oil and gas client to understand and prioritize its key business objectives to
develop an analytics environment that best meets the client's vision of how it wants to apply analytics.
It is critical for organizations to leverage professionals with domain and business expertise of this
caliber and an understanding of the nuances and complexities involved. This knowledge is essential to
know what data is needed to integrate multiple disparate systems into a holistic analytics environment
that provides a 360-degree view of oil field operations.

One end goal is to help an operator drive business agility by enabling faster and smarter business
decisions and sustained business outcomes. TCS has developed ValueBPS, which is a special
approach that enables an enterprise to achieve significant and sustained business outcomes by
leveraging the team's deep domain expertise and operations redesign methodologies.

The TCS approach also encompasses robotic process automation (RPA), analytics and insights,
business process as a service (BPaaS), and business process management (BPM). TCS deploys
oil and gas-specific analytics services, many of which are proprietary models, that are aligned to meet the
client's end-to-end needs in the upstream, midstream, and downstream that span the entire analytics value
chain including data management, reporting, descriptive analytics, predictive modeling, and optimization.

The TCS analytics services include:

 Engineering analytics: Provides operators with control systems; provides petroleum,

mechanical, and instrumentation engineers with a deep understanding of making technology
work in the operator's context (any digital technology)
 Operations performance analytics: Enables operators to understand and improve well
behavior and productivity
 Sales and marketing analytics: Helps marketers plan and manage the four Ps (product, price,
place, and promotion) and measure brands and channels across digital, social, and traditional
media to increase return on marketing investments
 Supply chain analytics: Equips management to make data-driven decisions to improve
effectiveness and efficiency of operations, which in turn translates into decreased costs and
improved profitability
 Risk analytics: Enables businesses to manage risk more efficiently by making risk-aware
decisions and provides financial institutions with end-to-end support in accurate, timely
reporting for regulatory compliance
 Financial insights: Equips businesses to manage expenditure, cash flow, and investments
better and uncovers insights that help in better financial planning and control

Challenges and Opportunities

 Disruption. A significant amount of disruption in the oil and gas industry has limited IT
investment spending to mostly strategic, short-cycle projects with high returns given the
current climate with low oil prices. The good news is that big data and analytics are considered
strategic and high-impact projects, so vendors need to approach clients in an incremental
fashion and show value in each project.
 Complexity. The sheer number of analytics tools and different approaches can vary depending
on the business objectives to be achieved. Vendors working in oil and gas will need to work
closely with clients to understand their current IT environment, establish clear objectives, and
present a solid road map for the technical deployment.

©2017 IDC #US42374517 8


In 2014, most oil and gas companies reported limited experience with big data and analytics. Today,
oil and gas companies have escalated the adoption curve. A majority of companies are working on
developing a solid data management framework environment for deploying analytics, while less than
half of companies believe they already have sufficient data management frameworks in place to
deploy analytics.

With the advent of drones, the IoT, and continued adoption of 3rd Platform technologies such as cloud
and mobile, there will be additional volumes of data available. An analytics framework will become the
hub for the adoption of such new technologies and approaches to rapidly adapt to changing
environments. Hence the need for a solid analytics platform and strategy will only grow over time.


Analytics is a top IT initiative for oil and gas companies, and IDC Energy Insights recommends gaining
active support from leadership due to the strategic nature and high impact analytics will have on the
culture and business workflows. Companies need to realize that data is a valuable asset, but only if it can
be used to solve some critical problem or enable a new capability that realizes value for the company.
This is why it is so important to first identify the business opportunity that data can help solve.

IDC Energy Insights advises companies to demonstrate value to the executives and other
stakeholders every three months by detailing progress and discussing lessons learned. One goal of
this exercise is to develop a framework for adding applications and data to be integrated over the next
one to five years. Over time, this framework approach will grow to connect with other systems and
sources of data to become the big data and analytics platform of the future.

Clear governance backed by top levels of the organization is required to ensure that analytics
becomes an integral part of the corporate culture. In the absence of corporate governance, business
units will work at cross-purposes or may not even use an analytics system. Governance percolates
throughout the organization, supporting consistent approaches at each level — a critical component to
support the alignment of IT with business objectives across all departments. It is crucial for each
department head to collaborate on system design and operations among different business units.


IDC Energy Insights recommends that oil and gas companies identify and configure their team of
internal and/or external experts before tackling analytics solutions. A competent team of professionals
is required with the business and technical knowledge and skills to identify, prioritize, and select the
key objectives for analytics and a technical architecture of the solution. The team should include
professionals with oil and gas domain and application knowledge and the requisite skills to develop an
integrated frameworks solution.

Analytics is a top IT initiative for oil and gas companies, and IDC Energy Insights recommends gaining
active support from leadership because of the strategic nature and high impact it will have on the
culture and business workflows in the company. It is wise to assess the current IT environment and
leverage what is already there before making investments in new technologies.

©2017 IDC #US42374517 9

Developing an analytics solution architecture is best achieved with a business strategy and project
plan and a data model as best practices for implementation. These tools help to illustrate how to
navigate through the complex multitude of applications, workflows, and data types and help to identify
what is required to integrate the right data into the framework, who will perform the work, and when the
work will be performed.

Professional services companies are excellent sources for domain knowledge and good digital
transformation consulting knowledge as well to help in other critical areas like determining strategy,
architecture design, process refinement, and development. Each company will need to assess what
resources are required to successfully deploy an analytics initiative and determine what roles to

IDC Energy Insights advises analytics initiatives should be led by a data architect expert with oil and gas
knowledge and skills who can collaborate with team members to identify clear objectives and help design
and develop the underlying technical solution. One goal is to develop a framework for adding applications
and data to be integrated over the next one to five years. Over time, this framework approach will grow to
connect with other systems and sources of data to become the big data and analytics platform of the future.

©2017 IDC #US42374517 10

About IDC
International Data Corporation (IDC) is the premier global provider of market intelligence, advisory
services, and events for the information technology, telecommunications and consumer technology
markets. IDC helps IT professionals, business executives, and the investment community make fact-
based decisions on technology purchases and business strategy. More than 1,100 IDC analysts
provide global, regional, and local expertise on technology and industry opportunities and trends in
over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients
achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology
media, research, and events company.

Global Headquarters

5 Speen Street
Framingham, MA 01701
Twitter: @IDC

Copyright Notice

External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press
releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or
Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right
to deny approval of external usage for any reason.

Copyright 2017 IDC. Reproduction without written permission is completely forbidden.