Slide 1

Machine Learning for the Grid

D. Deka, S. Backhaus & M. Chertkov +
A. Lokhov, S. Misra, M. Vuffray and K. Dvijotham

DOE/OE & LANL (Grid Science) + GMLC (1.4.9 + 2.0)

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 2

D.Deka S. Backhaus M. Vuffray A. Lokhov

S. Misra K. Dvijotham (Caltech) M. Chertkov

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 3

• Intro: Overview of Challenges and Approaches

• Technical Intro: Direct and Inverse Stochastic Problem

–Machine Learning for Grid Operations

• Machine Learning for Distribution Grid

• Machine Learning for Transmission Grid

• Graphical Models & New Physics=Grid Informed Learning Tools

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 4
Data Analytics can improve resiliency in the Dynamic Grid
Changes in the modern Grid:
• Penetration of Renewables
• Storage devices Vision:
• Loads becomes active (not controlled) Design Algorithms for smart meter data to
learn and control (state of the grid)
Challenges
• Strong fluctuations/uncertainty
• Needs real-time observability, control Features:
• Millions of devices, many entities • Build upon Physics of Power flow & the
network/graph features.
New (available) Solutions • Scalable and computationally tractable
• Hardware: • Address desired (spatio-temporal) sparsity
Smart meters, PMUs, micro-PMUs
• Software/New algorithms:
Machine Learning, IoT
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 5

• Intro: Overview of Challenges and Approaches

• Technical Intro: Direct and Inverse Stochastic Problem

–Machine Learning for Grid Operations

• Machine Learning for Distribution Grid

• Machine Learning for Transmission Grid

• Graphical Models & New Physics=Grid Informed Learning Tools

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 6

Grid should operate in spite of

uncertainty & fluctuations
uncertainty:
• Graph Layout (switching of lines) + other +/- variables (transformers)
• State Estimation (consumption & production)
• Deterministic static & dynamic models (e.g. relating s=(p,q) to v)
• Probabilistic (statistical) models =>

fluctuations: 𝑣𝑗 −𝑣𝑘 ∗
𝑠𝑗 = 𝑣𝑗 𝑧
𝑗𝑘
• Renewable generators (wind & solar) 𝑘~𝑗
• loads (especially if active = involved in Demand Response) Power Flow Eqs.

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 7

Direct Deterministic Problem:

Power Flow (static/minutes)
Given:
• operational grid=graph, inductances/resistances
• injections/consumptions (for example)

Compute:
𝑣𝑗 −𝑣𝑘 ∗
• power flows over lines 𝑠𝑗 = 𝑣𝑗 𝑧
𝑗𝑘
• voltages 𝑘~𝑗
Power Flow Eqs.
• phases

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 8

Direct Stochastic Problem:

Power Flow (static/minutes)
Given:
• operational grid=graph, inductances/resistances
• Probability distribution (statistics) of injections/consumptions (for example)
-- samples are assumed drawn (from the probability distribution), e.g. i.i.d.

Compute statistics of: 𝑣𝑗 −𝑣𝑘 ∗

𝑠𝑗 = 𝑣𝑗 𝑧
𝑗𝑘
• power flows over lines 𝑘~𝑗
• voltages Power Flow Eqs.
• phases
joint & marginal probability distributions
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 9
Inverse Stochastic Problem:
Power Flow (static/minutes)
𝑣𝑗 −𝑣𝑘 ∗
Given: 𝑠𝑗 = 𝑣𝑗 𝑧
𝑗𝑘
• operational grid=graph, inductances/resistances 𝑘~𝑗
• snapshots/measurements of power flows, voltages, phases Power Flow Eqs.
• parametrized representation for statistics of
injections/consumptions, e.g. Gaussian & white
Infer/Learn:
• parameters for statistics of the injection/consumption
• operational grid=graph
Sample/Predict:
• configurations of injection/consumption
=> direct problem (compute)

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 10

Machine Learning for the Grid (at least some part) =

Automatic Solution of the Inverse Grid Problem(s)

Many flavors:
• static vs dynamic
• transmission vs distribution
• blind (black box) vs grid/physics informed
• samples vs moments (sufficiency)
• principal limits (IT) vs efficient algorithms
• ML for model reduction
• individual devices vs ensemble learning

[focus only on some of these ``complexities” in the talk]

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 11

• Intro: Overview of Challenges and Approaches

• Technical Intro: Direct and Inverse Stochastic Problem

–Machine Learning for Grid Operations

• Machine Learning for Distribution Grid

• Machine Learning for Transmission Grid

• Graphical Models & New Physics=Grid Informed Learning Tools

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 D. Deka, S. Backhaus, MC Slide 12
Machine Learning for Distribution Grid arxiv:1502.07820, 1501.04131, +

Learn
• Switch statuses
• Load statistics, line impedances

Challenges
• Nodal Measurements (voltages)
• Missing Nodes
• Information limited to households Substation
Load Nodes

Key Ideas
• Operated Radial structure
• Linear-Coupled power flow model
• Graph Learning tricks
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 D. Deka, S. Backhaus, MC Slide 13
Machine Learning for Distribution Grid arxiv:1502.07820, 1501.04131, +

Linear-Coupled power flow model:

a

equivalent to
LinDistFlow (Baran-Wu) c b

Slack Bus d
Inverse Matrices
are computable
reduced reduced
Laplacian Incidence explicitly on trees
matrix matrix
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 D. Deka, S. Backhaus, MC Slide 14
Machine Learning for Distribution Grid arxiv:1502.07820, 1501.04131, +

Key Idea:
• Use variance of voltage diff. as edge weights
𝑐 a
𝑏
𝑏 𝑏
• Minimal value outputs the nearest neighbor
𝑐
𝑎 𝑐 𝑎

Learning Algorithm:
• Min spanning tree with variance of
voltage diff. as edge weights a
𝑏
 No other information needed
 Low Complexity: 𝑐
 Can learn covariance of fluctuating loads
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 D. Deka, S. Backhaus, MC Slide 15
Machine Learning for Distribution Grid arxiv:1502.07820, 1501.04131, +

Learning with missing nodes:

• Missing nodes separated by 2 or more hops
𝑎
Learning Algorithm:
• Min spanning tree with available 𝑎
𝑏
nodes 𝑙
• Starting from leaf, check missing node

Leaf Intermediate
𝑎

𝑏 𝑏
𝑎 𝑐 𝑐

𝑐 𝑏 𝑎 𝑎
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 D. Deka, S. Backhaus, MC Slide 16
Machine Learning for Distribution Grid arxiv:1502.07820, 1501.04131, +

Learning with missing nodes & reduced information:

• Missing nodes separated by 2 or more hops
• Model reduction, ensemble (sampling distributions)

Extensions:
 Learn using end-node (household) data accounting for
 mix of active (with control) & passive
 dynamics of loads/motors and inverters
 emergencies, e.g. FIDVR
 Coupling to other physical infrastructures
 Learn 3 phase unbalanced networks
- gas/water distribution
 Learn loopy grid graph - thermal heating
 cities (Manhattan) e.g. extending the learning methodology
 rich exogenous correlations (loops to the more general ``physical flow” networks
representing non-grid knowledge)

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 17
Recently Awarded GMLC:
Topic 1.4.9 Integrated Multi Scale Data Analytics and Machine Learning for the Grid

PIs:
Emma Stewart (LBNL)
Michael Chertkov (LANL)

NL involved:
LBNL,LANL, SNL, ORNL, LLNL, NREL, ANL

• Platform
- review
- development,
- data collection
• ML and Data Analytics for Visibility
• ML and Data Analytics for Resilience
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 18
Integrating Distrib.-Level (stochastic) Loads in Frequency Control
Idea: Use distribution level Demand Response (DR), specifically ensemble of
Thermostatically Control Loads (TCL), to balance SO signal through Aggregator (A)
• Thousands of TCLs are aggregated
• SO->Aggregator (A)->TCLs [top-> bottom]
• Aggregator is seen (from above) as a “virtual GEN” Results & Work in Progress:
Goal of the study to answer the principal question: • Builds on theory & simulation experience from
• Can A follow the SO’s real-time signal as an actual GEN? Nonequilibrium StatMech & Control
• … and do it under “social welfare” conditions [our novel approach]: • Stochastic/PDE/spectral methods for analysis of
TCLs are controlled by the aggregator in a least intrusive way the PD (“driven” Fokker-Planck) were developed
• broadcast of a few control signals and cross-validated
(switching [stochastic] rates, temperature band) • Ensemble Control Scheme (“second
• probability distribution (PD) over states (temperature, +/-) quantization”= Bellman-Hamilton-Jacobi
is the control variable approach for PD) is formulated … testing.

SO Aggregator
Virtual generator =Aggregator output
signal State Estimation
SO request & Actual
Gigawatts

(PD)
Broadcast control
decisions to TCLs
ON / OFF

Cooling
Tens of Secs-Mins-to-Hours
Heating
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 19
Fault-Induced Delayed Voltage Recovery

Challenges:
• Describe FIDVR quantitatively
• Learn to detect it fast
• Predict if a developing event will or
will not lead to recovery? Cascade?
• Develop minimal preventive
emergency controls

Results:
• Reduced PDE model was developed
• Distributed Hysteretic behavior was
described
• Effects of disorder and stochasticity
were analyzed
• Effect of cascading from one feeder to
Hysteretic behavior/stalling
Work in progress: another and possibly further to transm.
• Effects of other devices (controlled or not) was investigated
• Preventive/emergency control
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 20
Optimal Distributed Control of Reactive Power via ADMM
Challenges:
• Develop algorithm to control voltage and losses in distribution
• Do it using/exploring new degree of freedom
= reactive capabilities of inverters

= Losses

Power Flow Equations

Results: performance
• The developed control (based on the LinDistFlow
representation of the Power Flows in distribution is
Validated on realistic distribution circuits
• Distributed (local measurements
+ communications with neighbors)
• Efficient = implemented via powerful ADMM)
(Alternative Direction Method of Multipliers local vs global
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 21
Resilient Distribution
Goal: Withstand the initial impact of large-
Systems (Bent, Backhaus, Yamangil, Nagarajan)
scale disruptions
Develop tools, methodologies, and algorithms to enable
the design of resilient distribution systems, using
• Asset hardening
• System expansion by adding new:
• Lines/circuit segments
• Switching
• Microgrid facilities
• Microgrid generation capacity
• Binary decisions, mixed-integer programming
problem

Model Algorithm
Relaxations

Observations
- Rural networks require larger resilience
budgets/MW served.
- Microgrids favored over
hardened lines
- Urban budget is insensitive to critical
load requirements
- Minimal hardening of lines
achieves resilience goals

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 22

• Intro: Overview of Challenges and Approaches

• Technical Intro: Direct and Inverse Stochastic Problem

–Machine Learning for Grid Operations

• Machine Learning for Distribution Grid

• Machine Learning for Transmission Grid

• Graphical Models & New Physics=Grid Informed Learning Tools

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
Machine Learning for the Transmission Grid: Slide +
D. Deka, S. Backhaus, MC 23

Ambient Stage work in progress

Learn Key Ideas

• Inertia, damping for generators • Temporal scale separation:
• Key parameters for (aggregated) loads (state slow (tens of mins) vs fast (tens of secs)
estimation) • Learning stochastic ODEs – generalized
• Statistics of spatio-temporal fluctuations stochastic swing equations
(statistical state estimation) • Spatial aggregation – incorporating PDEs
• Critical wave-modes (speed of propagation, • Green function approach (extending
damping) Backhaus & Liu 2011 beyond
detailed balance)
Challenges
• Limited measurements
• Incorporating PMU with SCADA
• On-line requirements,
e.g. need linear scaling algorithms
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
Machine Learning for the Transmission Grid: Slide +
D. Deka, S. Backhaus, MC 24

Detection & Mitigation of Frequency Events work in progress

Learn
• Detect, localize & size frequency events
in almost real time, utilizing
ambient state estimation
Challenges
• Spatio-temporaly optimal, fast measurements
• Have a fast predictive power – is an extra
control needed? when? where?

Key Ideas
• Modeling: electro-mechanical waves over
1d+ and/or 2d aggregated media,
forerunner (shortest path), interference
pattern
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Machine Learning for the Transmission Grid: GMLC 2.0 proposal Slide 25
in collaboration with
Industry-grade Implementation LBNL, PNNL, Columbia U
Goal:
• Develop data aided architecture
• Database of past events
• Combine PMU with SCADA + (aggregated) uPMU

 Grid-informed ML Analysis (just discussed) and

New Tools (advanced visualization, events
detection)
 Validation against and developing industry
standards
- Principal Component Analysis
- Existing software (PPMV, FRAT)
 Optimal sizing/sampling of PMUs

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 26

• Intro: Overview of Challenges and Approaches

• Technical Intro: Direct and Inverse Stochastic Problem

–Machine Learning for Grid Operations

• Machine Learning for Distribution Grid

• Machine Learning for Transmission Grid

• Graphical Models & New Physics/Grid Informed ML-tools

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 27

01/18-22/16
cnls.lanl.gov/machinelearning

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 𝑥𝑗 Slide 28

Graphical Models for Power Systems

(and beyond) 𝑥𝑗→𝑘 𝑓𝑗𝑘
3-bus Power System 𝑓𝑗
v-voltage auxiliary
s-(apparent) power 𝑥𝑗 = 𝑣𝑗 , 𝑠𝑗 𝑥𝑗→𝑘 = 𝑣𝑗→𝑘 , 𝑠𝑗→𝑘
exogenous nodal
graph
j k a∈ 𝑗, 𝑘, 𝑙, 𝑗 → 𝑘, 𝑘 → 𝑗, 𝑘 → 𝑙, 𝑙 → 𝑘, 𝑗 → 𝑙, 𝑙 → 𝑗 statistics
joint probability
𝑓𝑗 𝑥𝑗 , 𝑥𝑗→𝑘 , 𝑥𝑗→𝑙 = 𝐼 𝑠𝑗 , 𝑠𝑗→𝑘 + 𝑠𝑗→𝑙 ∗ 𝐼 𝑣𝑗 , 𝑣𝑗→𝑘 , 𝑣𝑗→𝑙 *Prob(𝑠𝑗 ) distribution
𝑣𝑗→𝑘 −𝑣𝑘→𝑗 ∗ 𝑣𝑘→𝑗 −𝑣𝑗→𝑘 ∗
𝑎 𝑓𝑎 𝑥𝑎
l 𝑓𝑗𝑘 𝑥𝑗→𝑘 , 𝑥𝑘→𝑗 = 𝐼 𝑠𝑗→𝑘 , 𝑣𝑗→𝑘 𝑧𝑗𝑘
∗ 𝐼 𝑠𝑘→𝑗 , 𝑣𝑘→𝑗 𝑧𝑗𝑘 P(x)~

power flows
e.g. opens it up
for new
Universal formulations for all statistical objects of Interest:
• Marginal Probability of voltage at a node - P(𝑣𝑗 )= 𝑥\𝑣𝑗 𝑃(𝑥) Machine Learning +
• Most probable load/wind at a node [instanton] solutions
keeping voltages within a domain - 𝑎𝑟𝑔𝑚𝑎𝑥𝑠𝑗 𝑥\ 𝑠 𝑃(𝑥)𝑣 ∈𝐷𝑜𝑚𝑣
𝑗
• Stochastic Optimum Power Flows (CC-, robust-) + dynamic (multi-stage) + planning ++
• Allows to incorporate multiple “complications”
UNCLASSIFIED
- Any deterministic constraints (limits, inequalities), e.g. expressing feasibility
- Any mixed (discrete/continuous) variables,
Operated by Los Alamos e.g. LLC
National Security, switching
for the U.S. Department of Energy's NNSA
Complexity of Learning: Easy vs Hard 𝑥𝑗 Slide 29

𝑥𝑗→𝑘 𝑓𝑗𝑘
𝑓𝑗
• Direct Problem – Statistical Inference
auxiliary
(marginal, partition function, ML) graph
• Inverse Problem – Learning
(graphs & factors) from samples joint probability
distribution

P(x)~ 𝑎 𝑓𝑎 𝑥𝑎

• New Story (2015) – Don’t follow the sufficient statistics path

• Focus on Sample and • Computational
Statistical Inference Complexity (directofproblem)finite GMisLearning
difficult
• Traditional Approach: Sufficient Statistics =>
• Provably efficient “local” optimization schemes (binary, pair-wise GM)
Estimate Correlations from samples
• based on ``conditioning”
• Is learning to vicinity
(inverseofproblem)hard?
a local variable
• Sample & Computational Complexity are
[Bressler 2015]
(generally) exponential
• based on ``screening” interaction through an accurate choice
of the optimization cost [M. Vuffray, UNCLASSIFIED
A. Lokhov, S. Misra, MC 2016]
• generalizable – applies directly to an arbitrary GM
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 30

Summary & Path Forward

• ML for distribution – PF-aware spanning tree algorithm to learn structure
(forest) and correlations of loads

• ML for transmission – two-state on-line learning – ambient + emergency

[learning parameters of ODEs, model reduction, waves]

• Graphical Models – proper language for variety of stochastic grid

problems, e.g. related to learning.
– Recent progress in GM learning -light, distributed, provably exact schemes – applies
naturally to the grid-specific (and other physical network-specific) ML problems.
– New relaxation ideas based on adaptive Linear Programming – Generalized Belief
Propagation schemes – complementary to ``standard” relaxations for OPF & related

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 31
LANL Grid Science Team

S. Backhaus R. Bent M. Chertkov C. Coffrin A. Zlotnik S. Misra

M. Vuffray H. Nagarajan C. Borraz-Sanchez A. Lokhov E. Yamangil D.Deka

UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 32
Back to main presentation

The Ising Model Learning Problem

(1) (1)
Generate 𝑀 i.i.d. samples 𝜎1 , … , 𝜎𝑁 Reconstruct graph and

⋮ couplings with high

of binary sequences ⋮
(𝑀) (𝑀) probability
𝜎1 , … , 𝜎𝑁
𝜇 𝜎1 , … , 𝜎𝑁 ∝ exp 𝐽𝑖𝑗 𝜎𝑖 𝜎𝑗
𝑖,𝑗 ∈𝐸

𝜎1 𝜎3 𝜎1 𝜎3
𝜎2 𝜎2

𝜎5 𝜎5
𝜎4 𝜎4
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 33
Back to main presentation

Learning is Easy in Theory & Practice

Number of variables: 𝑁 Maximum node degree: 𝑑
Number of samples: 𝑀 Coupling intensity: 𝐽𝑚𝑖𝑛 ≤ 𝐽𝑖𝑗 ≤ 𝐽𝑚𝑎𝑥

𝑒 𝑐1 𝑑𝐽𝑚𝑎𝑥 𝑒 𝑐1 𝑑𝐽𝑚𝑎𝑥
Complexity: exp 𝑐 1 𝑁 2 log 𝑁 Samples Required: exp 𝑐1 log 𝑁 Bresler (2015)
𝐽𝑚𝑖𝑛 𝐽𝑚𝑖𝑛
Structure Learning

𝑒 8𝑑𝐽𝑚𝑎𝑥 𝑒 8𝑑𝐽𝑚𝑎𝑥
Vuffray et al. (2016)
3 log 𝑁
Complexity: 2 𝑁 log 𝑁 Samples Required: 2
𝐽𝑚𝑖𝑛
Structure + Parameter
𝐽𝑚𝑖𝑛
Learning
We develop new model estimators: (Regularized) Interaction Screening Estimators
They are consistent estimators for all graphical models (Continuous variables, general interactions, etc…)
Provably optimal on arbitrary Ising Models, distributed
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA
 Slide 34
Back to main presentation

The Screening Estimator(s)

Number of samples: ∞ Number of samples: 𝑀

1 (𝑘) (𝑘)
𝑓𝑢 𝜃 = exp −𝜃𝑗𝑢 𝜎𝑗 𝜎𝑢 𝑓𝑢𝑀 𝜃 = exp −𝜃𝑗𝑢 𝜎𝑗 𝜎𝑗
𝑀
𝑗≠𝑢 𝑘=1,…,𝑀 𝑗≠𝑢

𝐽𝑢 = argmin 𝑓𝑢 𝜃 𝐽𝑢𝑀 = argmin 𝑓𝑢𝑀 𝜃 + 𝜆𝑁,𝑀 𝜃 1

𝜃 𝜃

Regularizer reduces # of samples required:

𝑂 𝑁 ln 𝑁 ⟶ 𝑂 ln 𝑁
UNCLASSIFIED
Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA