You are on page 1of 88

Energy Management Agents Understanding the Energy Consumer

Matthew Bruchon, Edward Burnell, Marnix Hollander

Presentation Outline
The Energy Management Agent (EMA) Role in the evolving electricity market Current state Understanding the energy customer
The home Energy storage Occupants behavior Classifying household energy consumption

Implications of Energy Management Agents


2

Reference: MIT Energy Box Project


The Energy Box: Comparing Locally Automated Control Strategies of Residential Electricity Consumption under Uncertainty
Dan Livengood, ESD PhD 2011 Advisors: Richard Larson, Steve Graves, Jim Kirtley, Steve Connors

Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing
Woei Ling Leow, ESD PhD 2012 Advisors: Richard Larson, Jim Kirtley, Roy Welsch, Steve Connors

The Energy Management Agent

Energy Management Agent (EMA)


Processor dedicated to unobtrusively providing continuous monitoring and optimized control of energy consumption
Local hardware or remotely hosted software Isolated or connected Drop in Control of energy using devices, as well as generation

Primary goal is to reduce energy cost through:


Load shifting and demand management Constantly optimized control, independent of user
Machine learning Zonal control Occupancy prediction

Energy Management Agent (EMA)


Processor dedicated to unobtrusively providing continuous monitoring and optimized control of energy consumption
Local hardware or remotely hosted software Isolated or connected Drop in Control of energy using devices, as well as generation

Primary goal is to reduce energy cost through:


Load shifting and demand management Constantly optimized control, independent of user
Machine learning Zonal control Occupancy prediction

Evolution of the grid market


Legacy Supply follows demand, static pricing
Coordinate power generation and distribution to optimally match demand Build capacity to match peak demands

Demand Response ISO controls demand


Create secondary market for negawatts to reduce peaking load for more efficient generation

Direct supply and demand with consumers


Consumers manually optimize between cost and comfort

EMA: automated responsive consumer demand


Technology replaces active manual intervention Must integrate with human behavior and existing infrastructure

Understanding the customer


In order to act effectively as agent, the EMA must understand:
Thermal properties of the home Energy storage options Human behavior
Comfort Occupancy patterns and lifestyle

Through data and machine learning techniques, EMAs can classify energy customers behavior, adding nuance to the optimization strategy
8

EMA Assumptions

Model Predictive Control

Specified zones (heating / appliances)


Environmental inputs (weather, RTP)
10

Heating Zones

Zone states are related by transfer functions idences and modeled by the EMA learned

Zoning and Occupancy-Moderation for Residential Space-Conditiong, Leow, Woei Ling.

11

Regression Modeling

No memory, but there is storage

12

Regression Modeling Example

13

Regression Modeling Example

14

Regression Modeling Example

15

Regression Modeling Example

16

Autoregression

c is a constant is the constant parameter vector is gaussian white noise of mean 0 Each state depends on arbitrarily many previous states, providing memory
17

Autoregression Example
AR(p) is an autoregressive series that looks p time steps back AR(0) is just white noise AR(1) is an integration if 0<<1, blows up if >1, and oscillates if <0 In general, AR(p) can oscillate at period p or less if the coefficients have different signs, and blows up if the roots of the characteristic polynomial have a magnitude greater than 1
18

Autoregression Modeling

Can be solved with a shifted feature vector Feature dimensionality as length(x)p Error drops off quickly, then flattens

AutoRegression Analysis, Paul Bourke: http://paulbourke.net/miscellaneous/ar/

19

Autoregression Modeling Example

20

Autoregression Modeling Example

21

Autoregression Modeling Example

22

Autoregression Modeling Example

23

Autoregression Modeling Example

24

Autoregression Summary
Memory makes sense for a thermal system
Modeling air mixing and heating furniture

Autoregression cant be backsolved for MPC

as easily

25

Energy Storage
Thermal mass
Shiftable loads Batteries All storage requires time-variance in the cost of fuel, because they can all be seen as timeshifting loads
26

Thermal Mass
Heating zones
incl. appliances like refrigerators, hot water heaters, etc.

Slow to store Leaky Comfort could be an issue Automatically optimized by the MPC

27

Batteries
E is the energy going from / to the battery

s is the storage state is the decay rate is the charging / discharging efficiency
The Intertemporal Utility of Demand and Price-Elasticity, Roozbehani et al.

28

Shiftable Loads
Incorporate the disutility p(k) associated with backlogging demand into the cost function
Example shiftable appliances:
Dishwasher

Dryer
Computer

The Intertemporal Utility of Demand and Price-Elasticity, Roozbehani et al.

29

Future Work
Integration of thermal and electrical systems
Ex: refrigerator heats the surrounding room, cools its interior Hot water heaters demand and temperature could fit occupancy and comfort models

Statespace analysis of the regression models stability Modeling of solar thermal systems, radiant heating
30

Challenges in Modeling Human Behavior


How to quantify the comfort preferences of an end user? How to predict occupancy at a given time? How to classify types of behavior patterns?

Quantifying Comfort Preferences


Overall goal of EMA is to minimize some cost function across time:

Livengood, Dan. The Energy Box: Comparing Locally Automated Control Strategies of Residential Electricity Consumption under Uncertainty. 2011.

Quantifying Comfort Preferences


Overall goal of EMA is to minimize some cost function across time:

What form should the cost of comfort take?


Livengood, Dan. The Energy Box: Comparing Locally Automated Control Strategies of Residential Electricity Consumption under Uncertainty. 2011.

Quantifying Comfort Preferences


mcool = 1 mwarm = 2

StAC < 74oF tempmin.comfort = 74oF k: scaling parameters m: shape parameters

StAC > 78oF tempmax.comfort = 78oF

$
kcool

$
kwarm

Livengood, Dan. The Energy Box: Comparing Locally Automated Control Strategies of Residential Electricity Consumption under Uncertainty. 2011.

Predicting Zonal Occupancy


Option 1: Nave occupancy learning (frequency counter) Robust in the presence of noisy data
p(7am) = 0.5
p(7am) = 0.15

p(7am) = 0.35

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Predicting Zonal Occupancy


Option 2: Markov model At a given hour of the day, calculate the likelihood of an occupants next location depending on his current location
p= 0.6

p=0.1

P=0.3 p=0.2
p=0.1

p= 0.5

p=0.4

p= 0.7

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

P=0.1

Classifying Types of Behavior Patterns


Human behavior is highly variable:
Between people Between days for a given person

In many sustainability problems related to human behavior, methods to classify people or behavior types are useful

Classifying Types of Behavior Patterns


Expectation-Maximization (E-M) Algorithm
Reduces complexity by introducing latent variables Ex: mixture model where each data point has a latent variable: its label

Self-Organizing Maps K-Means Clustering

en.wikipedia.org/wiki/K-means_clustering

E-M Algorithm: Notation


Data set D with two parts:
Observed data X Latent data Z

Generative parameter Complete likelihood of D


p(D|) = p(X,Z|)

Marginal likelihood of X
p(X|) =
p(X,z|)

Dahua Lin: An Introduction to Expectation-Maximization. <http://courses.csail.mit.edu/6.867/wiki/index.php?title=6.867_Machine_Learning_%282011_Fall%29>

E-M Algorithm: Objective


Maximum likelihood estimation (MLE) of |X Find parameter that maximizes marginal log-likelihood
= arg max log[p(X|)]

Dahua Lin: An Introduction to Expectation-Maximization. <http://courses.csail.mit.edu/6.867/wiki/index.php?title=6.867_Machine_Learning_%282011_Fall%29>

E-M Algorithm: Steps


1. Initialize (0) to a random value, or a best guess 2. For t = 1,2,, repeat:
a) Expectation step b) Maximization step

3. Stop iterating once a convergence criterion is met. Examples:


| (t) - (t-1) | < | log[p(X|)](t) -log[p(X|)](t-1) | <

Dahua Lin: An Introduction to Expectation-Maximization. <http://courses.csail.mit.edu/6.867/wiki/index.php?title=6.867_Machine_Learning_%282011_Fall%29>

E-M Algorithm: Expectation Step


Assume the current parameter values, (t-1), are correct Use (t-1) to compute q(t)(Z), the posterior distribution of Z at t:
q(t)(Z) = p(Z|X; (t-1))

Dahua Lin: An Introduction to Expectation-Maximization. <http://courses.csail.mit.edu/6.867/wiki/index.php?title=6.867_Machine_Learning_%282011_Fall%29>

E-M Algorithm: Maximization Step


Find a new value of (t) to be used at the next iteration. The new value should maximize the expectation of the complete log-likelihood with respect to q(t)
(t) = arg max Eq(t)( log p(X,Z|) ) = arg max
(t) q (z) log p(X,z|)

Dahua Lin: An Introduction to Expectation-Maximization. <http://courses.csail.mit.edu/6.867/wiki/index.php?title=6.867_Machine_Learning_%282011_Fall%29>

E-M Algorithm: Caveat


Convergence to a global maximum is not guaranteed in non-convex space. There are ways to address this:
Random restart Simulated annealing

http://en.wikipedia.org/wiki/Hill_climbing

E-M Algorithm: Application to Energy Box Simulator (Livengood, Leow)


Goal: to run Monte Carlo simulations of Energy Box zonal space conditioning given various stochastic inputs
Climate, electricity pricing, local generation, etc.

Simulations must also include configurable human behavior parameters


Randomness and similarity of occupancy Category of lifestyle (stay-at-home or worker)

How to get data that maps to these parameters?


Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Occupancy Patterns for Use in Modeling


American Time Use Survey
12,000 respondents surveyed Ex: food and drink preparation

Proxy data set of occupancy patterns created from survey data


shopping = out of house sleeping = in bedroom

Final proxy data set includes roughly 100,000 total occupancy patterns
Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Structure of Occupancy Patterns


Occupancy vectors v {0,1,K}N
K: number of zones N: number of time periods (24 per day)

Example
v = [1 1 1 1 1 1 1 2 0 0 0 0 0 0 0 0 0 0 0 2 2 2 1 1] 0: out of house 1: bedroom 2: living room or dining room

Occupancy vectors must be classified into categories of lifestyle (e.g., the above is a worker)
Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Classifying Occupancy Vectors


How to classify occupancy vectors into lifestyle categories such as:
Stay at home Day worker

Complexity is a non-trivial concern


100,000 categorical random data points 24-dimensional

Missing data
Occupancy vectors arent labeled
Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Classifying Occupancy Vectors


How to classify occupancy vectors into lifestyle categories such as:
Stay at home Day worker

Complexity is a non-trivial concern


100,000 categorical random data points 24-dimensional

Missing data
Occupancy vectors arent labeled

An Expectation-Maximization (EM) algorithm


Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

EM Algorithm Application
Latent data Z: the lifestyle category label Generative parameters :
p(z), the likelihood of each lifestyle category pz,i(k), the likelihood of occupying zone k for lifestyle category z at time period i

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Other Parameters in Monte Carlo Modeling


In addition to the lifestyle category variables, there is also a need to simulate different levels of:
Randomness: how much a given persons occupancy patterns vary Similarity: for multi-occupant houses, how similar the occupants schedules are

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Simulated Results

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Simulated Results

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Simulated Results

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Simulated Results

How can this gap be closed?


Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Improving Performance via Connected Devices


Remote Schedule Updates (RSU)

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Improving Performance via Connected Devices


Performance Improvement (%) of remote schedule updates over use of expected return time to the house (6pm)
(Uniformly distributed return time, from 2pm to 11pm)

20%

RTP Flat Price

10%

0%
Resident prefers Cost savings over Comfort Resident prefers balanced Comfort and Cost savings Resident prefers Comfort over Cost savings

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Improving Performance via Connected Devices


Future work: learning occupancy based on
Personal online calendars TV listings Sports schedules

Leow, Woei Ling. Zonal and Occupancy-Moderation for Residential Space-Conditioning under Demand-Driven Electricity Pricing. 2012.

Classifying Electricity Customers


Customer behaviors
Occupancy Load Profile / consumption

House type and construction EMA can adjust control strategy based on occupant and house Recommend improvements for improved energy efficiency
60

K Means Classification
Clustering technique N observations gathered into K clusters Goal is to minimize sum of squares distance within each cluster to the cluster centroid
Each S is a set of clustered observations
=1 2

Caveats
Computationally difficult (NP-hard) Requires specification of K clusters Heuristic, not guaranteed to find global maximum
61

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
62

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
63

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
64

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
65

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
66

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
67

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
68

The K-means algorithm


1. Place K initial points randomly in the space (initial centroids) 2. Assign each data point to the closest centroid 3. Calculate the centroid of each group of points to get the new centroid 4. Repeat 2 and 3 until centroid no longer moves

Observation space
69

The K-means algorithm


Goal

=1

Random initial means : 1 , ,

Assign observations
= : 1

Update means
=
1

Converged when assignments do not change


70

Self Organizing Maps (SOMs)


Relatively new visualization technique for high dimension data sets Displays similarity and disparity in a data set Based on neural networks Naturally forms clusters around data set Used for classifying electrical consumers by home properties, behavior, or usage patterns

Chicco, G., Napoli, R., Piglione, F., Postolache, P., Scutariu, M., & Toader, C. (2004). Load PatternBased Classification of Electricity 71 Customers, 19(2), 1232-1239.

SOMs Algorithm
Components Observations / Training Data
N observation input vectors of k dimensions E.g. for RGB color (255,100,0)

Node network
Each node consists of topological position (x, y coordinate) and vector of weights of k dimension

Randomly generated node network with k = 3 dimensions


72

Basic SOMs Algorithm


1. Initialize node network 2. Randomly select observation and find closest node 3. Scale neighboring nodes 4. Repeat 2-4

73

Basic SOMs Algorithm


1. Initialize node network 2. Randomly select observation and find best matching node 3. Scale neighboring nodes 4. Repeat 2-4 until
Randomly generated node network with k = 3 dimensions
74

Basic SOMs Algorithm


1. Initialize node network 2. Randomly select observation and find closest node 3. Scale neighboring nodes 4. Repeat 2-4 From observation set, randomly select input vector V Search all nodes, W, using Euclidean distance formula
min
=1

75

Basic SOMs Algorithm


1. Initialize node network 2. Randomly select observation and find closest node 3. Scale neighboring nodes 4. Repeat 2-4 1. Determine neighbors
1. Typically Gaussian neighborhood function with width ()

2. Scale neighboring nodes weight


1. 2.
+ 1 = + ()( ) () =
1.
2 (t)exp( 2 ) 2 ()

t is learning rate

factor, monotonically decreasing 0 < t < 1


76

Basic SOMs Algorithm


1. Initialize node network 2. Randomly select observation and find closest node 3. Scale neighboring nodes 4. Repeat 2-4
1. Neighborhood size and learning-rate factor both decrease monotonically with iterations 2. Winning nodes and neighbors are rewarded with larger adjustments towards sampled vector

77

Using clustering to classify electricity customers


Load profile
Understand usage profiles and habits Allow EMA or ISO to identify high cost customer activity Set control algorithms or pricing structures appropriately Input is vector of energy usage

Chicco, G., Napoli, R., Piglione, F., Postolache, P., Scutariu, M., & Toader, C. (2004). Load Pattern78 Based Classification of Electricity Customers, 19(2), 1232-1239.

Classify customers infrastructure


EMAs can use information on building type, thermal properties, and location to classify Gathered information can be used to provide recommendations and improved control Comparison between nearby homes provides information and incentive to customers

79

Salo, Finland study


Goal was to create comparison groups based on building characteristics 8,000 electrical customers 27 input variables
Building construction year # Floors # Rooms Material: concrete, brick, wood, other Heating: water, air, electric, wood, none Features: solar panel, sauna, AC, etc

Rsnen, T., Ruuskanen, J., & Kolehmainen, M. (2008). Reducing energy consumption by using self-organizing maps to create more personalized electricity use information. Applied Energy, 80 85(9), 830-840. doi:10.1016/j.apenergy.2007.10.012

Data chain
Used SOM to compress and visualize data Ran k means cluster for several different k values Clustered comparison groups and compared energy consumption

81

Individual comparison

Rsnen, T., Ruuskanen, J., & Kolehmainen, M. (2008). Reducing energy consumption by using self-organizing maps to create more personalized electricity use information. Applied Energy, 82 85(9), 830-840. doi:10.1016/j.apenergy.2007.10.012

Comparison groups
Comparing 12 clusters by energy use by floor area Allows EMA to compare performance to those groups or areas which are least or most efficient

83

Energy Management technology is critical to impact of smart grid

84

Critical factors for energy management agent adoption


Energy Prices
Real Time Pricing (RTP)

Infrastructure readiness
Distributed generation capability Device/appliance connectedness

Customer behavior and incentives


Tied to energy prices Education and willingness to trade comfort for savings
85

Implications on grid
Generation
Reduce peak and overall demand, significantly reducing infrastructure investment Balance demand with non-dispatchable energy sources Potential for oscillation in price and demand Price event triggers simultaneous demand change

Household
Better control of home energy use Savings without costly upgrades Ease integration of distributed renewables

Machine learning has the potential to improve the performance of EMAs and reduce overall strain on the electrical grid
86

Backup

87

Potential Stability Issues of Time-varying Pricing Tariffs

Image from Agent-based Control for Decentralised Demand Side Management in the Smart Grid by Ramchurn et. al., University of Southampton, UK, 2011

88

Potential Stability Issues of Timevarying Pricing Tariffs


Electricity Demand (GW) * Simulation with moderate price elasticity of demand

Image from Volatility of Power Grids under Real-Time Pricing by Roozbehani, Dahleh and Mitter, MIT, 2011

89

Conclusions
Energy Box can take many forms but provides cost saving to the consumer through constant monitoring and control Potentially significant benefits to consumer and grid Leveraging machine learning to identify occupancy patterns allows for time specific energy use Zonal control allows for directed HVAC and combined with occupancy patterns can significantly reduce energy use
90