Storm Damage Data Analysis

Xiang Li
September 20, 2017

Synopsis

This data analysis project uses U.S. National Oceanic and Atmospheric Administrations (NOAA) storm
database. The goal for this project is to figure out the impact of severe weather events in population health
and economy. According to the data analysis, tornato is most harmful with respect to population health, and
flood leads to greatest economic consequences.

Data Loading

# Load the data

storm = read.csv("Q:/Desktop/StormData.csv.bz2",na.strings = c("","NA"))

# Explore the dataset

names(storm)

## [1] "STATE__" "BGN_DATE" "BGN_TIME" "TIME_ZONE" "COUNTY"

## [6] "COUNTYNAME" "STATE" "EVTYPE" "BGN_RANGE" "BGN_AZI"
## [11] "BGN_LOCATI" "END_DATE" "END_TIME" "COUNTY_END" "COUNTYENDN"
## [16] "END_RANGE" "END_AZI" "END_LOCATI" "LENGTH" "WIDTH"
## [21] "F" "MAG" "FATALITIES" "INJURIES" "PROPDMG"
## [26] "PROPDMGEXP" "CROPDMG" "CROPDMGEXP" "WFO" "STATEOFFIC"
## [31] "ZONENAMES" "LATITUDE" "LONGITUDE" "LATITUDE_E" "LONGITUDE_"
## [36] "REMARKS" "REFNUM"
summary(storm)

## STATE__ BGN_DATE BGN_TIME

## Min. : 1.0 5/25/2011 0:00:00: 1202 12:00:00 AM: 10163
## 1st Qu.:19.0 4/27/2011 0:00:00: 1193 06:00:00 PM: 7350
## Median :30.0 6/9/2011 0:00:00 : 1030 04:00:00 PM: 7261
## Mean :31.2 5/30/2004 0:00:00: 1016 05:00:00 PM: 6891
## 3rd Qu.:45.0 4/4/2011 0:00:00 : 1009 12:00:00 PM: 6703
## Max. :95.0 4/2/2006 0:00:00 : 981 03:00:00 PM: 6700
## (Other) :895866 (Other) :857229
## TIME_ZONE COUNTY COUNTYNAME STATE
## CST :547493 Min. : 0.0 JEFFERSON : 7840 TX : 83728
## EST :245558 1st Qu.: 31.0 WASHINGTON: 7603 KS : 53440
## MST : 68390 Median : 75.0 JACKSON : 6660 OK : 46802
## PST : 28302 Mean :100.6 FRANKLIN : 6256 MO : 35648
## AST : 6360 3rd Qu.:131.0 LINCOLN : 5937 IA : 31069
## HST : 2563 Max. :873.0 (Other) :866412 NE : 30271
## (Other): 3631 NA's : 1589 (Other):621339
## EVTYPE BGN_RANGE BGN_AZI
## HAIL :288661 Min. : 0.000 N : 86752
## TSTM WIND :219940 1st Qu.: 0.000 W : 38446
## THUNDERSTORM WIND: 82563 Median : 0.000 S : 37558
## TORNADO : 60652 Mean : 1.484 E : 33178

1
## FLASH FLOOD : 54277 3rd Qu.: 1.000 NW : 24041
## FLOOD : 25326 Max. :3749.000 (Other):134990
## (Other) :170878 NA's :547332
## BGN_LOCATI END_DATE END_TIME
## COUNTYWIDE : 19680 4/27/2011 0:00:00: 1214 06:00:00 PM: 9802
## Countywide : 993 5/25/2011 0:00:00: 1196 05:00:00 PM: 8314
## SPRINGFIELD : 843 6/9/2011 0:00:00 : 1021 04:00:00 PM: 8104
## SOUTH PORTION: 810 4/4/2011 0:00:00 : 1007 12:00:00 PM: 7483
## NORTH PORTION: 784 5/30/2004 0:00:00: 998 11:59:00 PM: 7184
## (Other) :591444 (Other) :653450 (Other) :622432
## NA's :287743 NA's :243411 NA's :238978
## COUNTY_END COUNTYENDN END_RANGE END_AZI
## Min. :0 Mode:logical Min. : 0.0000 N : 28082
## 1st Qu.:0 NA's:902297 1st Qu.: 0.0000 S : 22510
## Median :0 Median : 0.0000 W : 20119
## Mean :0 Mean : 0.9862 E : 20047
## 3rd Qu.:0 3rd Qu.: 0.0000 NE : 14606
## Max. :0 Max. :925.0000 (Other): 72096
## NA's :724837
## END_LOCATI LENGTH WIDTH
## COUNTYWIDE : 19731 Min. : 0.0000 Min. : 0.000
## SOUTH PORTION : 833 1st Qu.: 0.0000 1st Qu.: 0.000
## NORTH PORTION : 780 Median : 0.0000 Median : 0.000
## CENTRAL PORTION: 617 Mean : 0.2301 Mean : 7.503
## SPRINGFIELD : 575 3rd Qu.: 0.0000 3rd Qu.: 0.000
## (Other) :380536 Max. :2315.0000 Max. :4400.000
## NA's :499225
## F MAG FATALITIES INJURIES
## Min. :0.0 Min. : 0.0 Min. : 0.0000 Min. : 0.0000
## 1st Qu.:0.0 1st Qu.: 0.0 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median :1.0 Median : 50.0 Median : 0.0000 Median : 0.0000
## Mean :0.9 Mean : 46.9 Mean : 0.0168 Mean : 0.1557
## 3rd Qu.:1.0 3rd Qu.: 75.0 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :5.0 Max. :22000.0 Max. :583.0000 Max. :1700.0000
## NA's :843563
## PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## Min. : 0.00 K :424665 Min. : 0.000 K :281832
## 1st Qu.: 0.00 M : 11330 1st Qu.: 0.000 M : 1994
## Median : 0.00 0 : 216 Median : 0.000 k : 21
## Mean : 12.06 B : 40 Mean : 1.527 0 : 19
## 3rd Qu.: 0.50 5 : 28 3rd Qu.: 0.000 B : 9
## Max. :5000.00 (Other): 84 Max. :990.000 (Other): 9
## NA's :465934 NA's :618413
## WFO STATEOFFIC
## OUN : 17393 TEXAS, North : 12193
## JAN : 13889 ARKANSAS, Central and North Central: 11738
## LWX : 13174 IOWA, Central : 11345
## PHI : 12551 KANSAS, Southwest : 11212
## TSA : 12483 GEORGIA, North and Central : 11120
## (Other):690738 (Other) :595920
## NA's :142069 NA's :248769
##
##
## GREATER RENO / CARSON CITY / M - GREATER RENO / CARSON CITY / M

2
## GREATER LAKE TAHOE AREA - GREATER LAKE TAHOE AREA
## JEFFERSON - JEFFERSON
## MADISON - MADISON
## (Other)
## NA's
## LATITUDE LONGITUDE LATITUDE_E LONGITUDE_
## Min. : 0 Min. :-14451 Min. : 0 Min. :-14455
## 1st Qu.:2802 1st Qu.: 7247 1st Qu.: 0 1st Qu.: 0
## Median :3540 Median : 8707 Median : 0 Median : 0
## Mean :2875 Mean : 6940 Mean :1452 Mean : 3509
## 3rd Qu.:4019 3rd Qu.: 9605 3rd Qu.:3549 3rd Qu.: 8735
## Max. :9706 Max. : 17124 Max. :9706 Max. :106220
## NA's :47 NA's :40
## REMARKS REFNUM
## : 24013 Min. : 1
## Trees down.\n : 1110 1st Qu.:225575
## Several trees were blown down.\n : 568 Median :451149
## Trees were downed.\n : 446 Mean :451149
## Large trees and power lines were blown down.\n: 432 3rd Qu.:676723
## (Other) :588295 Max. :902297
## NA's :287433
# Keep the needed columns only
stormData = storm[,c("EVTYPE","FATALITIES","INJURIES","PROPDMG","PROPDMGEXP","CROPDMG","CROPDMGEXP")]
# Take a Look at the Data
head(stormData)

## EVTYPE FATALITIES INJURIES PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP

## 1 TORNADO 0 15 25.0 K 0 <NA>
## 2 TORNADO 0 0 2.5 K 0 <NA>
## 3 TORNADO 0 2 25.0 K 0 <NA>
## 4 TORNADO 0 2 2.5 K 0 <NA>
## 5 TORNADO 0 2 2.5 K 0 <NA>
## 6 TORNADO 0 6 2.5 K 0 <NA>
# Look at missing data pattern
library(mice)
md.pattern(stormData)

## EVTYPE FATALITIES INJURIES PROPDMG CROPDMG PROPDMGEXP CROPDMGEXP

## 279566 1 1 1 1 1 1 1
## 4318 1 1 1 1 1 0 1
## 156797 1 1 1 1 1 1 0
## 461616 1 1 1 1 1 0 0
## 0 0 0 0 0 465934 618413
##
## 279566 0
## 4318 1
## 156797 1
## 461616 2
## 1084347
The output tells us that 279,566 samples are complete, 4318 samples miss the PROPDMGEXP value,
156,797 samples miss the CROPDMGEXP value, 461,616 samples miss both the PROPDMGEXP value and
CROPDMGEXP value. We dont need to impute the missing data at this point.

3
Data Processing

# Check unique values for property damage exponent

unique(stormData$PROPDMGEXP)

## [1] K M <NA> B m + 0 5 6 ? 4 2 3 h
## [15] 7 H - 1 8
## Levels: - ? + 0 1 2 3 4 5 6 7 8 B h H K m M
# Transform exponent variable for property damage
stormData$PROPEXP[stormData$PROPDMGEXP == "-" | stormData$PROPDMGEXP == "?" | is.na(stormData$PROPDMGEXP
stormData$PROPEXP[stormData$PROPDMGEXP == "+" | stormData$PROPDMGEXP == "0"] <- 1
stormData$PROPEXP[stormData$PROPDMGEXP == "1"] <- 10
stormData$PROPEXP[stormData$PROPDMGEXP == "H" | stormData$PROPDMGEXP == "h" | stormData$PROPDMGEXP == "2
stormData$PROPEXP[stormData$PROPDMGEXP == "K" | stormData$PROPDMGEXP == "k" | stormData$PROPDMGEXP == "3
stormData$PROPEXP[stormData$PROPDMGEXP == "4"] <- 1e+04
stormData$PROPEXP[stormData$PROPDMGEXP == "5"] <- 1e+05
stormData$PROPEXP[stormData$PROPDMGEXP == "M" | stormData$PROPDMGEXP == "m"| stormData$PROPDMGEXP == "6"
stormData$PROPEXP[stormData$PROPDMGEXP == "7"] <- 1e+07
stormData$PROPEXP[stormData$PROPDMGEXP == "8"] <- 1e+08
stormData$PROPEXP[stormData$PROPDMGEXP == "B" | stormData$PROPDMGEXP == "b"] <- 1e+09
# Validation: check the transformed exponent variable
unique(stormData$PROPEXP)

## [1] 1e+03 1e+06 0e+00 1e+09 1e+00 1e+05 1e+04 1e+02 1e+07 1e+01 1e+08
# Check unique values for crop damage exponent
unique(stormData$CROPDMGEXP)

## [1] <NA> M K m B ? 0 k 2
## Levels: ? 0 2 B k K m M
# Transform the exponent variable for crop damage
stormData$CROPEXP[stormData$CROPDMGEXP == "?" | is.na(stormData$CROPDMGEXP)] <- 0
stormData$CROPEXP[stormData$CROPDMGEXP == "0"] <- 1
stormData$CROPEXP[stormData$CROPDMGEXP == "2"] <- 100
stormData$CROPEXP[stormData$CROPDMGEXP == "K" | stormData$CROPDMGEXP == "k"] <- 1e+03
stormData$CROPEXP[stormData$CROPDMGEXP == "M" | stormData$CROPDMGEXP == "m"] <- 1e+06
stormData$CROPEXP[stormData$CROPDMGEXP == "B"] <- 1e+09
# Validation: check the transformed exponent variable
unique(stormData$CROPEXP)

## [1] 0e+00 1e+06 1e+03 1e+09 1e+00 1e+02

# Calculate property damage and crop damage with exponent variables
stormData$PROPERTY_DAMAGE = stormData$PROPDMG * stormData$PROPEXP
stormData$CROP_DAMAGE = stormData$CROPDMG * stormData$CROPEXP
# Calculate Injuries, Fatalities, Property Damage and Crop Damage by event type
library(data.table)
stormData = data.table(stormData)
stormData = stormData[, list(INJURIES = sum(INJURIES),
FATALITIES = sum(FATALITIES),
PROPERTY_DAMAGE = sum(PROPERTY_DAMAGE),
CROP_DAMAGE = sum(CROP_DAMAGE)), by = "EVTYPE"]

4
# Sort number of injuries and fatalities in a descending order
temp1 = head(stormData[,1:3][order(stormData$INJURIES, decreasing = TRUE)],10)
temp2 = head(stormData[,1:3][order(stormData$FATALITIES, decreasing = TRUE)],10)
# Calculate Total
temp1$Total = rowSums(temp1[,2:3])
temp2$Total = rowSums(temp2[,2:3])

# Combine injuries and fatalities data

healthData = merge(temp1,temp2)
healthData = head(healthData[order(healthData$Total, decreasing = TRUE)],5)

Across the United States, which types of events (as indicated in the EVTYPE variable) are
most harmful with respect to population health?

# Reformat the data for ggplot2 plots

library(reshape2)
healthData = melt(healthData,id.vars = "EVTYPE")
# Create bar chart to visualize the most damaging storm type to human health
library(ggplot2)
ggplot(data = healthData, aes(x = reorder(EVTYPE, -value), y = value)) +
geom_bar(aes(fill = variable), position = "dodge", stat = "identity") +
xlab("Type of Storms") + ylab("Number of Injuries/fatalities/Total") +
ggtitle("Top 5 Harmful Storm Types to Population Health") +
theme(plot.title = element_text(hjust = 0.5)) # to center the title

Top 5 Harmful Storm Types to Population Health

100000
Number of Injuries/fatalities/Total

75000

variable
INJURIES
50000
FATALITIES
Total

25000

TORNADOEXCESSIVE HEATTSTM WIND FLOOD LIGHTNING

Type of Storms

5
According to the plot, Tornato, Excessive Heat, TSTM Wind, Flood and Lightningis are most harmful with
respect to population health across the United States.

Across the United States, which types of events have the greatest economic consequences?

# Sort numbers of Property Damage and Crop Damage in a descending order

temp3 = head(stormData[, c(1,4,5)][order(stormData$PROPERTY_DAMAGE, decreasing = TRUE)],10)
temp4 = head(stormData[, c(1,4,5)][order(stormData$CROP_DAMAGE, decreasing = TRUE)],10)
temp3$Total = rowSums(temp3[,2:3])
temp4$Total = rowSums(temp4[,2:3])

# Combine Property Damage and Crop Damage data

econData = merge(temp3,temp4)
econData = head(econData[order(econData$Total, decreasing = TRUE)],5)

# Reformat the data for ggplot2 plots

library(reshape2)
econData = melt(econData,id.vars = "EVTYPE")
# Create bar chart to visualize the most damaging storm type to economy
library(ggplot2)
ggplot(data = econData, aes(x = reorder(EVTYPE, -value), y = value)) +
geom_bar(aes(fill = variable), position = "dodge", stat = "identity") +
xlab("Type of Storms") + ylab("Damage to Property/Crop/Total (in USD$)") +
ggtitle("Top 5 Harmful Storm Types to Economy") +
theme(plot.title = element_text(hjust = 0.5)) # to center the title

Top 5 Harmful Storm Types to Economy

1.5e+11
Damage to Property/Crop/Total (in USD$)

1.0e+11
variable
PROPERTY_DAMAGE
CROP_DAMAGE
Total
5.0e+10

0.0e+00

FLOOD
HURRICANE/TYPHOON
HAIL FLASH FLOOD
HURRICANE
Type of Storms

6
According to the plot, Flood, Hurricane/Typhoon, Hail and Flash Flood cause greatest economic consequences
across the United States.

Results

According to the above data analysis, Tornato is most harmful with respect to population health, followed by
Excessive Heat, TSTM Wind, Flood and Lightning across the United States.
In terms of the greatest economic consequences, Flood is the top one, followed by Hurricane/Typhoon, Hail
and Flash Flood.