E6895 Advanced Big Data Analytics:

Safest Route Prediction in New York City

Team Members: Gabriel Thomas (gtm2122), Anubha
Bhargava (ab3955)

May 12, 2016

1

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Motivation
Develop a useful, problem solving tool that displays the safest walking
route in a city

The application will be designed for those unfamiliar with a city or

uncomfortable walking at night.
The application will
Display the safest route on a map interface
Provide walking directions to the user

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Datasets, Software Languages and APIs Used

Datasets: NYPD Major Felony Incidents Crime Dataset, spotcrime.com
Software Languages and packages: Python, JavaScript, HTML, Spark
APIs: Yelp, imaplib, GoogleMaps, MapQuest

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Front-End Interface

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Visualization Map

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Algorithm

1. Get 24 hour shops from Yelp API

2. Load historical crime datasets and Spotcrime.com data
3. Get circle around the origin and destination
4. Get all the coordinates for 24 hour shops within this area
5. Use Gaussian Mixture Modelling to fit the crime locations
6. Use the mixed Gaussian Multivariate Distribution on the locations of the
24 hour shops to check safety
7. Use these locations as waypoints to plot the route in Google Maps and
provide walking directions
8. Plot the crime data on using a visualization plot on Google Maps

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Safest Walking Route Prediction

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Safest Walking Route Prediction

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Safest Walking Route Prediction

Google Maps Original Route:

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Safest Walking Route Prediction

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Safest Walking Route Prediction

Google Maps Original Route:

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Opportunities for Future Development

Instead of only using 24 hour shops, using additional waypoints would
aid in determining safety.
A more precise technique would be determining the safety at the
coordinates of each leg.
The current algorithm holds each crime at the same weight, but a
more robust algorithm would hold more dangerous crimes with more
weight.
Each crime is currently modeled as a independent and identically
distributed Gaussian mixture model.
It would be useful to experiment with different mixture models.

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University

Questions?

E6895 Advanced Big Data Analytics Final Project

2015 CY Lin, Columbia University