9/24/2016

How to Get a Job In Deep Learning

MENU

How to Get a Job In Deep

Learning
22 SEPTEMBER 2016

If youre a software engineer (or someone whos learning the

craft), chances are that youve heard about deep learning
(which well sometimes abbreviate as DL). Its an interesting
and rapidly developing

eld of research thats now being used

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

1/12

9/24/2016

How to Get a Job In Deep Learning

in industry to address a wide range of problems, from image

classi cation and handwriting recognition, to machine
translation and, infamously, beating the world champion Go
player in four games out of

ve.

A lot of people think you need a PhD or tons of experience to

get a job in deep learning, but if you're already a decent
engineer, you can pick up the requisite skills and techniques
pretty quickly. At least, that's our philosophy. (So even if you're
a beginner with deep learning, you're welcome to apply for one
of our open positions.)
Important point: You need to have motivation and be able to
code and problem solve well. That's about it.
Here at Deepgram were using deep learning to tackle the
problem of speech search. Basically, were teaching machines
to listen to and remember the contents of recorded
conversation, phone calls, online videos, podcasts, and
anything else that has audio of people talking. But listening is
just half of it. Were also teaching machines to recall key words
and phrases from these recordings in a similar way to how our
brains search for memories of conversation: by the sound of
those key words and phrases you type into the search bar. (In
case you havent already played around with Deepgram yet, we
have a little demo to show some of its capabilities.)
Getting involved in deep learning may seem a bit daunting at
rst, but the good news is that there are more resources out
there now than ever before. (Theres also a huge, pent up
demand for engineers who know how to implement deep
learning in software.) So, if you want to get yourself a job in
deep learning but need to get yourself up to speed
http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

rst, let this

2/12

9/24/2016

How to Get a Job In Deep Learning

be your guide! (If you already know a lot about deep learning
and youre just looking for information about getting a job in
the

eld, skip to the bottom.)

What Is Deep Learning?

In a nutshell, deep learning involves building and training a
large arti cial neural network with many hidden layers
between the input side of the network and the output side. It's
because of these many hidden layers that we call this kind of
neural network "deep". Deep neural networks have at least
three hidden layers, but some neural networks have hundreds.

Neural networks are complex statistical models that allow

computers to create a remarkably accurate abstract
representation of information. What kind of information, you
ask? Like we mentioned, Deepgram's deep neural network is
speci cally trained to "understand" and act upon spoken word
data, but deep neural networks have been used in plenty of
other contexts, from detecting cancers in medical scans to
forecasting energy prices and modeling the weather.

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

3/12

9/24/2016

How to Get a Job In Deep Learning

There are a number of notable players in the deep learning

space. On the academic side, the Geo rey Hinton's lab at
University of Toronto, Yann LeCun's group at New York
University and Stanford's AI lab are some of the major leaders
in deep learning research. On the private side, Google has led
the way in applying deep learning to search and computer
vision, and Baidu's Chief Scientist, Andrew Ng, is a major
contributor to the scienti c literature around deep learning on
top of being the cofounder of Coursera.
Why is deep learning so accessible today, even for newcomers
to the

eld? There are two primary factors. First, computing

hardware is now fast and cheap enough to make deep learning

accessible to just about anyone with a decent graphics card in
their PC. (In our own testing, we've found that one GPU server
is about as fast as 400 CPU cores for running the algorithms
we're using.) Second, new open source deep learning platforms
like TensorFlow, Theano and Ca e make spinning up your own
deep neural network fairly easy, especially when compared to
having to build one from scratch.
There's a lot more to deep learning, of course, but that's what
this guide is for!

What You Should Already Know

Before Diving Into Deep Learning
Speaking of math, you should have some familiarity with
calculus, probability and linear algebra. All will help you
understand the theory and principles of DL.

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

4/12

9/24/2016

How to Get a Job In Deep Learning

Obviously, there is also going to be some programming

involved. As you can see from this list of deep learning
libraries, most of the popular libraries are written in Python
and R, so some knowledge of Python or R would also be helpful.
If you need to bone up on your math or programming skills,
there are plenty of very high quality resources online to use.
Also, as we mentioned above, having a decent graphics card (or
accessing a GPU instance through a cloud computing platform
like Amazon Web Services or one of the other hosting providers
listed here).

Where To Learn About Deep

Learning
Talks and Articles About DL
If youre brand new to the

eld and youre looking for some

high-level explanations of the concepts behind deep learning

without getting lost in the math and programming aspects,
there are some really informative talks out there to familiarize
yourself with the concepts and terminology.
http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

5/12

9/24/2016

How to Get a Job In Deep Learning

The University of Wisconsin has a nice, one-webpage

overview of neural networks.
Brandon Rohrer, Microsofts principal data scientist, gave a
talk that aims to explain and demystify deep learning
without using fancy math or computer jargon at the Boston
Open Data Science conference. He has the video and slides
on this page.
Deep learning pioneer Geo rey Hinton was the

rst to

demonstrate the use of backpropogation algorithms for

training deep neural networks. He now leads some of
Googles AI research e orts when hes not attending to
academic responsibilities at the University of Toronto. He
gave a brief but illuminating talk on "How Neural Networks
Really Work" that we really like. You can also

nd a list of

his papers on DL without much math on his faculty page.

Steve Jurvetson, the founding partner of DFJ, a large Silicon
Valley venture capital

rm, led a panel discussion at the

Stanford Graduate School of Business on the subject. If

youre interested in learning about deep learning from the
perspective of some startup founders and engineers
implementing DL in industry, check out the video.
If you just want to dive right in and are comfortable with
some math, simple code examples, and discussions of
applying DL in practice check out Stanford grad Andrej
Karpathys blog post on "The Unreasonable E ectiveness of
Recurrent Neural Networks".

Online Courses
http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

6/12

9/24/2016

How to Get a Job In Deep Learning

If youre the type of person who enjoys and gets a lot out of
taking online courses, youre in luck. There are several good
courses in deep learning available online.
Andrew Ngs Stanford course on machine learning is very
popular and generally well-reviewed. Its considered one of
the best introductory courses in machine learning and will
give you some rigorous preparation for delving into deep
learning.
Udacity has a free, ten week introductory course in machine
learning that focuses on both theory and real-world
applications. Again, its a decent preparatory course for
those interested in eventually pursuing deep learning.
Caltechs Yaser S. Abu-Mostafas self-paced course,
"Learning From Data" is less mathematically dense, but its
still a very solid survey of machine learning theory and
techniques.
Andrej Karpathys "CS231n: Convolutional Neural Networks
for Visual Recognition" at Stanford is challenging but welldone course in deep neural networks, and the syllabus and
detailed course notes are available online.
Geo rey Hintons course on "Neural Networks for Machine
Learning" is good, and its taught by one of the godfathers
of the

eld.

Books
Maybe online courses arent your thing, or maybe you just
prefer reading to watching lectures and reviewing slide decks.

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

7/12

9/24/2016

How to Get a Job In Deep Learning

There are a few good books out there that are worth checking
out. We recommend:
Andrew Trasks Grokking Deep Learning aims to give a
really accessible, practical guide to deep learning
techniques. If you know some Python and passed algebra in
high school, youre 100% prepared for this book.
Ian Goodfellow, Yoshua Bengio and Aaron Courvilles book,
Deep Learning, which will be published by MIT Press. For
now, there is an early version of the book available for free
online, plus lecture slides and exercises.

Other Learning Resources & Websites

Metacademy is a very cool site with a very, very solid
overview of deep learning and tons of links to speci c topics
in the

eld.

Denny Britz of the Google Brain team has a pretty

comprehensive glossary of deep learning terminology on his
website, WildML. He also curates a weekly newsletter that
contains links to both technical and non-technical articles
about machine learning and deep learning.

Where to Practice Deep Learning

Once you have some of the basics under your belt, youll be
ready to sink your teeth into some actual data and exercises.
Here are a few websites where you can

nd sample datasets

and coding challenges:

Kaggle has a fairly large collection of datasets ranging from
SF/Bay Area Pokemon Go spawn points to Y Combinator
http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

8/12

9/24/2016

How to Get a Job In Deep Learning

companies to the giant text corpus that is Hillary Clintons

leaked emails.
UC Irvine also has a big collection of datasets to train deep
neural networks on.
For those who like python notebooks, there's a nice 5 part
tutorial put together by Alexander Johansen on how to use
TensorFlow (with links to other DNN library tutorials).

Where to Find People Interested

in Deep Learning
Regardless of whether youre a rank amateur or a PhD at the
bleeding edge of deep learning research, its always good to
connect with the community. Here are some places to meet
other people interested in deep learning:
You should see if your city has a machine learning or deep
learning group on a site like Meetup.com. Most major cities
have something going on.
There are several online communities devoted to deep
learning and deriving insights from data:
Deeplearning.net is one of the major online hubs for
deep learning related information. Resources include: a
comprehensive reading list, a list of deep learning
research labs, and a collection of nifty demos so you can
see DL in practice.
Datatau is kind of like Hacker News, but speci cally
focused on data and machine learning. The comments
http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

9/12

9/24/2016

How to Get a Job In Deep Learning

sections arent very active but there are new links

posted regularly.
There is a machine learning subreddit thats fairly
active. (They also have a very helpful wiki with even
more resources.) The deep learning subreddit is a little
quieter.
Theres a surprisingly active Google Plus group devoted
to deep learning with over 30,000 members. (Who knew
people still used Google Plus?)

Where To Find A Job in Deep

Learning
The good news is that basically everyone is hiring people that
understand deep learning.
You probably know all the usual places to go looking: AngelList,
the monthly "Whos Hiring" thread on hacker news, the
StackOver ow jobs board, and the dozens of general-purpose
job search sites.
One of the few jobs boards that specializes in DL positions is
found at Deeplearning.net, and there is a more general machine
learning jobs board on Kaggle.
These are de nitely great points. Most companies looking for
DL/ML talent aren't interested in setting up HR hoops for the
applicant to jump through.

What To Do When Applying

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

10/12

9/24/2016

How to Get a Job In Deep Learning

Companies want to see if you did cool stu

before you applied

for the job.

If you didn't then you won't get an interview, but if you did
then you have a chance no matter what your background is. Of
course, the question of "what is cool stu ?" comes up.
If your only experience is building small projects with only a
little bit of success, that probably won't do it (although it might
work for larger companies, or companies that need light
machine learning performed). But if it is:
"I built a twitter analysis DNN from scratch using Theano and
can predict the topic with pretty good accuracy:
here's the accuracy I achieved
here's a link to my write up
here's a link to github for the code"
That type of thing will get you in the door. Then you can work
your magic with coding chops and problem solving skills during
the interview.
Deepgram is also hiring, so if youre interested in solving hard
problems and building great tools, give us a holler!

This article was written in collaboration with Jason D. Rowley.

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

11/12

9/24/2016

How to Get a Job In Deep Learning

Scott Stephenson

Share this post

Dark matter physicist and Co-Founder of Deepgram

we are the world's most accurate audio search.

San Francisco, CA

http://www.deepgram.com

YOU MIGHT ENJOY

Cheap Automatic Speech

Transcription
Industry Price: $1.50/hr Deepgram Price: $0.75/hr We'll match
anybody's price. We don't care. We'll do

Deepgram Blog 2016

http://blog.deepgram.com/how-to-get-a-job-in-deep-learning/

Proudly published with Ghost

12/12