Reporting on

Artificial Intelligence

Miranda Bogen
Upturn
A quick agenda for today

- What is Upturn
- What are we talking about when we talk about AI
- The anatomy of an algorithm (and why it matters)
- Discussion/Questions
What is Upturn?

We’re a DC-based nonprofit that promotes equity and justice in the

design, use, and governance of digital technology.

We research things like:

- Algorithms and AI in the courtroom and the criminal justice
system (e.g. predictive policing)
- Facial recognition and body-worn cameras
- Discriminatory ad targeting on digital platforms and ad
transparency
- Alternative data in finance and FinTech
- AI and predictive tools in hiring (new report soon!)
- Second-order effects of autonomous vehicles
What are we talking about?
What we talk about when we talk about AI
What we think we’re talking about
What we’re often talking about
The underlying technology can take many
forms — but it doesn’t always matter.
Software and statistics

Other stuff
The anatomy of an algorithm
Criminal Justice: Biased Numbers Lead to Jail?
Facebook Trending Stories: Who Decides?
Amazon’s sexist AI recruiter?
There are a lot of interesting things to uncover
about any given AI system

Existence Purpose Constitution Impact

Policies Inputs / Outputs Training Source

Data Code
Existence: What decisions are being
automated in the first place?
Existence
Purpose: What is this automated
decision intended to accomplish?
(And is that what it’s actually doing?) Purpose
Constitution: How much of the
decisions is really automated? Where
do humans play a role, and how Constitution

much discretion do they have?

The “constitution” of a
credit scoring system...

670

DATA CREDIT CREDIT SCORE

FILE SCORING
ALGORITHM
...is different than the constitution
of credit card underwriting system
CREDIT
APPLICATION

670

DATA CREDIT CREDIT SCORE CREDIT DECISION

FILE SCORING APPLICATION
ALGORITHM ASSESSMENT

SYSTEM: CREDIT SCORING

SYSTEM: AUTOMATED CREDIT CARD UNDERWRITING

Which is different than the constitution
of a mortgage application system
MORTGAGE
APPLICATION

DATA

670

DATA CREDIT CREDIT SCORE MORTGAGE DECISION

FILE SCORING APPLICATION
SYSTEM ASSESSMENT

SYSTEM: CREDIT SCORING

SYSTEM: SEMI-AUTOMATED MORTGAGE UNDERWRITING

Impact: What are the observable
consequences of this system?
Impact
Policies: What are the rules that govern
this system and how they enforced?
Policies
Inputs/Outputs: What data does the
system take in, and what comes out?
How are the outputs used? Inputs &
Outputs
Training Data: What data was used to train
this system, and was it correct, complete
and representative? Training Data
Source Code: Can we learn anything from
technical descriptions of a given system?
Source Code

Traditional Software Predictive Model

Back to those stories . . .
Criminal Justice: Biased Numbers Lead to Jail?

Courts are using The models are Judges maintain

predictive models meant to predict some discretion
risk of recidivism

Disparate error Output: Mix of police data

rates by race numerical score and intake interview
Facebook Trending Stories: Who Decides?
STAGE 1

Accused of bias Mix of recommender Policies may not

against conservative system and human be clear enough
outlets editors to ensure balance
STAGE 2

More false stories Entirely automated Should this

system exist?
Amazon: What Happened?

The system aimed It penalized Because training

to highlight passive women data was on past
candidates and candidates by recruitment
predict who downgrading decisions which
recruiters would certain words included mostly
prefer men
Questions?

Existence Purpose Constitution Impact

Policies Inputs / Outputs Training Source

Data Code