Responsible Data Science

Introduction xix
 
Part I Motivation for Ethical Data Science and Background Knowledge 1
 
Chapter 1 Responsible Data Science 3
 
The Optum Disaster 4
 
Jekyll and Hyde 5
 
Eugenics 7
 
Galton, Pearson, and Fisher 7
 
Ties between Eugenics and Statistics 7
 
Ethical Problems in Data Science Today 9
 
Predictive Models 10
 
From Explaining to Predicting 10
 
Predictive Modeling 11
 
Setting the Stage for Ethical Issues to Arise 12
 
Classic Statistical Models 12
 
Black-Box Methods 14
 
Important Concepts in Predictive Modeling 19
 
Feature Selection 19
 
Model-Centric vs. Data-Centric Models 20
 
Holdout Sample and Cross-Validation 20
 
Overfitting 21
 
Unsupervised Learning 22
 
The Ethical Challenge of Black Boxes 23
 
Two Opposing Forces 24
 
Pressure for More Powerful AI 24
 
Public Resistance and Anxiety 24
 
Summary 25
 
Chapter 2 Background: Modeling and the Black-Box Algorithm 27
 
Assessing Model Performance 27
 
Predicting Class Membership 28
 
The Rare Class Problem 28
 
Lift and Gains 28
 
Area Under the Curve 29
 
AUC vs. Lift (Gains) 31
 
Predicting Numeric Values 32
 
Goodness-of-Fit 32
 
Holdout Sets and Cross-Validation 33
 
Optimization and Loss Functions 34
 
Intrinsically Interpretable Models vs. Black-Box Models 35
 
Ethical Challenges with Interpretable Models 38
 
Black-Box Models 39
 
Ensembles 39
 
Nearest Neighbors 41
 
Clustering 41
 
Association Rules 42
 
Collaborative Filters 42
 
Artificial Neural Nets and Deep Neural Nets 43
 
Problems with Black-Box Predictive Models 45
 
Problems with Unsupervised Algorithms 47
 
Summary 48
 
Chapter 3 The Ways AI Goes Wrong, and the Legal Implications

GRANT FLEMING is a Data Scientist at Elder Research Inc. His professional focus is on machine learning for social science applications, model interpretability, civic technology, and building software tools for reproducible data science.
 
PETER BRUCE is the Senior Learning Officer at Elder Research, Inc., author of several best-selling texts on data science, and Founder of the Institute for Statistics Education at Statistics.com, an Elder Research Company.