14: Unsupervised Learning, etc.

Objectives

Reading

Unsupervised Learning

• Skim the Wikipedia article on cluster analysis
  • What is a “clustering”?
  • What is the difference between “hard clustering” and “soft clustering”?
  • Evaluation: What is the difference between “internal” and “external” evaluation?
  • Applications: what are some applications of clustering in two diferent fields of interest to you?
• k-means clustering
  • Wikipedia article
  • Scikit-learn documentation
  • Scikit-learn example
  • Can you explain what’s wrong in each subplot of the “Unexpected KMeans clusters” figure?

Interpretation and Explainability

• Interpretable Machine Learning

Specific methods:

• sklearn User Guide: Permutation feature importance
• sklearn User Guide: Partial dependence plots
• Shapley and SHAP
  • Interpretable Machine Learning book chapter (source for the bikeshare example)
  • Kaggle course (source for the soccer example): SHAP Values
  • SHAP: A Unified Approach to Interpreting Model Predictions
• Decision Rules: Interpretable Machine Learning book chapter

Going further:

• Interpretable Machine Learning
• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges
• Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead | Nature Machine Intelligence Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare - ScienceDirect

Ethics and Fairness

• Book: Fairness and machine learning
  • chapter we referenced in class: When is automated decision making legitimate?