Calibration

Topics

• Calibration, Trade-offs, and a case-study (COMPAS II)

Reading (Required)

• Chapter 2 of Fair ML Book (You may find the source paper Equality of Opportunity in Supervised Learning useful, especially sections 1-3, 6-8)

• Inherent Trade-Offs in the Fair Determination of Risk Scores (Section 1) by Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan

• A Different Way of Thinking About Cancel Culture by Ezra Klein

Reading (Optional)

TBA

Reading Responses

From Fair ML Book:

• From the ‘Inherent Limitations of Observational Criteria’:
  • What is the main point made by this section.
  • How does Scenario I/II makes this main point?
  • Think of your own example, using a different classifier, that illustrates this point.
• From Ezra Klein piece:
  • What are the decision making classifiers he considers?
  • What are the harms? Allocative or Representational (or both)?
  • What are the utilities being considered?
  • Discuss the interplay between algorithmic decision and the decisions that consititute direct harms? How does he view repsonsibilities for the impacts of these decisions?