Class 11 Notes

Algorithmic Governance in Medical Decision-Making: Ethics, Race, and Public Participation

Guest: David Robinson (Cornell/Author)Guest: Dr. Giovanni Parmigiani (Harvard)

Class Overview

Part 1: David Robinson on algorithmic governance in kidney allocation systems

Focus: Democratic oversight of high-stakes medical algorithms and the balance between technical optimization and ethical fairness

Part 2: Dr. Giovanni Parmigiani on race-based adjustments in medical algorithms

Key Issue: The MDRD study controversy and the 2021 transition to race-free kidney function estimation

Key Insights from Class 11

Algorithmic Governance in High-Stakes Decisions

Medical algorithms require transparent, participatory governance processes that include both technical experts and affected communities to maintain public trust and fairness

The Technical-Ethical Divide

Statistical accuracy doesn't automatically ensure ethical fairness - technical optimization can perpetuate or amplify social inequities without deliberate ethical consideration

From Proxies to Direct Measurement

Using race as a proxy for biological factors like muscle mass opens doors to bias - direct measurement of relevant biological markers provides better outcomes

Part 1: The Evolution of Kidney Allocation Systems

David Robinson - From Seattle God Committee to Modern Algorithmic Governance

The Historical Journey

The story begins with Belding Scribner's 1960 invention of the Teflon tube that enabled repeated dialysis. With only four dialysis machines but overwhelming demand, Scribner made a radical choice: medical experts would determine eligibility, but laypeople would make the ethical decisions about who receives treatment.

1960

Teflon Tube Invention

Description: Belding Scribner invents Teflon tube for repeated dialysis

Impact: Enabled long-term dialysis but created scarcity problem

1962

Seattle God Committee

Description: First lay committee to decide who gets life-saving treatment

Impact: Established precedent for public input in medical rationing

1980s

National Organ Transplantation Act

Description: Congress mandates national computer-based allocation system

Impact: Created legal framework for algorithmic organ distribution

2007

LYFT Proposal

Description: Life Years From Transplant system proposed in Dallas

Impact: Sparked debate about age-based allocation and patient advocacy

The Dallas 2007 Moment

The LYFT Proposal
  • • Life Years From Transplant system
  • • Save 10,000+ additional life years annually
  • • Complex mathematical modeling
  • • Age as major factor (25% of score)
Patient Advocacy Response
  • • Clive Graw's methodological objections
  • • Age as imperfect proxy for health
  • • Perverse incentives for younger patients
  • • Final compromise balancing utility and equity
Four Techniques for Algorithmic Governance

Transparency

Making algorithmic decision-making processes visible and understandable

Examples:
  • Public meetings like Dallas 2007
  • Open data analysis and reporting
  • Clear documentation of allocation rules

Participation

Including affected communities in algorithm design and oversight

Examples:
  • Patient advocacy representation
  • Lay committees for ethical input
  • Public comment periods

Forecasting

Predicting and modeling the distributional effects of algorithmic changes

Examples:
  • Age distribution impact analysis
  • Racial equity assessments
  • Life years saved projections

Auditing

Ongoing monitoring and evaluation of algorithmic outcomes

Examples:
  • Annual transplant rate reports
  • Cross-racial outcome comparisons
  • Continuous system performance review

The Cornell "Any Person, Any Study" Philosophy

Robinson's work exemplifies interdisciplinary scholarship that welcomes diverse perspectives and methodologies. Just as Cornell's founding motto embraces broad access to education, algorithmic governance requires bringing together technical expertise with public input and ethical consideration.

Part 2: The MDRD Study and Race-Based Medical Algorithms

Dr. Giovanni Parmigiani - Statistical Methods and Social Consequences

The GFR Estimation Challenge

The Problem: Kidney function assessment requires measuring glomerular filtration rate (GFR), but direct measurement is complex and expensive. The MDRD study developed an equation to estimate GFR from serum creatinine and demographic factors.

The Controversy: Including race as a variable improved statistical accuracy but created systematic differences in care recommendations across racial groups.

The Statistical Rationale

Black participants showed higher creatinine levels for same kidney function

Explanation: Race used as proxy for muscle mass differences

Impact: Better statistical prediction accuracy

The Equity Problem

Race-based adjustments led to different care recommendations

Explanation: Higher estimated GFR for Black patients = less urgent care

Impact: Potential under-treatment of Black patients

The Proxy Problem

Race as a social construct used for biological prediction

Explanation: Mixed ancestry, self-reporting biases, and cultural variability

Impact: Introduces systematic bias into medical decision-making

The 2021 Solution

New algorithms achieve similar accuracy without race

Explanation: Task force developed race-free GFR estimation methods

Impact: Maintains clinical utility while improving equity

The Regression Analysis: Technical vs. Social Considerations

Statistical Rationale
  • • Race coefficient: +1.18 for Black patients
  • • Improved model calibration and accuracy
  • • Parallel lines assumption for different groups
  • • Cross-sectional study design with validation
Social Consequences
  • • Different clinical recommendations by race
  • • Embedded racial categorization in workflows
  • • Self-reporting biases and mixed ancestry issues
  • • Perpetuation of social constructs in medicine
Technical-Ethical Challenges in Algorithmic Design

Quantification as Moral Anesthetic

Mathematical formulas can obscure the ethical weight of life-and-death decisions

Example: LYFT scores made age-based rationing appear purely technical

Impact: Difficult choices get removed from public moral discourse

Algorithm Shifts Moral Attention

System architecture determines which ethical issues get debated

Example: Focus on age allocation overshadowed transplant list access inequities

Impact: Some unfair practices remain hidden 'off-stage'

Participation Creates Opinions

Public engagement shapes values rather than just measuring them

Example: Clive Graw's advocacy changed how people thought about age fairness

Impact: Democratic deliberation is constitutive, not just consultative

Shared Infrastructure Requirements

Public participation needs institutional support and data infrastructure

Example: Structured meetings, analysis tools, and public reports enable engagement

Impact: Technical and social infrastructure must be co-designed

Key Lessons for Data Scientists

  • • Statistical optimization alone doesn't ensure ethical outcomes
  • • Algorithm architecture shapes which ethical issues get attention
  • • Public participation requires both technical and social infrastructure
  • • Proxy variables introduce social biases into technical systems
  • • Deliberation is costly but necessary for democratic legitimacy
Understanding Regression Analysis in Medical Algorithms

Linear Relationship

Mathematical model relating creatinine levels to kidney function

Formula: log(GFR) = α + β₁(creatinine) + β₂(age) + β₃(sex) + β₄(race) + ε
Interpretation: Each coefficient represents the effect of that variable

Dichotomous Variables

Binary (yes/no) variables like sex and race

Formula: Race coefficient = +1.18 for Black patients
Interpretation: Vertical shift in prediction line for different groups

Statistical Bias vs. Social Bias

Model accuracy doesn't guarantee social fairness

Formula: Better R² ≠ Better equity outcomes
Interpretation: Technical optimization can perpetuate social inequities

The Muscle Mass Hypothesis

The MDRD researchers hypothesized that racial differences in creatinine levels reflected differences in average muscle mass, since creatinine is a byproduct of muscle metabolism. This raised the question: if muscle mass is the real biological factor, why not measure it directly?

Student Discussion: Class explored whether replacing race with muscle mass would reduce bias while maintaining predictive accuracy, highlighting the difference between social constructs and measurable biological variables.

Class Discussion Questions and Perspectives

Should muscle mass replace race in medical algorithms?

Key Considerations:
  • Biological relevance vs. measurement costs
  • Individual variation within demographic groups
  • Social determinants affecting muscle mass

When is using demographic variables in algorithms justified?

Key Considerations:
  • Life-or-death situations vs. lower stakes decisions
  • Historical discrimination and current equity
  • Availability of alternative predictors

How should technical experts balance accuracy with fairness?

Key Considerations:
  • Professional medical obligations
  • Broader social responsibility
  • Long-term trust in medical systems

What role should patients play in medical algorithm design?

Key Considerations:
  • Technical complexity vs. democratic input
  • Individual preferences vs. population outcomes
  • Representation across diverse communities

Role-Playing Exercise: Multiple Perspectives

As a Patient

How would you feel if your race affected your medical care?

As a Doctor

Balance individual care with population-level guidelines

As a Data Scientist

Optimize for accuracy while considering social impact

As a Policy Maker

Create fair systems that maintain public trust

Practical Applications Beyond Medicine

Medical Algorithm Governance

Applying democratic oversight to clinical decision support systems

Use Case: Ensuring AI diagnostic tools are transparent and equitable

Resource Allocation Systems

Extending kidney allocation lessons to other scarce resources

Use Case: Hospital bed allocation, vaccine distribution, educational resources

Algorithmic Auditing

Systematic monitoring of algorithmic outcomes across populations

Use Case: Regular assessment of AI system impacts on different demographic groups

Proxy Variable Assessment

Evaluating when demographic variables should be included in algorithms

Use Case: Criminal justice risk assessment, credit scoring, hiring algorithms

The 2021 Resolution: A Model for Reform

Led by Neil Powell and Cynthia Delgado, a National Kidney Foundation task force developed new GFR estimation methods that achieve similar clinical accuracy without using race. This demonstrates that technical and ethical goals can be aligned through deliberate effort.

Assignment Framework: Students should design studies comparing old and new algorithms, considering data collection, follow-up measures, success metrics, and public explanation strategies across diverse cultural backgrounds.

Future Research and Development Directions

Technical Developments

Development of race-free medical algorithms that maintain clinical accuracy
Creation of standardized algorithmic governance frameworks for healthcare
Research on effective public participation methods for technical decision-making

Governance Innovations

Investigation of alternative biological markers to replace demographic proxies
Development of real-time equity monitoring systems for medical algorithms
Creation of interdisciplinary training programs combining technical and ethical expertise

Cross-Cultural Communication Challenge

As Professor Meng emphasized, explaining algorithmic decisions across different cultural backgrounds presents unique challenges. Effective governance requires not just technical transparency but culturally sensitive communication that respects diverse values and worldviews while maintaining scientific integrity.

Key Takeaways and Critical Skills for Data Scientists

Technical Insights

Statistical accuracy doesn't guarantee ethical fairness or social acceptance
Proxy variables can introduce systematic bias even when improving prediction
Direct measurement of relevant factors is preferable to demographic proxies

Governance Insights

High-stakes algorithms require democratic oversight and public participation
Transparency must be coupled with accessible explanation and engagement
Technical and social infrastructure must be co-designed for effective governance

Essential Skills for Ethical Data Science

Technical Skills:
  • • Understanding when proxies introduce bias
  • • Evaluating model fairness across groups
  • • Designing studies that consider social impact
  • • Communicating uncertainty and limitations
Social Skills:
  • • Facilitating public participation in technical decisions
  • • Translating complex methods for diverse audiences
  • • Recognizing historical context and power dynamics
  • • Building trust through transparency and accountability
Session Summary

This session provided a masterclass in the intersection of technical expertise and democratic governance in high-stakes algorithmic systems. David Robinson's account of kidney allocation evolution demonstrated how medical algorithms require both technical sophistication and public legitimacy to function effectively in society.

Dr. Parmigiani's analysis of the MDRD controversy illustrated how well-intentioned statistical methods can perpetuate social inequities when they embed demographic proxies without sufficient consideration of downstream consequences. The 2021 resolution shows that technical accuracy and ethical fairness can be achieved simultaneously through deliberate effort.

The extensive student discussion highlighted the complexity of these issues and the importance of considering multiple perspectives - patients, doctors, data scientists, and policymakers - when designing systems that affect real lives. The session emphasized that effective data science requires both technical competence and social responsibility.