enrolled in aca

Overview

This model predicts whether a voter is enrolled in a health insurance plan through the Affordable Care Act marketplace.

Trained on survey data, it draws on age, political engagement, household composition, and area-level demographics to estimate enrollment likelihood.

These scores help organizations identify ACA enrollees for outreach around open enrollment, coverage protections, and health care policy.

  • approach: boosted tree classifier
  • training rows: 5,373
  • last updated: 2026-09-06 17:47:01

Validation

metric value
area under ROC curve 0.71
area under precision-recall curve 0.14
accuracy 0.54
precision 0.11
recall 0.81
f1 score 0.19
mean log loss 0.94
gini coefficient 0.43
brier skill score 0.03

Lift Chart

Each bar shows how much more common the outcome is within a score decile than in the population overall. Bars taller than the dashed line (lift of 1) mean the model’s top-ranked groups concentrate the outcome; a steady decline from decile 1 to 10 indicates the model is ranking effectively.

Calibration Plot

Bars of similar height within each decile indicate well-calibrated predictions. Systematic differences reveal where the model over- or under-predicts.

Cumulative Gains

The further the model curve bows above the diagonal, the better the model concentrates the outcome in its top-ranked predictions.

Decile Analysis Table

decile count mean actual mean predicted mae lift
10 138 0.20 0.19 0.01 2.99
9 137 0.11 0.09 0.02 1.67
8 137 0.04 0.07 0.03 0.67
7 138 0.12 0.06 0.06 1.88
6 137 0.06 0.06 0.002 0.89
5 137 0.03 0.05 0.02 0.45
4 138 0.06 0.04 0.01 0.89
3 137 0 0.04 0.04 0.00
2 137 0.007 0.03 0.03 0.11
1 138 0.03 0.03 0.003 0.44

Architecture

This model is a boosted-tree classifier trained with BigQuery ML.

Hyperparameter configuration

  • max iterations: 20
  • learn rate: 0.4667
  • l1 regularization: 1.913e-08
  • l2 regularization: 2e-04
  • min relative progress: 0.01
  • early stop: TRUE
  • input label columns: benefits_enrolled_in_aca
  • data split method: AUTO_SPLIT
  • num trials: 12
  • max parallel trials: 4
  • hparam tuning objectives: ROC_AUC
  • enable global explain: TRUE
  • auto class weights: FALSE
  • max tree depth: 3
  • subsample: 0.7158
  • min split loss: 0
  • category encoding method: LABEL_ENCODING
  • booster type: GBTREE
  • num parallel tree: 1
  • tree method: AUTO
  • min tree child weight: 1
  • instance weight column: sample_weight
  • xgboost version: 0.9

Features

Importance

Sourcing

Attom Property Data API

  • land market value
  • last sale price
  • lot size (acres)

CDC PLACES

  • tract % adults with a routine checkup

CDC WONDER

  • routine doctor checkup, past year (adults)

Georgetown University Center for Children and Families

  • county medicaid / chip coverage rate

State voter files (via TargetSmart)

  • age

TargetSmart

  • children present score
  • double negative democrat score
  • gun owner score
  • income rank
  • local election voter score
  • marriage score
  • mormon score
  • non-presidential primary turnout score
  • race: latino
  • race: native american

U.S. Census American Community Survey

  • block group % high school diploma
  • block group % income $150–200k
  • block group % income $50–100k

U.S. Census CPS Annual Social and Economic Supplement

  • current medicare coverage
  • dwelling ownership

US Department of Labor

  • county median weekly childcare cost, preschool

Implementation

Prediction distribution

statistic value
mean 0.05
std dev 0.03
min 0.02
25th 0.03
median 0.04
75th 0.05
max 1

Segments

segment lowest score highest score share of records
Most likely 0.09 1 4.4%
Somewhat likely 0.07 0.08 4.1%
Least likely 0.02 0.06 91.5%

External validity

Our model indicates that 5% of people with scored records are enrolled in an ACA marketplace health plan. CMS 2024 Marketplace Open Enrollment Report puts marketplace enrollment at roughly 7% of U.S. adults.