Measuring Fairness is the flagship research project of Fair Trial Analysis. It addresses a basic problem in criminal law: courts are often required to decide whether a defendant received a fair trial, yet they lack a rigorous method for measuring fairness in objective terms. The article develops a valid and practical framework for estimating whether trial errors, omissions, or procedures materially changed the probability of conviction or punishment. It does so by combining case-specific survey research, a model of jury deliberation, and transparent statistical analysis, then demonstrates the method on real criminal cases involving ineffective assistance of counsel, coerced confessions, improper prosecutorial arguments, and flawed jury instructions.

---

How fairness is measured

The article begins from a familiar legal standard: a trial is unfair when its errors or deficiencies create a reasonable probability of a different outcome. The problem, as the article explains, is that courts have not had a workable way to quantify that probability in real cases. Past decisions and intuition may offer general guidance, but they do not provide a disciplined way to estimate how much a specific error changed the defendant’s chances at trial.

Measuring Fairness proposes a concrete solution. It measures fairness by comparing two probabilities: the probability of a guilty verdict in the actual trial condition and the probability of a guilty verdict in a hypothetical error-free trial condition. The difference between those two probabilities is treated as a measure of unfairness. In other words, the method asks not simply whether an error occurred, but how much that error changed the defendant’s risk of conviction or punishment.

To estimate that difference, the article lays out a four-step framework:

1. Estimate verdict preferences in the relevant jury pool through case-specific survey research.
2. Translates those individual-level preferences into the likely distribution of starting votes on a jury.
3. Use a jury-deliberation model—grounded in decades of empirical research—to estimate how those starting positions translate into final verdict probabilities.
4. Compare the outcome probabilities in the actual and hypothetical trial conditions and report the difference, along with uncertainty estimates.

A central insight of the article is that juror preferences alone are not enough. Even if one knows how many jurors initially lean guilty, that does not by itself reveal the probability of conviction. Deliberation matters. The framework therefore connects jury-pool preferences, initial jury factions, and final verdict probabilities in a single analytic structure. That is what allows it to estimate quantities that would otherwise remain out of reach.

Why this is important

The article argues that this problem matters because fairness cannot be taken seriously if it cannot be measured. Courts are tasked with deciding whether errors were harmless or harmful, but without a rigorous method they are often left with speculation, intuition, and hindsight. Measuring Fairness responds to that gap by offering a more transparent and evidence-based way to evaluate whether trial conditions likely affected the outcome.

The article also explains why modest changes in juror preferences can have large effects in close cases. A shift from one trial condition to another may only slightly change the proportion of jurors who initially favor conviction, yet that shift can substantially alter the probability of a guilty verdict once deliberation dynamics are taken into account. That is one reason fairness questions cannot be answered reliably by instinct alone.

Just as forensic testing brought greater rigor to questions of guilt and innocence, the article suggests that survey-based fairness analysis can bring greater rigor to questions of prejudice, harmlessness, and trial reliability. The comparison is not rhetorical. The article presents fairness measurement as a practical tool that can help courts uphold fair trials, identify unfair ones, and reduce reliance on subjective judgments where legal doctrine already asks empirical questions.

Finally, the article is designed to be usable. It does not stop with theory. It applies the framework to real criminal cases and makes the underlying analytic tools available through open-source software. That combination—legal relevance, methodological transparency, and practical application—is what makes Measuring Fairness the foundation of the broader research program at Fair Trial Analysis.