Citation
Spamann, H. (2022). Judges in the lab. The Practice: Harvard Law School Center on the Legal Profession. https://clp.law.harvard.edu/knowledge-hub/magazine/issues/judicial-decision-making/judges-in-the-lab/
Key Takeaways
Controlled judicial experiments show that formally binding precedent often exerts substantially less influence on judicial decisions than standard legal theory predicts; Legally irrelevant contextual cues, such as defendant sympathy or nationality, can significantly affect outcomes even when the governing law is held constant; Judges from civil law and common law systems display strikingly similar reasoning patterns in experimentally controlled cases, calling into question claims that legal tradition fundamentally shapes cognitive style; In routine cases with clear governing rules, judges frequently depart from those rules, particularly where standards are flexible or discretion is implicitly available; Judicial decisionmaking exhibits systematic, measurable regularities across jurisdictions and case types, indicating that outcomes are shaped by structured features of decision environments rather than by doctrine alone.
Dataset Description
The article synthesizes results from two randomized controlled experiments involving sitting judges. In the first experiment, 361 judges from seven countries (Argentina, Brazil, China, France, Germany, India, and the United States) decided a simulated appeal from the International Criminal Tribunal for the former Yugoslavia, with random variation in cited precedent and defendant characteristics. In the second experiment, 61 U.S. federal judges decided a domestic tort case involving a choice of law question, with both the applicable legal rule (lex loci versus Restatement-based standards) and party sympathy experimentally manipulated. In both studies, judges were provided with realistic case materials, including briefs, statutes, and trial judgments, and were given a fixed period to reach a decision.
Methodology
Randomized experimental design using survey-based judicial decision tasks; comparative analysis across legal systems; quantitative/statistical analysis of outcomes; qualitative examination of reasoning paths and cited authorities.
Key Findings
Across both experiments, variation in precedent had little to no measurable effect on judicial outcomes, while small changes in extra-legal context produced large and statistically significant shifts in decisions. In the international criminal law experiment, defendant sympathy strongly influenced conviction rates despite being legally irrelevant, whereas assigned precedent did not meaningfully affect outcomes. In the domestic tort experiment, judges frequently disregarded governing choice of law rules, including mechanical territorial tests, and applied standards inconsistently when discretion was available. Analysis of judges’ document navigation and reasoning patterns shows substantial similarity across civil and common law systems, indicating that legal tradition does not meaningfully differentiate cognitive approaches to adjudication in these settings. Overall, the findings demonstrate that judicial decisions are systematically shaped by context, discretion, and decision structure rather than being tightly constrained by formal doctrine.
Summary
Judges in the Lab offers a rare experimental examination of judicial decisionmaking by placing sitting judges in controlled but legally realistic adjudicative settings. By holding facts and law constant while manipulating precedent, legal standards, and contextual cues, the study directly tests foundational assumptions about how judges reason and decide cases.
The experiments reveal a consistent pattern across jurisdictions and legal traditions: judges often rely less on precedent and formal rules than legal theory assumes, particularly in cases involving flexible standards or ambiguous doctrinal guidance. Instead, outcomes are sensitive to contextual factors and to how discretion is structured within the decision environment, with judges across civil and common law systems exhibiting similar reasoning paths and citation behavior.
The article concludes that judicial behavior is neither idiosyncratic nor purely rule-bound, but systematically shaped by institutional design, cognitive constraints, and the framing of legal choices. By demonstrating these regularities through randomized experiments, the study contributes to a more empirically grounded understanding of adjudication and highlights the importance of analyzing legal outcomes through observable decision structures rather than through formal doctrine alone.
How the Study Advances Empirical Understanding of Legal Outcomes
The study finds that, across controlled experimental settings, judicial decisions exhibit consistent and repeatable patterns shaped by decision environments and contextual cues rather than by formal legal authorities alone. The results indicate that legal outcomes are structured by how information is presented, the clarity or flexibility of governing rules, and the institutional conditions under which discretion is exercised, challenging assumptions that doctrine reliably constrains results. By isolating legal materials and decision contexts through randomized experiments, the study’s methodology aligns with Pre/Dicta’s emphasis on empirical, case-based analysis of legal outcomes as a necessary foundation for understanding how institutional design and decision structure systematically influence adjudication.
