Citation
Mahari, R., & Lera, S. C. (2024). Early-career citations capture judicial idiosyncrasy and predict judgments. arXiv. https://arxiv.org/pdf/2410.00725
Research Question
Do judges’ early-career citation patterns and biographical characteristics reveal persistent idiosyncrasies that systematically predict civil case outcomes in U.S. District Courts despite random case assignment?
Key Takeaways
Random case assignment in U.S. District Courts does not eliminate judge effects: many judges have systematically higher or lower plaintiff win rates; Legally extraneous biographical and contextual factors, especially a judge’s past plaintiff win rate, significantly predict future case outcomes; Early-career citation patterns encode durable judicial priors and can forecast later decisions as well as, or better than, biographical data; A measurable minority of judges are individually predictable in their rulings based solely on early citation behavior; Circuit-level differences and temporal trends in plaintiff success mean forum and timing choices materially affect litigation prospects; Data-driven judicial intelligence tools can now quantify judge-specific risk, informing forum selection, settlement strategy, and pre-appointment vetting.
Dataset Description
The authors assemble a merged dataset of 112,312 U.S. federal civil District Court cases decided between 1880 and 2018 across all 94 district courts. They integrate (1) the Federal Judicial Center’s Integrated Database of civil cases since 1970 (2.5 million filings), (2) the Harvard Case Law Access Project’s 302,986 civil district court opinions (with full texts, judges, and a citation network of 1.7 million federal opinions), and (3) Federal Judicial Center judicial biographical data for 2,394 district judges. Using Longformer-based NLP models trained on 31,222 cases that could be textually matched across datasets, they infer case outcomes and six broad case types for the full CAP sample with high-confidence predictions, then restrict to high-confidence labels to produce the final 112,312-case analysis set. They also construct judge-level early-career citation profiles (first 10% of each judge’s cases) over the 2,403 most-cited opinions and compress them using non-negative matrix factorization to obtain 30-dimensional citation embeddings per judge.
Methodology
statistical/quantitative, machine learning
Key Findings
Across all 94 U.S. District Courts, case assignment by type and party identity is effectively random, yet judges’ civil plaintiff win rates diverge substantially from the system-wide 25% baseline: about 38–40% of judges show statistically significant deviations over their careers. Gradient-boost models using only extraneous biographical features, such as party of appointing president, gender, experience, workload, promotions, circuit, decision date, and especially each judge’s historical plaintiff win rate, predict future case outcomes on balanced out-of-sample data at 56–65% overall accuracy, surpassing 70% in high-confidence bins for civil rights, torts, and prisoner petitions. Low-dimensional embeddings of judges’ early-career citation behavior match or exceed these biographical models in predictive power, despite being based solely on citations from the first 10% of a judge’s cases and excluding those early cases from the outcome prediction sample. These citation embeddings also predict judges’ biographical traits, including promotions (pseudo-R² around 0.23), confirming that citation choices are a strong behavioral signature. At the individual-judge level, after multiple-testing corrections, roughly 6–8% of active district judges show systematically predictable decisions based purely on their early citation records, implying that a nontrivial minority of judges apply stable, outcome-relevant priors even when cases are randomly assigned and predominantly “easy.” The authors also document trends in time and circuit-level differences in plaintiff success, with a general decline in plaintiff win rates over time and substantial cross-circuit heterogeneity.
Summary
The paper investigates whether judges’ personal tendencies can be detected and used to predict case outcomes even within a system that relies on random assignment to secure impartiality. Focusing on U.S. federal civil cases in the District Courts, the authors assemble an unusually comprehensive dataset that combines docket-level information, full-text opinions, citation networks, and biographical data on nearly all district judges over more than a century. They first test the foundational institutional claim that cases are randomly assigned across judges within courts and case types.
Using fine-grained judge–decade–case-type cells, the authors show that, with rare exceptions, the distribution of parties and case types across judges is consistent with random assignment; only a tiny fraction of cells show statistically significant deviations after multiple-testing adjustments. This result strengthens the interpretation that any systematic differences in plaintiff win rates across judges are not artifacts of skewed case mix. Against this backdrop, the authors document substantial and statistically robust dispersion in plaintiff success: roughly 38–40% of judges have plaintiff win rates that differ significantly from the system-wide baseline of about 25%, indicating persistent judge-specific tendencies.
The study then examines whether these tendencies are predictable using information that should, in principle, be legally irrelevant. Machine-learning models, primarily gradient-boosted trees, are trained on balanced datasets to predict whether the plaintiff will win. Inputs include biographical and contextual variables (such as appointing president’s party, gender, tenure, workload, promotion history, circuit, and year) as well as each judge’s historical plaintiff win rate. These models achieve 56–65% overall accuracy and above 70% in high-confidence subsets for areas such as civil rights, torts, and prisoner petitions, far exceeding chance. The influence of a judge’s prior plaintiff win rate underscores the presence of stable, outcome-relevant priors that persist across time and case categories.
The paper’s most novel contribution is to demonstrate that judges’ early-career citation choices serve as a compact, powerful signal of these priors. The authors construct early-career citation vectors that track how frequently each judge cites the most influential opinions in the first 10% of their cases. Using non-negative matrix factorization, they compress these high-dimensional vectors into 30-dimensional embeddings that capture latent citation styles. Models that rely solely on these early-career citation embeddings, controlling for circuit and time, perform at least as well as biographical models in predicting later case outcomes. Moreover, the embeddings significantly explain variation in judge characteristics, such as the likelihood of promotion, suggesting that underlying ideological or methodological orientations shape both citation behavior and career trajectories.
Finally, the authors move from aggregate prediction to individual-level assessment. By fitting models and then testing predictive performance judge by judge, they find that, after correcting for multiple comparisons, about 6–8% of active district judges are individually predictable in their decisions based exclusively on early citation patterns. This implies that a meaningful subset of judges consistently favors or disfavors plaintiffs in a way that can be detected early and quantified. For legal practitioners, this means that the identity of the judge can be as consequential as the formal legal merits, and that these effects can now be modeled systematically for purposes of forum choice, motion strategy, settlement valuation, and appellate planning. For policymakers, the findings raise questions about selection, monitoring, and the design of institutions that depend on an assumption of interchangeable, neutral adjudicators.
How the Study Advances Empirical Understanding of Legal Outcomes
The study finds that even under random case assignment in U.S. District Courts, civil outcomes exhibit stable, repeatable patterns over time, with persistent dispersion in plaintiff success rates that cannot be explained by case mix alone, indicating structured regularities in decision environments rather than randomness. The results indicate that recognizing these regularities is analytically important because it clarifies how institutional design and historically observable patterns shape outcomes independently of formal legal merits, helping separate genuine neutrality from systematic variation. By combining large-scale case data, textual analysis of opinions, and longitudinal measurement of judicial behavior, the study exemplifies an empirical, case-grounded approach to understanding legal outcomes, consistent with Pre-Dicta’s emphasis on disciplined analysis of decision contexts as a necessary component of high-level litigation practice.
