The General Accuracy of District and Appellate Judgeship Case-Related Workload Measures
GAO-09-1050T, Sep 30, 2009
Biennially, the Judicial Conference, the federal judiciary's principal policymaking body, assesses the need for additional judges. The assessment is based on a variety of factors, but begins with quantitative case-related workload measures. This statement focuses on (1) whether the judiciary's quantitative case-related workload measures from 1993 were reasonably accurate; and (2) the reasonableness of any proposed methodologies to update the 1993 workload measures. This statement is based on work completed and reported in 2003 and discussed in testimony on June 17, 2008.
In 2003, GAO reported that the 1993 district court case weights were reasonably accurate measures of the average time demands that a specific number and mix of cases filed in a district court could be expected to place on the district judges in that district. At the time of GAO's 2003 report, the Judicial Conference was using case weights approved in 1993 to assess the need for additional district court judgeships. The weights were based on data judges recorded about the actual in-court and out-of-court time spent on specific cases from filing to disposition. This methodology permitted the calculation of objective, statistical measures of the accuracy of the final case weights. In 2003, GAO reviewed the research design the Judicial Conference's Subcommittee on Judicial Statistics had approved for updating the 1993 district court case weights, and had two concerns about the design. First, the design assumed that the judicial time spent on a case could be accurately estimated by viewing the case as a set of individual tasks or events in the case. Information about event frequencies and, where available, time spent on the events would be extracted from existing databases and used to develop estimates of the judge-time spent on different types of cases. However, for event data, the research design proposed using data from two data bases that had yet to be integrated to obtain and analyze the data. Second, unlike the methodology used to develop the 1993 case weights, the design for updating the case weights included limited data on the time judges actually spent on specific types of cases. Specifically, the proposed design included data from judicial databases on the in-court time judges spent on different types of cases, but did not include collecting actual data on the noncourtroom time that judges spend on different types of cases. Instead, estimates of judges' noncourtroom time were derived from the structured, guided discussions of about 100 experienced judges meeting in 12 separate groups (one for each geographic circuit). Noncourtroom time was likely to represent the majority of judge time used to develop the revised case weights. The accuracy of case weights developed on such consensus data cannot be assessed using standard statistical methods, such as the calculation of standard errors. Thus, it would not be possible to objectively, statistically assess how accurate the new case weights are--weights on whose reasonable accuracy the Judicial Conference relies in assessing judgeship needs. The case-related workload measure for courts of appeals judges is adjusted case filings in which all cases are considered to take an equal amount of judge time except for pro se cases--those in which one or more of the parties is not represented by an attorney--which are discounted. In our 2003 review, we found no empirical basis on which to assess the accuracy of this workload measure. Although a number of alternatives to the adjusted filings measure have been considered, the Judicial Conference has been unable to agree on a different approach that could be applied to all courts of appeal.