Abuses of LSAT Scores
Don LeDuc, President and Dean | January 22, 2015
Commentators, including academics who should know better, routinely disregard published limitations on the proper use of the Law School Admissions Test. This commentary discusses the widening gap between the proper use of the LSAT and the improper use made of the LSAT to justify assertions regarding the quality of potential law students and the law schools they consider.
Lately, assertions based on at least two false premises have dominated the blogs. The first is that a lower LSAT number in a school’s entering class means that the school’s graduates will be less capable of passing the bar. The second premise is even worse—that a lower LSAT number means that a school’s graduates will be less capable to practice as lawyers and that those below a certain number will be incapable of practicing law.
These assertions are blatantly contrary to limitations on the use of LSAT scores issued by the Law School Admissions Council, which creates and administers the test. In light of the claims now being made about the impact of changes in LSAT scores, let’s see what the LSAC has to say.
Is the use of an LSAT score for any purpose other than law school admissions proper?
Unequivocally, no. The LSAC’s cautionary policies say “[t]he LSAT was designed to serve admissions functions only. It has not been validated for any other purpose.” How much clearer can it be? Using the LSAT to make judgments about the capability of an individual—or a group of individuals like a law school’s entering class—to pass a bar examination is one of the misunderstandings and misuses that the LSAC warns against.
Those asserting that an individual will be unable or unlikely to pass a bar examination because that person’s LSAT score is below a certain level should be held to account. The LSAC has declared that the LSAT has not been validated as a predictor of future bar results. Anyone claiming that there is a relationship between an LSAT score and bar passage should be called to account and required to provide supporting evidence. And the burden should be on the person making the assertion, since the LSAC cautionary policy has already established that the LSAT has not been validated for that purpose.
Bad as that claim is, the assertion that a low LSAT score foretells incapacity to practice law is even worse. Again, the LSAT has not been validated in that regard. Nor has anyone been able to define what is or is not the capacity to practice law, much less measure who is likely to practice it well. Any claim that the LSAT does so should be completely disregarded, if not derided.
The idea that a test score on a brief examination taken by someone who has not even entered law school is a predictor of practice success many years later is preposterous. Success in practice by any definition is the result of education, training, intellect, integrity, and effort. After forty years of involvement in legal education and law school admissions, I can certainly attest that LSAT scores, whether high or low, do not predict who will be a successful practitioner and who will not.
The LSAC goes to considerable effort to educate those who use the LSAT in the context of the law school admissions process. It particularly warns that admissions officers and admissions committees should be knowledgeable about tests and test data. The LSAC has produced a reference manual for that purpose and conducts workshops and conferences to assure that the test is used appropriately.
Can the LSAT be used as the sole criterion for admission to law school?
No, and again the answer is unequivocal—"[t]he LSAT should be used as only one of several criteria for evaluation and should not be given undue weight solely because its use is convenient. . . . [because] the LSAT does not measure every discipline-related skill necessary for academic work, nor does it measure other factors important to academic success." While the LSAT provides “a standard measure of an applicant’s proficiency in the well-defined set of skills included in the test," the LSAC states that LSAT scores “do not measure, nor are they intended to measure, all the elements important to success at individual institutions." Further, “LSAT scores must be examined in relation to the total range of information available about a prospective law student."
The test measures the relative ability of those who take the test, some of whom already have the skills that are well-developed. But those skills can be learned. Law schools teach students to think like lawyers, to learn legal methods, to understand the meaning of a case and its relationship to other cases. The LSAT score demonstrates relative proficiency in the skills which the test measures, but law students and lawyers need other skills. Those are not tested by the LSAT.
Is the LSAT the best predictive measure, at least for law school success?
According to the LSAC, the LSAT is intended to help in the selection process at law schools—“[i]n order to assist in assuring that there is a demonstrated relationship between quantitative data used in the selection process and actual performance [at a particular school].” That demonstration is made through correlation studies provided by the LSAC that check the relationship among LSAT scores, undergraduate grade-point average, and law school grades that apply to the particular school. The key is the relationship of these factors at that school. The validation of the LSAT for a particular school comes through the regression analysis done for that school in the correlation study for that school. These studies invariably find that neither the LSAT score alone nor the UGPA alone are better predictors than the use of those two factors combined in the context of previous levels of academic success at that institution. The relative weight to accord each factor is a function of the performance of those who have gone before the candidate. Indeed, what the LSAT does is attempt to predict the likely first-year grade point average for the person being considered for admission to the particular school, based on how well those previously admitted with the same combination of the two factors did. If two or more applicants present the same combination, the test cannot differentiate between them.
Regardless of the correlation, is there a bright line LSAT score below which a school should not offer admission?
No, there is not. The LSAC declares that “[c]ut-off LSAT scores (below which no applicants will be considered) are strongly discouraged. Such boundaries should be used only if the choice of a particular cut-off is based on a carefully considered and formulated rationale that is supported by empirical data . . . .” The admonition continues that a school using the cut-off should include consideration of a standard error of measurement to avoid distinctions not sufficiently substantial to be reliable.
This portion of the LSAC cautionary policies concludes with two important observations. First, cut-off scores may have a greater adverse impact upon applications from minority groups than upon the general applicant population. And second, an LSAT cut-off score should be combined with the undergraduate grade-point average before an assessment of the probability of success is determined.
A separate cautionary policy adds that LSAT scores are approximations of ability, not exact measures. Any distinctions among applicants based on LSAT scores should be made only if the score differences are reliable.
What conclusions can be drawn regarding the current discussions of declining law school admissions LSAT scores?
The foremost conclusion is that nearly all of the discussion is at best misconceived and at worst harmful. The concept that LSAT scores bear any relationship to the ability to practice law is indefensible. Those who assert that such a connection exists are ignorant at best.
The assertion that LSAT scores predict bar results is without support. Those who create and provide the test have stated that a correlation of LSAT scores to the passage of a bar examination has never been validated, certainly not by them. While such a correlation could certainly be made, no one claiming a derogatory effect on bar results due to a declining LSAT score has offered any support to that end.
One reason that a valid connection is not possible is that there is no such thing as “the bar examination.” Each state administers its own examination. While many use the multi-state multiple choice examination, the passing score varies from state to state, meaning that a person with identical LSAT scores and multi-state scores could pass in one state and fail in another. The same is true of those using the uniform essay test, since each state will set its own passing score and independently establish the relative weight to be given to the multiple choice and essay scores. Each state will select different essay questions and grade the answers itself. And the results will also vary from test to test as the authorities revise their practices.
Unfortunately, we are fascinated with labeling and ranking. We rank schools based on LSAT scores, we rank schools based on bar results, and we characterize law schools by tiers and students by LSAT scores. We assert that students with LSAT scores below a certain number are inferior and unworthy, but fail to take into consideration that the creators of the LSAT affirmatively declare that such a conclusion is not valid and is an improper use of the LSAT score. We assert that the LSAT test measures the ability to practice law, again contrary to the proper use of the LSAT and again despite the fact that there is no evidence to substantiate the claim. We disregard that minority groups are arguably adversely affected by multiple-choice tests.
In the land of opportunity, those who would deny anyone who has graduated from college the opportunity to fulfill a dream based on an LSAT cut-off score are elitist, paternalistic, and ignorant of the purpose of the very test they rely upon as the basis for their denial. Law school applicants and law schools deserve better treatment. Law school applicants, law school students, and law school graduates are not just numbers. The Law School Admissions Council and the Section of Legal Education and Admissions to the Bar should take steps to counter the unwarranted assertions and misconceptions that unfortunately are becoming common.