So, I’ve got wine on the brain. Mrs. Sherpa and I recently spent a long weekend in the Napa Valley tasting wine and enjoying some time away from the craziness of life.
Since I’ve been back the Freakonomics blog has posted several times on the blind tasting of wine and how most people can’t really tell the difference between a $10 bottle and a $150 one. The discussion is infused with some interesting studies on the topic and, Levitt’s ultimate conclusion that the more you know about wine the more you end up spending on it.
I was swishing this idea around in my head when I read this choice quote from the Case Western Law Dean criticizing the U.S. News law school rankings:
[I]t seems beyond debate that it is truly depressing that law deans, who have so many important educational issues to address, feel the pressure they undeniably feel to make important decisions about their schools in response to a popular magazine’s educationally unsophisticated decisions about ranking methodology.
Read the whole piece. It’s good, if not real original (at least for those of us critical of the rankings). The fact that his piece is, in part, a response to a U.S. News proposal to count the entry credentials of evening students in rankings is for another post. What seems to be lacking in these rankings is a more personal, less manipulable way to measure law school quality.
So why don’t we do what the prominent wine critics do – line up a bunch of glasses, do a blind tasting, and rate purely based on the taste of the wine? The critics supposedly do not know the producer of the wine or its vintage. Just it’s type (syrah, merlot, etc.).
As part of the U.S. News calculation I propose appointing a panel of experts – legal scholars, judges, attorneys, etc – and have them take several steps to measure the quality of education. This panel could listen to classes without knowing who is teaching or at what school, but it would know the area of law. Rate based on overall quality of instruction, discussion, etc. The panel might also read law review articles or student papers to gauge the quality of scholarship being done and promoted at the school.
Perhaps I still have some tannin residue in my brain, but it strikes me that adding a component like this (not to mention Dr. Bombay’s law school value calculation) could help to bring a more personal, less manipulable component to the rankings. Or maybe the U.S. News team just needs a nice Napa Cab when they’re tabulating the rankings.
If you enjoyed this article, please consider sharing it!
12 Responses to Use Blind Tasting To (Help) Rank Law Schools
- Countdown to Graduation:
0 days. 
Connect
Twitter
LinkedIn
Google Profile
Delicious
Blog
I like the calculation, but I think the Rankings focus a lot on reputation…which although completely arbitrary, has some importance.
I see the US-News rankings effecting how my undergrad is run. A lot of upper-level people are hired based on their degrees and not their experience, the result is higher rankings but internal chaos and inefficiency.
I agree that the rankings generally have some importance. You’ve hit upon an important point, I think. Schools base personnel and other decisions on the rankings, not on educational quality.
Reputation is definitely important for something (but see vault’s ranking of underrated law schools which shows that the rankings might miss a critical reputation measure – the perception of the law firms that hire them).
The point also remains as to what value reputation really has if you’re getting a substandard education. The Case Western dean’s point is that the Rankings result in a sort of dumbing down of legal education.
The problem with using jobs as the barometer is that occasionally you get anomalies in the marketplace the make everything go haywire. If, for example, you wanted a high ranking political appointment in the Bush administration, Regent was the place to go to law school even though its only been accredited by the ABA since 1989 and U.S. News sticks it in the 4th tier nationally.
My main problem with the reputation yardstick is that the curriculum is standardized by the ABA. I’m supposed to therefore glean from the rankings that Harvard has someone who can teach contracts and civil procedure better than the University of Maryland? How the heck do you quantitatively evaluate pedagogical skill?
That’s the problem. I’m not sure you can quantitatively measure pedagogical skill. But I also don’t think it’s safe to assume that Harvard has a better contracts professors than Maryland (or Minnesota) by virtue of its rankings. Having a panel(s) consider pedagogical skill though might help provide some qualitative perspective, however, that gets beyond jobs and reputation – or at least help to balance them in a way that is more difficult to manipulate.
Warning: Long rant coming. And it may be a bit disjointed in thought.
Reputation is everything. As trite as it sounds, you can assume that the professors at Harvard are better than the professors at Maryland because Harvard draws the better professor candidates “on paper” and can likely offer them more money. Similarly, the majority of law students more than most other students, go to law school for the bottom line: a lucrative career. The USNR is a crib sheet for where little Bobby can go to school to maximize his chances to get the 160k firm job.
Let’s be honest for a second. I think it is faulty to think that the relative quality of legal education at law schools amounts to anything substantive. A Harvard grad is as ill prepared for real practice as a Catholic grad or any other grad. Measuring schools “objectively” by how good the actual classes are doesn’t translate to the practice of law. One of the huge criticisms of law school in the US is how it does not prepare you at all for “real life.” So ranking schools in something that doesn’t translate into success as a lawyer is pointless. I, for one, don’t think that my Torts class and how well it is taught is really going to matter all that much if I am personal injury litigator. What will matter more is everything I learn hands on at the firm, from my colleagues, and experience.
Since legal education is 6 of one/half dozen of another at any school, it is in the firm’s bests interests to stock their associate roster with name brands like Harvard. These are the “smartest” lawyers who probably can be taught “real” law easier and more cheaply than from other schools. And it’s smart business because if I was a deep pocket who needed representation, I would go to the firm with the most T14 lawyers. It’s the risk averse thing to do. Sure, there can be a top notch lawyer from Cooley, but do I really want to take the chance on a guy whose peers were sporting 145 LSATS and 2.5 GPA’s? No, I’ll play it safe and take the 175 LSAT with the 3.8 GPA.
Now, there are some people that think the T14 crowd are privileged slackers that muscle out hardworking students from lower ranked law schools. That’s sour grapes considering anyone would jump at the chance to attend one of those schools. And there are “slackers” at every school. Should I begrudge the kid who is naturally smarter than me or who worked harder in undergrad than me or figured out the LSAT better than me? Law is a meritocracy and 1L taught us all that bitter lesson from Day 1.
The people who complain about the rankings are usually hard working students at lesser ranked law schools (like me). And what we want is to be evaluated as a student without regard to our school. We want no rankings. We want all schools to be equal. But because that is the impossible dream, we concoct ways to try to convince everyone that our school is better. So instead of trying to sell ourselves, we sit around and devise ways to sell our school. So we feed into this rankings frenzy, but we try to slant it to benefit us.
I agree with Jack S. In fact, I’ll go one step further and say that the teaching is substantively the same at ALL ABA approved law schools. Sure, Yale may have more “star” teachers than Cooley. But most people learn exactly the same stuff, because of the standardized curriculum. The thing is, the law schools are just a very expensive sorting mechanism. That’s all they exist for, plus a little bit of education that is mostly unhelpful. So all the law school deans need to stop ranting about the USNWR, because that is all law schools are good for, just feeding the USNWR sorting mechanism.
We need to stop pretend that the rankings reflect the quality of the education. In fact, they only reflect the quality of students who matriculate at those schools. Really, it would be ridiculous to say that the average student at Cooley is anywhere near as smart or hardworking as the average kid at Yale. Completely ridiculous.
It makes every bit of sense in the world for firms to hire at the best schools they can, because those graduates were the hardest working/smartest/ did the best on the LSAT in undergrad. Some might argue that the LSAT is a bogus measure. I disagree. I think that one’s ability to figure out a meaningless test is actually a good indicator of one’s ability to figure out the law. That may be a sad commentary on the American political/legal systems, but there you go.
And really, if the USNWR stopped ranking schools, there would just be some other sort of ranking mechanism. So people need to stop getting angry at USNWR, because they are making the American legal education system efficient for everybody.
I appreciate the points that you make, CES and Jack S. If I were a hiring partner at a law firm I would lean towards the “top” schools in whatever rankings I respected most. Naturally there would be a ranking system if USNWR went away. And you’re right that criticism of the rankings has a hint of sour grapes mixed in with it.
Though there are plenty of critics of the rankings who went to T14 schools or teach at them and I continue to agree that the rankings discount the financial value component.
However, I’m not at all convinced that the teaching is qualitatively the same at all schools. Granted, I don’t know what the teaching is like at other schools. But I don’t think you can discount teaching quality and classroom experience simply because everyone learns the same thing.
The ABA should do rankings. They’re the ones who evaluate the schools for accreditation, so they’re in the best position to get all of the data and make the decisions fair.
That being said, there are far too many people with too much invested in the current system to keep that from ever happening.
Sherpa,
I appreciate your point that teaching is not the same everywhere. It’s almost definitely not and there are assuredly schools that do a better job at teaching than US News and other rankings recognize, if they even do at all.
My point, however, is that it doesn’t matter. The quality of your Contracts class is not going to give you a headstart on a career as a contracts attorney. Law schools classes at any school really don’t prepare you for practice. Firms know this and so it is in their best interest to take the safest approach and go for the “smart” kids, whether it be LSAT or class rank or quality of school. Rest assured that even in T4 schools, top 10% opens doors to good jobs that the majority of students couldn’t even sniff. I would venture a guess and say that top 10% at a T3/T4 school has better career prospects than a person last in their class at Georgetown.
I never liked you, Jack S. Ok. Maybe I do. Just not when I have to admit that you’re right… at least, in part.
It’s just that I don’t like the idea that my classes don’t really make a difference, yet my grades do. But I’m starting to think you’re right. 4 out of the 5 lawyers I’ve asked this week agree with you, Jack S. Obviously not scientific but a pretty decent qualitatively sample. Naturally, it makes sense that the real world is very different from the classroom.
I’m not about to say that I’m completely off-base. I think there’s something to my thought of teaching/classroom experience playing a role in the rankings. But it’s probably more useful for students who aren’t going to a top 25 or top 40 school. If anything, if you can find high quality teaching and classroom experiences, it seems to follow that you’ll get better grades (to your last sentence about being top 10% at a T3/T4 school). In other words, it might be a useful tool for distinguishing among the “bottom” 100 schools.
In an indirect way, the quality of instruction DOES matter. And I agree with your sentiment that if you have a dynamite contracts professor, it may not help you be a great contract attorney, but you will “get” what is expected of you quicker and your grade may reflect that.
And your quick survey of 5 attorneys jives with my own experiences. I am good friends with several lawyers and in talking with them the one thing they all seem to moan about was that law school was interesting but not at all relevant to actually practicing.
I also agree with you that your suggested approach may help with finer distinctions in the rankings and think it is more useful than you suggest. If you look at the rankings from the point of view of an employer, there are probably clusters of schools all throughout the rankings that are equal in their eyes. That being the case, it would be helpful for students to know the actual quality of the classroom experience.
For instance, if I wanted to practice in Texas and I was trying to decide between Baylor (USN 55) and U of Houston (USN 55), then yes, classroom experience would be a good tiebreaker, all other things being equal. But the worst Civ Pro class at Harvard is better than the Civ Pro class taught at Cooley by the ghost of John Marshall. When the rankings are far enough apart, almost any additional measurement other than prestige and reputation, is likely irrelevant.
[...] also note that Ms. Goodling’s attorney’s went to Emory and Michigan, so I guess that settles the ranking questions we’ve been kicking [...]