Perhaps appropriately on the last day of this fascinating stream of mobbloging, I thought I would try and offer a partial defense of the-way-things-are-right now on the law review front:
- Don’t romanticize the alternative: When one begins to publish in the peer-reviewed world, the whole romantic notion of blind review becomes somewhat tainted — in all of the fields i am familiar with — psychology, economics, business, political sci, history and sociology reviews (and also in the fields i am less familiar with, according to my hard scientist friends) there is still A LOT of noise in the system: because of conferences, networks, ssrn and the like, it is VERY easy to figure out for reviewers who the authors are; there is a lot of tit-for-tat — same folks publishing and editing in the A-journals in their fields. To think that a top name (the Cass Sunstein equivalent in any one of those fields) does not impact the decision of her peers is naive. And even more so because the double blindness from the author’s side is also often tainted — at times, people can guess pretty easily (or hear about it in the process) who the reviewers are. At least with law review editors, the repeat players problem (which of course has many strengths as well in terms of expertise and knowledge about the field and quality of scholarship).
- There are in fact quite a bit of law or law related peer-reviewed journal so we are already live in a world of choice and a world of competition.
- Law reviews themselves do consult quite frequently it seems with faculty advisors on accepting pieces. It is also my understanding that some of the top law reviews, like the Harvard Law Review, do really have blind review process, and their process is a multi-layered, multi-tiered serious on.
- The law review kids actually offer a lot of good editing after the article has been accepted, often better, and most of the time involving more eyeballs, than those of peer-reviewed journals.
- I suspect us law professors, like other faculty, would not like Ann’s suggestion to take away the prestige, the wins and losses of the game, of sending out one’s article to journals outside one’s own institution. Our work is intangible enough to not want to give away some passing rewards and signals of success however noisy they are. This is similar to suggesting that we do away with rankings — we should keep rankings, just make them far more meaningful. Remember, type A people strive on competition.
- There is something really cool about the speed and timeliness in which law reviews make decisions and then publish articles. Peer-review articles often take years upon years to come out, the review process may mean that things are not relevant by the time the article sees the light of day.
- There is also something cool about law review editors having some productive ignorance. Its ok if they sometimes get things “wrong” — its part of what shakes things up, puts some off ideas on the frontlines and disrupts some of the inner-breeding and self-referential work that at times stales the legal academy.
Bottom-line, the baby water and the baby have some advantages.
First of all, I have been too busy with end-of-term stuff to focus on this wonderful conversation as much as I would like. But it’s all preserved for another day. Congrats, Mike, for making this the center of the blogosphere for a while.
Orly, I’d agree that nothing ought to be romanticized, but I will simply offer the narrative that blind peer review has several distinct advantages, particularly in this brave new world of cross-disciplinarity. Assuming the system itself has integrity, in the sense that one would only know the author by way of form or substance of the piece being reviewed, and not by cracks in the blinds, it does serve the purpose of validating work for new or not-well-known authors, or authors who are at “lower-tier” institutions, or authors whose CVs otherwise might underwhelm the student editors. Not to be too modest about it, but I just today had a piece accepted by a peer-reviewed journal, and I don’t have any particular confidence that, given its length and subject matter, that its crossing the desk of a “top 15” review articles editor accompanied by my CV would have sent bells a-ringing.
My other observation, having felt this in the pit of my stomach when submitting to peer review, is that the prospect of failure in the placement is higher, and more significant than with the student-edited journals, particularly when the decision is accompanied by the reviewers’ comments.
I guess, all in all, I think blind peer review is not the end-all, but it’s more likely than the current system to be based on intrinsic merit versus recursive reputational effect.
Thanks for this post.
I hear the complaint with some frequency that reviewers will know the identity of the authors they’re reviewing. I’m not so sure that’s so true. It hasn’t been in my experience. Of the peer reviews I’ve done for history journals, I knew the identity of only one of the authors–and that was happenstance (I saw him give a conference paper on the topic). Maybe if I were better connected, I’d know more. But I think there’s a lot of anonymity that remains in the system. Moreover, I know the identity of very few of the people whose reviews I’ve received (in one case, someone waived anonymity; in a couple of other cases, where there was single-blind review, I’ve heard from reviewers).
It’s not just the blind review process that makes peer review superior, of course; it’s that the reviewers presumably (and in my experience this has been the case) are well-informed on the subject, so that good work gets recognized (or poor work does not).
To respond to the April 11 post about law reviews, “Because things are not so bad the way they are. . .(on the law review front).â€
Well, yes, they are, and it’s even worse than you may think.
Peer review, first, is not only the norm, but the gold standard for professional publications in literally every other academic discipline. The reason for this is fairly obvious: it is the best system yet devised to judge the merit of academic writing. Is peer review perfect or fool-proof? Of course not. But it has two unassailable advantages not found (with a literal handful of exceptions) in the world of law reviews: first, publication decisions are made by professionals with subject matter expertise, which means that the submitted work can be evaluated on its merits; and second, the review process is normally blind, meaning that the reviewers’ names are not known to the author, so that the reviewers (under professional editorial control) can offer full and frank evaluations. The primary check against possible abuse by vengeful or unfair reviewers is knowledgeable editors who can weigh the relative merits of reviewer comments, or solicit additional reviews if needed.
Compare this to student-run law reviews, where publication decisions are made by hard working and well-intentioned people, but who are students. That is, they do not possess, and cannot be expected to possess, the knowledge to evaluate submitted work on its merits. Yes, they can find missing punctuation and erroneous citations, but these minor technical errors pale in the light of the fact that they are unqualified to judge whether a submission makes a legitimate argument, reflects proper knowledge of the field, does or does not duplicate existing writing, or even presents arguments and facts fairly and correctly.
While it is at least possible for authors to figure out who a given reviewer might be, I know of no evidence to support the idea that such revelations are easy to discern or are frequently discerned. Quite the contrary. The most likely way that a reviewer’s identity would be discerned is if the reviewer deliberately reveals specific information in the review that allows the author to identify the person. But again, there’s simply no evidence to support the idea that this occurs beyond the rarest instances. And my own experience of publishing dozens of articles in peer reviewed journals, and reviewing many dozens of articles for many different journals, is that revelations just don’t occur.
Among the over 600 publications emanating from the nation’s 220 law schools, no more than a literal handful rely on peer review, and if I’m not mistaken, none of the top law reviews have peer review in the way that term is defined in every other academic discipline. Student law review editors certainly do consult with departmental faculty at times, depending on the publication and the institution, but this is not peer review, either.
It is certainly true that law reviews can publish quickly compared to other disciplines, but two considerations undercut this potential virtue. First, there are many ways to publish an important idea rapidly, either in professional publications that specialize in reporting new or important findings (most disciplines have such publications, including my discipline of political science), or as newspaper op-eds, magazine articles, or on the internet; second, and more important, the virtue of rapid publication is negated if what is published is defective or wrong. As others have noted, the maxim of the medical profession applies to scholarship, too: first, do no harm. Better to be slow and right than fast but wrong.
This new book (see link below; disclosure: I’m the author) argues that these very traits of law review publishing lead to the formation of wayward theories about the Constitution:
http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521721721
I’m just going to chime in with three comments here.
(1) Peer review does a good job in the middle of a field and a very poor job at the edges and interdisciplinarily. It also does a very poor job in developing fields that are still attempting to develop an identity. Consider, for example, what a “cyberspace law” journal might have looked like in 1992… and when you’ve stopped laughing/cringing, try delving into the peer-reviewed journals in literary studies of the 1960s and early 1970s, or the peer-reviewed bioinorganic chemistry journals of the 1970s and 1980s. It’s not a pretty picture.
Peer review’s value is in pointing out that someone else has already either made or refuted the same argument. Of course, this is rather ironic in law, because so much of US legal writing (even in academia) is founded on the idea that an argument without citations to prior authority is worthless. It’s an almost Swiftian (Battle of the Books) irony: Peer review does a much less satisfactory job of pointing out substantive weakness than it does of pointing out doctrinal ignorance.
(2) I could nonetheless buy a peer-review system IF, and only if, it also kept the primary attraction of the current law-review submission system: Simultaneous submission. That, however, also means that a given reviewer might be asked to review the same work for different publications, which will require some administrative creativity.
(3) No “reform” of the submission/editorial system in the law reviews will make any difference unless we demand better writing. As a rule, legal writing is a mess, and that’s putting it very kindly for a family-friendly forum. It’s rather ironic that a profession that lives and dies by the written word writes so badly; it’s rather disturbing that things are no better among those who teach the subject than among those who practice it for billable hours.
N.B. I’m a refugee from peer review in other fields, on both the scholar’s and the editor’s sides of the desk.
Comments are closed.