External Critiques (or Challenging Priors)

[cross-posted at Prawfsblawg]

Dave Fagundes just posted a nice discussion about the difference between internal and external critiques of a paper and how those critiques should be addressed at workshops. The internal critique challenges the internal logic of a thesis – whether it is supported by the evidence and argument. The external critique challenges priors – whether the thesis is normatively preferred, whether there is some better approach to finding a solution, or whether the thesis is even in the right field of study (e.g. whether physics has anything to say about a legal problem).

The post begins by arguing that external critiques are a valid challenge to a thesis, but ends by wondering whether such challenges should be made a workshops because they can derail the discussion into issues unrelated to the paper.  It seems to me that this is an interesting and difficult question – certainly interesting enough to write about while I procrastinate doing everything else I should be doing today.

Assuming for a moment that the workshop is the only way to raise the critique (maybe so for many attendees), I think the answer is that external critiques should be made, for reasons I discuss below. At the same time, the speaker must exert some skill to avoid being derailed.

Dave is right that external critiques are valid critiques. The art of a well drafted paper is determining just how much of the external critiques to address, and in what depth. One reason I like to write essays is that the shorter form allows for more hand waving: “I realize there are many external critiques, but this is an essay introducing an idea, and thus I’m not going to address any of them.”

For longer articles, however, every scholar must determine just how much of an external critique to address. If you are an economist writing a law and economics paper, you probably don’t need to spend a lot of time justifying Kaldor-Hicks.

If, however, you are a patent scholar arguing that software should be patentable as a doctrinal matter, you might need to spend more time addressing external critiques of software patents, such as harm to innovation. There are a few choices:

1) ignore it (probably not a good choice)

2) address it and dismiss it (“some argue that software patents harm innovation; while this may be so, as a doctrinal matter they are allowed”)

3) address it and confront it (“some say software patents harm innovation; the evidence does not support this inference because….”)

Options 2 and 3 have their own pitfalls. Dismissing the critique can be harmful because there may be no other way to stop the problem but for some external solution. For example, since Congress is unlikely to change the law, courts outlawing software patents may be the only way to stop harm to innovation. On the other hand, going into a lot of detail about a tangential issue will just waste paper, distract the reader, and probably not convince those who disagree with you anyway.

This, I think, is why external critiques are fair game (and valuable) at workshops. First, you may not have realized the critique existed (shame!), and thus by not mentioning it you look like you ignored it. As one mentor told me early on, one should be expected to find the critiques in his or her own field, but the benefit of workshopping is to find the critiques in other fields.

Second, the workshop can help you determine whether option 2 or option 3 is the better way to approach the problem. If you find the discussion being derailed, then you probably need to go with option 2. If you can quickly address the critique with some evidence, then perhaps option 3 will work, even in a footnote.

Even the basic critique – such as whether Kaldor-Hicks is normatively preferable – can lead to an improvement of the paper. If you are assuming a particular normative framework, it is better to make that clear so that others who do not share your worldview at least understand where you are coming from. The purpose of the workshop is to find out if you missed a worldview, and that can be a good thing.