**Updated September 16: I am grateful to JDH for publishing these posts along with various responses. Soon after my posts were published, JDH approached Roopika and I to ask if we would approve publishing all the correspondence, with identifying information redacted. We asked our contributors for their opinions, and as a group came to the decision that it would fracture our field further to make all the correspondence public.
We offered to explore the option of co-publishing a post on best practices and lessons learned with experimental forms of publication with JDH, but the option was declined. However, we remain hopeful that this option to work with JDH to work on a co-authored piece may later become available. Thank you all for weighing in.
For those just jumping into the discussion, I’ve published a follow-up blog post here, “Crowdsourcing Best Practices for Experimental Practices: Transparency.” I hope you will join in and offer suggestions.
The problems of traditional peer review are well known. Peer review is not transparent; it takes too long; the true blindness of peer review is questionable, especially in small fields; its gatekeeping function encourages the conservatism of scholarship. To address these concerns, the Journal of Digital Humanities published its first issue in Winter 2011 as an effort to work through some of these problems. JDH champions a “post-publication” model of review where its editors collate some of the best existing DH work and publish it in journal form. How well does this publication model work in practice?
As my collaborator Roopika Risam and I discovered, the JDH model is far from transparent and, if anything, can intensify some of the problems of traditional peer review by inflating expectations around transparency. While working with JDH to publish a special section on the postcolonial digital humanities (#dhpoco), we learned that:
The journal promised us one process for working with them, and once we had submitted everything, presented a completely new one; and that
This sudden about-turn suggests that the journal is not fulfilling its mission to offer an improvement over traditional peer review.
Roopika Risam and I approached JDH to publish a special section on the postcolonial digital humanities (#dhpoco) in July. They appeared enthusiastic, asking us to aim for a very ambitious four-week deadline. It was agreed that we had four weeks to solicit revised versions of already published material to be submitted to JDH in four weeks for line editing and minor proofreading. Three weeks after submission the special section was to be published. (See the images at the end of the post for the agreed-upon process.) The timeline was short, but we pushed our authors to get the work done in time, and they did.
Five days after submitting our special section, Roopika and I were surprised to receive an email saying that the journal suddenly wanted us to submit our special section to a blind external review. On the one hand, we would have been glad for review, knowing that it would only make our work better. But upon further discussion, we started thinking about the following:
Why was this request not made up front? I have had multiple dealings with special sections/issues of peer reviewed journals, and if something is marked as such, it does not go through external peer review unless this is requested at the outset.
Since JDH emphasizes a “post-publication” model, and our essays had already been available in blog posts and forum responses, didn’t our editorial process serve as the second round of collation and post-publication–as the journal claims is its experimental role? From speaking with other special section editors, none had undergone a similar process. One person even related that she received an email from JDH asking to publish her already-published blog post in a week, and if she would make revisions that would be great, if not, they would publish it anyway.
I raise these claims to highlight JDH’s apparent lack of transparency on the following issues:
We appeared to be the only special section ever singled out for external peer review
JDH promised us one process, but then insisted upon another one
All of our essays had been published on the Internet through blog posts, the #dhpoco website and the summer school. This meant that any external review could hardly be blind. Why did JDH think that this would be blind, given that names and previous forms of our essays were already freely available online?
If blind peer review was not possible, why did JDH want to appoint a blind reviewer? Since it is an experimental journal, we would have been amenable to an open peer review.
Is JDH’s Post-Publication Model an Improvement Upon The Current Model for Peer Review?
This leads me to the following question: Is JDH’s post-publication model an improvement over the existing peer review model?
Our experience with JDH has been anything but transparent. How, then, does this journal further claims about making the process of peer review more transparent?
JDH seems to publish work by the same people, multiple times–and from what I have heard anecdotally, they have been asked for minimal revision, and for no external review. Do our experiences indicate a–hopefully unconscious–bias within the journal?
If this bias exists, how is JDH’s model an improvement over traditional peer review? What is the process that determines what posts needs to go through external review, and what doesn’t?
Has JDH put any measures in place to ensure that its policies are transparent, fair and unbiased?
These types of problems are not exclusive to JDH. They concern any publication offering an alternative model to peer review. I believe that we do need to fix the existing issues within the system of peer review. But if what we experienced is an alternative model, it does not offer an improvement over the original system. This is even more disturbing because by being a pioneer in alternative forms of peer review, JDH could serve as a role model for other outlets wanting to experiment.
These new experimental journals will need to find a way to address these problems. By suddenly changing their process, JDH wasted a lot of our time and the time of our contributors, who had to rush to put the pieces together to meet their deadline. If we had known about the external review to begin with, we would have not had to rush our writers. Their sudden reversal broke our trust in their process, a necessary quality to have for any new form of peer review to work.
I hope that I am mistaken about JDH. Again, I am sure that our essays–as would every piece of work that has already been published–would benefit from review. What I am not sure about, however, is why we were singled out for special treatment, and why the journal had not been transparent with us from the beginning. This makes me worry about the new forms of review that the journal claims to champion and about the unseemly ways it can be put to use. However, we do believe that the review process is a useful one. We are taking our special section to another venue where it will be peer reviewed.
Going forward, I hope that JDH will consider making its measures for fairness and transparency public, and to implement some checks and balances to provide some accountability for these measures. We need to do better.
** I thank Roopika Risam, Natalia Cecire, Alex Gil, Jesse Stommel and Fiona Barnett for their comments on my previous draft.
Responses to this Post
- Michelle Moravec, “I Already Know What Happened and I Wasn’t Even There”
- Laurie N. Taylor, “Procedural Equity and Peer Review”
- Siobhan Senier, “Can DH be Decolonized?”
Previous Correspondence With JDH