Making Peer Review More Open

“Marginalia” by Open Library is licensed under CC BY 2.0.

Traditional peer review relies on anonymous reviewers to thoughtfully assess and critique an author’s work. The idea is that blind review makes the evaluation process more fair and impartial–but many scholars have questioned whether this is always the case.

Open review has the potential to make scholarship more transparent and more collaborative. It also makes it easier for researchers to get credit for the work they do reviewing the scholarship of their peers. Publishers in both the sciences and the humanities and social sciences have been experimenting with open review for almost two decades now, but it is only recently that open review seems to have reached a tipping point. So what exactly is open review, and what does it entail?

There are two main types of open review: named review and crowd-sourced review. With named review, the names of the peer reviewers as well as their reports are published online alongside the scholarship in question, making them available for anyone to read. A number of open access journal publishers use named review, including BioMed Central, Frontiers, F1000, and eLife. The open access journal Nature Communications allows authors to choose whether or not they want open review. About 60% of them do so. Elsevier recently started allowing peer-review reports to be published alongside articles for a few of its journals. However, peer reviewers do not have to attach their names to reports (and a little less than half do so). Giving reviewers the choice to remain anonymous if they wish, while making the reports open, can be one way to address one of the main critiques of open review: that it will cause scholars to be less candid. Early career faculty might be particularly sensitive to this issue.

With crowd-sourced review, a draft of the article or book is made available online for the public to comment on before it is officially published. This allows for the authors to get feedback from a greater variety of individuals, including people who might never have been approached to be a peer reviewer. One of the first monographs to go through this type of review was Kathleen Fitzpatrick’s Planned Obsolescence (NYU Press, 2010). MediaCommons Press, which hosted this draft of Fitzpatrick’s book, used CommentPress, a WordPress plugin, to facilitate feedback from interested readers. Most recently, Matthew J. Salganik used crowd-sourced review for his book, Bit By Bit: Social Research in the Digital Age (Princeton University Press, 2017). Salganik created the Open Review Toolkit (which uses the annotation tool hypothes.is) in order to help other scholars do the same thing. Crowd-sourced review can occur before traditional peer review, or concurrently. One of the challenges associated with this type of review is letting the world know that the manuscript is online and ready to be commented on. It may require that the author take a more active role in the review process than they are used to. Another challenge is directing readers to leave the type of substantial comments that would be most useful to the author.

In addition to these more formal types of open review, which are usually facilitated by publishers, researchers are also taking it upon themselves to use the web in different ways to solicit feedback on their work. More and more scholars, particularly in the sciences, are posting preprints. Some humanities scholars are even sharing drafts of their work in public Google docs and enabling commenting.

What do you think of open review? Would you be willing to make your peer-review reports public?

Your Preprint Questions, Answered

“Early bird” by Katy Warner is licensed under CC BY-SA 2.0.

Last year, we noted that preprints were “having a moment.” Since that time, a number of new discipline-specific preprint servers have launched (PsyArXiv, LawArXiv, and engrXiv), and more are on the way (Chemrxiv, PaleorXiv, and SportRxiv, to name a few). In addition, funding organizations, such as the Chan Zuckerberg Initiative, have begun to provide financial support for preprint servers. Still doubt the rising popularity of preprints? There’s even a new app for rating preprints in the life sciences called Papr, which calls itself “Tinder for preprints.”

Are you thinking of posting a preprint? Here are some things you might be wondering about:

What exactly is a preprint?

A preprint is usually defined as a piece of scholarship that has not been peer reviewed or formally published. Many preprints do go on to be published in academic journals. One 2013 study, for example, found that 64% of the work that is posted in arXiv has been published in academic journals. However, there is also small group of scholars who have begun posting what they call “final version preprints.”

Why should I post a preprint of my work?

Posting a preprint allows you to get your research out into the world quickly and easily. That’s good for the advancement of knowledge, but it’s also good because it enables you to position yourself as the originator of a certain claim or technique, even before your article is formally published. Posting a preprint is also a great way to get feedback on your work from others, and make your scholarship even better.

Can I still submit my manuscript to a journal if I previously posted it on a preprint server?

In most cases, yes. A growing number of journals welcome manuscript submissions that first appeared as preprints. BioRxiv, for example, has a manuscript transfer process which makes it easy for researchers to submit their preprint to over 120 scholarly journals. That having been said, there are still a few journals that consider the posting of preprints to be “prior publication.” Make sure to read the policies of the journal you are interested in submitting to. Wikipedia also has a list preprint policies by journal.

How will people find my preprint?

Many preprint servers assign DOIs (digital object identifiers) to preprints, which make them easier to discover (although the popular arXiv does not). In addition, a number of preprint servers are indexed by Google Scholar. Nevertheless, if you want people to read your preprint, you should be prepared to do your own promotion. Use social media to draw attention to your work.

How should I license my preprint?

As the author, you automatically own the copyright to your work. However, adding a Creative Commons (CC) license tells people how your preprint can be reused. Some preprint servers require a CC license for any work that is posted. Others, such as SSRN or Humanities Commons CORE, do not. We recommend adding a CC license to all preprints you post.

Can I cite a preprint?

Yes. If you have evaluated a preprint and find it to useful to your research, definitely go ahead and cite it. Just make sure to note in your citation that it is a preprint. Also make sure you are citing the version that you actually used. One caveat: there are a few journals that do not allow researchers to cite preprints, although this policy seems to be changing. If you are unsure, ask your editor. Writing a grant application? The NIH recently announced that investigators are free to cite their own preprints in research proposals or projects reports.

Have another question about preprints that we didn’t answer? Let us know in the comments.

2016-2017 Recipients of the OA Publishing Fund

“Open” by Auntie P is licensed under CC BY-NC-SA 2.0.

In the fall of 2016 the Libraries launched a pilot Open Access Publishing Fund to support faculty and graduate students who want to publish their research in an open access journal but do not have the money to do so. We have profiled a few of our recipients in past posts on this blog. We are pleased to announce the complete list of 2016-2017 recipients. Congrats to all!

College of Liberal Arts

Eunice Chen, Lauren Alloy, Susan Murray, Jared O’Garro-Moore, Angelina Yiu, and Kalina Eneva, Psychology

Kevin A. Henry and Allison L. Swiecki-Sikora, Geography and Urban Studies

Joshua Klugman, Sociology/Psychology

College of Education

Doug Lombardi and Janelle Bailey

College of Science and Technology

Xiaoxing Xi, Narendra Acharya, Matthaeus Wolak, Teng Tan, and Namhoon Lee, Physics

Xiaoxing Xi, Teng Tan, and Matthaeus Wolak, Physics

College of Engineering

Fei Ren and Bosen Quan

Shenqiang Ren, Beibei Xu, Himanshu Chakraborty, Vivek K. Yadav, Zhuolei Zhang, and Michael L. Klein (Physics)

School of Medicine

Andrew Gassman and Judy Pan

Andrew Gassman and Richard Tyrell

Jian Huang, Chao Wu, and Hong Wang

Parth Rali

He Wang, Baidarbhi Chakraborty, Linda Mamone, Shiguang Lui, and Nirag C. Jhala

School of Dentistry

Vinodh Bhoopathi and Ting Dai

A Few Cues About Peer Review

The following is a guest post from Ryan Mulligan, Editor at Temple University Press.

One way a scholar might come into contact with a university press is through being asked by an editor to serve as a peer reviewer for a book manuscript the press is considering publishing. Peer review is essential to checking the scholarly credentials of the books university presses publish and helps ensure that they will find an appreciative audience and have the bona fides to be assets to the academic community. Here are some considerations you might wonder about if you are asked to serve as a peer reviewer.

Good peer reviewers know that they are writing for two audiences: the press (including its editorial board) and the author. They balance their suggestions for the author with evaluation, and even some summary, for the press’s benefit. It is sometimes useful to clarify, for the editor and the editorial board’s benefit, the background into which the manuscript enters and what is at stake in whatever intervention it seeks to make. Good peer reviews note what is new and impactful about the project and point out any weaknesses in the scholarship. They might consider the manuscript’s suitability for course use, the level of the writing, or what set of scholars and fields constitute the manuscript’s likely audience. Beyond who could theoretically find this book relevant, realistically, if the press published this book, is there an appreciable set of scholars who will consider it their job to read it? Reviewers’ comments should also be useful to the author as they revise the book. Have they considered the relevant counter arguments? Are there relevant sources or alternative explanations they haven’t considered? Have they defined their terms sufficiently? Are they inventing terms that aren’t useful? And then the reviewer might need to play editor. Does the argument flow? Does the organization of the manuscript make sense? As a reader, do you feel confident at a given moment in the text, where the author is going and how the section you’re reading contributes to the overall argument?

You don’t need to be a copyeditor – most university presses will take care of the spelling and grammar in a later stage of the publishing process. But especially in fields that are heavy on technical language and jargon, the press may be dependent on you to make some editorial interventions. I see many reports that offer a main section of broad discussion of the manuscript and then a separate section of more detailed comments that tackles individual errors or points of weakness one by one. Some reviewers like to open up the “Comments” section of their word processor and mark up the manuscript as they go. There’s no way for me as an editor to present those comments to my board, but many authors find this generative, so go for it if you like, but be cautioned that it’s much harder for me to keep your identity as a reviewer anonymous from the author when I pass on these comments.

I’ve seen some effective concise reviews of only a page, but most good reviews tend to be two to five pages, more if they catalogue errors or offer detailed commentary. Some reviews are framed as letters to the editor, press, or editorial board, but most are just reports not addressed to anyone in particular. Some presses, including Temple, have a list of questions they will include when they send along the manuscript for review. Reviewers can consider these questions one at a time in turn or just keep them all in mind as they write a more free-form review. Presses, editorial assistants especially, appreciate when the reviewer remembers to turn in any supporting forms completed with their review. They also appreciate reviewers sticking to their deadlines, but being forthcoming and communicative when they can’t make a deadline.

Let’s talk for just a moment about bad peer reviews! The worst reviews I’ve seen are a couple paragraphs thrown together before deadline that just say “you should publish this book.” That’s not going to help the book get past the editorial board; the press is just going to have to turn right around and ask someone else to write a whole new review. But that doesn’t happen very often. Most commonly, a peer review will disappoint me in that its criticisms boil down to, “this isn’t the book I would have written if I’d tackled this topic myself.” I understand the instinct to write that report. But it’s not useful to the press. You will naturally have certain interests that you would have brought to the study to nudge it in a different direction that would be more exciting to you. Your focus and emphasis might have been different. And your writerly voice would of course have been different. But try to put yourself in the press’s shoes and evaluate what the book does for the discipline, what it brings to the table, and how it could be framed to better accomplish its own goals, which may be different from yours.

I’ve seen peer reviews from scholars of all levels of experience; seniority is no predictor of the quality of a review. The quality of the review comes from the reviewer’s ability to speak to both author and press, consider the book’s realistic impact and audience, evaluate its success in reaching its goals, and offer helpful suggestions for improvement. I hope my own suggestions here are helpful for anyone asked to review a book project for a scholarly press.

For more perspectives on peer review for academic presses, including some of the ways that procedures might vary for different presses, take a look at the Association of American University Presses handbook of Best Practices for Peer Review.

A New Collaborative Model for Open Textbook Publishing

“Supporting OER [open educational resources] means supporting maximum equity and access within education, allowing all students to learn with the up-to-date content, regardless of their economic background” – Cable Green, Director of Open Education, Creative Commons

Adopting open textbooks saves students around the world millions of dollars and allows faculty to share their knowledge globally. Most open textbooks use a CC BY attribution license from Creative Commons that makes it easy to share and adapt the content.

Many aspects go into creating an open textbook, so why not crowdsource talent? That is exactly what is happening with an exciting new model for open textbook publishing created by the co-founders of The Rebus Foundation, a Canadian, not-for-profit organization funded by a grant from the William and Flora Hewlett Foundation. Executive Director Hugh McGuire and Strategic Director Boris Anthony have created an online collaborative community called The Rebus Community, to create a global, connected network of open textbook creators. All are welcome to join the community and contribute ideas for open textbooks or to become a contributor. It is the perfect way to support the creation of OER without necessarily having a lot of experience in open textbook publishing. They develop the software and tools to manage the publishing process; all you need to do is sign up! All open textbooks will be available in an open, remixable e-book format supported by Pressbooks.

Current pilot projects underway include:

The Rebus Community seeks a variety of collaborators for all stages of open textbook publishing and may include everything from chapter authors to peer reviewers and proofreaders. To stay alerted of open calls for contributors, join the Rebus Community forum at http://forum.rebus.community or follow their tweets @RebusCommunity.

Fair Use from a Scholarly Publisher’s Perspective

This week we’re celebrating Fair Use Week at the Libraries. To find out more about the role of fair use in scholarly book publishing, we interviewed Nikki Miller, Rights and Contracts Coordinator at Temple University Press.

Why is fair use important to the Press?
Fair use is important to the Press for a few reasons. From a practical standpoint, obtaining permissions tends to be costly and time-consuming, so fair use allows authors to build upon others’ ideas without the added cost and time expense of gaining permission. Our mission at the Press is to participate in the dissemination of academic discourse, and without fair use, it would make this a lot harder to accomplish.

Authors are generally required to tell their publisher what third party materials they want to use in their book. They are also usually responsible for obtaining permissions. Is this how it works at the Press?
Yes, we have a clause in our standard contracts that states it is the author’s responsibility to gain permission for any copyrighted materials. Though I am always happy to discuss with our authors what material needs permission and offer to help him/her to find the appropriate rights’ holder. It is not always easy to find the rights’ holder so having more than one person searching can make the process quicker and easier.

Do you double-check every single article/image/table authors use to make sure they really have gotten permission?
I do. As soon as an image or text is deemed to be not fair use, I flag it and discuss options with the author. The Press has a standard permission form that we send to authors to send to the rights’ holders; this form grants us all the rights we need to move forward including others that we like to have, such as promotional use. If the rights’ holder has its own permission form, I read it to make sure it gives us all of the rights we need to include it in the author’s publication.

Would you accept a fair use argument from an author? Under what circumstances?
I would only accept a fair use argument for the inclusion of text, never for an image or table. Unless the author uses data from an original figure to create his/her own table, as that qualifies as transformative use. However, I have had authors argue that their usage of copyrighted material is fair use when I think it is not. When this happens, I listen to the author’s argument, and we usually reach a compromise with what to do next. Most of the time this occurs when the ratio of quoted material to analysis is too low, so I ask the author to both cut some of the quote and to add additional analysis to his/her argument.

When analyzing if borrowed material is fair use, I tend to be stricter on the analysis of poetry and song lyrics. This is because a lot of the times poetry and songs are carefully kept under copyright and are usually shorter as works. However, there are times that I deem poetry lines as fair use. In those instances, a very short piece of the poem is included and has a lot of analysis to accompany it. Additional factors I consider with poetry include whether the author is using it as ornamental text in the body of the work and if the author needs the exact language for his/her argument. If it is used for ornamental purposes in the body of the work then I ask the author to remove the poetry.

How do you evaluate whether use of Press content constitutes fair use?
When we receive permission requests from authors to include material from Temple University Press titles in their works, I consider the same four fair use factors as when I evaluate our own authors’ manuscripts and whether the borrowed material they include is fair use or not (purpose and character of the use, nature of the copyrighted work, amount and substantiality of the portion used, and the effect of the use on the market). If the requested material does not fall under fair use, I grant permission (if TUP has permission to do so) to the requestor. I have received permission requests before for material that will qualify as fair use, and in those situations, I let the requestor know and ask that a proper citation is included in the work.

What do you wish more authors knew about fair use?
I get a lot of notes from authors who think that because they cited borrowed material appropriately that the citation automatically deems material fair use and makes gaining permission unnecessary. Citing has no role in deciding whether the inclusion of borrowed material qualifies as fair use or not. Plagiarism and copyright infringement are not the same thing. Citing protects an author form plagiarizing, and only granted permission or fair use protects an author from infringing on copyright.

Thank you Nikki!

Owning Your Impact

ownyourimpact

“Measure a thousand times, cut once” by Sonny Abesamis is licensed under CC BY 2.0.

Scholars are routinely called upon to demonstrate the impact of their research, whether for tenure and promotion, or for grant and fellowship applications. Traditionally, citation counts and journal impact factors were used to determine research impact. Today, there is widespread acknowledgement that both of these metrics are seriously flawed. One recent study, for example, showed that men cite their own papers 56% more than women (which means some men may have inflated citation counts). Another study pointed out that publishing in a journal with a high impact factor does not necessarily mean that your own work will be highly cited. Still, it’s clear that traditional metrics are not going away. Just last December, the publisher Elsevier launched their own journal impact calculator, CiteScore.

Metrics can cause significant anxiety among scholars. While such anxiety is understandable (no one wants to be reduced to a number), the more proactive a scholar is when it comes to documenting their impact, the better off they will be. Scholars should learn to take control of their metrics, and use them to craft their own story about their research. There are four main ways to do this:

First, build and maintain your online presence. Make sure your faculty profile on your department page is up to date. Register for an ORCID iD and use it. Create profiles on various academic social networks, and on Google Scholar. Consider having your own website. Join Twitter and connect with colleagues around the world.

Second, make as much of your work openly available as you can. People can’t cite or talk about your work if they can’t read it. Publish your research in open access journals, or post a preprint or postprint to a disciplinary repository. Can’t do either of these things? Figure out alternative ways of sharing your scholarship. Blog or tweet about research in progress. Use Figshare or Slideshare to call attention to unpublished scholarly output. Or, ask your publisher if you can share a small part of your monograph (such as the table of contents) on an academic social networking site (such as Academia.edu or ResearchGate).

Third, keep track of your citations. Regularly check your citations on Web of Science, Scopus, and Google Scholar. But be skeptical of the results, particularly if you are a humanities scholar. Edited collections, for example, are often not indexed. New journals may also not be indexed. Ultimately, your best bet is to do your own research on who is citing you by searching the appropriate databases for your field (such as JSTOR, Proquest, or Google Books). Once you have a list of all your citations, dig a little deeper. Quantity matters, but so too does quality. If you can show that you are being cited by leaders in your field, or by scholars from outside your discipline, you can make a stronger case for your impact.

Fourth, pay attention to altmetrics. Altmetrics can provide you with more details about your research impact than citation counts alone. Most sharing platforms make it easy for you to see how many people are viewing or downloading your scholarship (disciplinary repositories usually offer scholars similar analytics). Sign up for ImpactStory to help keep track of what people are saying about your work on social media. Try searching the Open Syllabus Explorer to see the frequency with which your book or article is taught and what related work is taught alongside it. Here again, however, it’s important to remember that existing tools can only tell you so much. You must do additional research to really find out all the ways in which your work is being used.

For more information on enhancing your impact, check out the Library’s guide to the topic.