Home » A Statistician's View, Departments

A Tale of Two Researchers

1 October 2012 One Comment

Here is a tale of two researchers living in side-by-side universes. They are working on the same research project. However, one of them lives with a new form of academic publishing. Both academics are named Jane.

The New Universe

Jane 2.0 has discovered something important and she publishes her research on a social network dedicated to academic publishing. The network doesn’t have to be like Facebook, MySpace, LinkedIn, or Twitter. Several websites have found the advantages of linking users without making the network the central focus of the site. For example, YouTube and Yelp both host user-generated content.

The network automatically alerts her colleagues to her new publication, and it automatically allows for a discussion thread beneath the link to her paper. Her research is controversial. It sparks a discussion among the researchers who follow her work. After discussing the problem for a couple days, the handful of commenters (who are not anonymous, but are well-known researchers to Jane) agree she needs additional evidence. They suggest an additional simulation experiment.

Since Jane’s research has attracted quick attention, the network starts to suggest her paper to others who might be interested. At the same time, Jane runs the other experiment and links the results as an addendum to her paper. This link appears directly next to the original paper; everyone who has previously downloaded her paper is alerted to the addendum. (Two days later, she posts a revised version of her paper that incorporates this addendum.) This additional evidence addresses the controversy and several of the researchers who joined the discussion “republish” Jane’s research on their own pages. The people who follow these other researchers will be notified of Jane’s research.

Jane’s paper is thought to open up several interesting lines of inquiry. In the two weeks after Jane 2.0 first published her research, she has gone through a round of revisions and her paper has gone viral in her community. One year later, there are 43 other papers that follow up on her research. They have been published on the network and are all linked in a common thread of papers. In that time, the editors of two journals have sent a message to Jane 2.0 asking if they may conduct a formal peer review. Jane selects which journal she prefers, the article is reviewed and accepted, and then linked on the journal’s page inside the network. The people who follow these journals are alerted to her research. This propels her research further into the network. Two years later, there are 211 follow-up papers. Compare this story to Jane 1.0.

The Current Universe

Jane 1.0 lives in the present. When Jane 1.0 wants to publish her research, she emails the research to her colleagues. They respond individually, unaware of the others’ comments. They all think additional work is needed for publication. However, because the commenters are not aware of the others’ comments, no individual colleague can identify the controversy that Jane 2.0 was able to address quickly.

Jane 1.0 also posts her research in several places: (1) on her web page, (2) as a technical report on her department’s page, and (3) on arXiv. Additionally, she submits her research to a prestigious journal. Three months later, she receives a rejection from the journal. Fortunately, the referee reports lead her to discover the reason people are so frustrated. She runs the additional experiment (that Jane 2.0 ran within a few days of first posting her research) and resubmits to the prestigious journal with a long letter explaining why she is resubmitting after being rejected. This letter sits on the editor’s desk for at least a week before the editor decides to send it out again. After sitting on the referees’ desk for another three weeks, they uniformly agree that this is groundbreaking research. After worrying about typesetting and other typographical issues, the paper is printed in the journal six months later. Since not all researchers read through the abstracts of every single journal, several researchers do not see the paper until four months later, when one of their graduate students mentions it. (In Jane 2.0’s universe, content recommendation software alerts researchers to interesting/relevant papers.)

Follow-up research begins roughly one year after Jane 1.0 initially published her research. This research is not published for an additional year. So, two years after Jane originally published her research, there are only 22 follow-up papers.

What We Should Expect from a Web-Enabled Publishing Environment

The pace of research in the current universe stalls for several reasons. Some of these are emphasized in the above stories. Others are not.

  1. There is no public forum to discuss a paper. Sometimes, there are discussions at conferences; however, it is difficult to invite other people into these discussions. It is difficult to make these discussions public for others to observe.
  2. For-profit journals used to serve a purpose of printing ink on paper. This used to speed up research. They no longer add this value. Now, they slow down research by putting up pay walls. Editors, referees, and researchers create the value and donate their time to the community. They are often paid with taxpayers’ dollars. Their efforts should not be hidden behind a pay wall.
  3. The current system recommends content by sorting research into journals. However, our research interests are much more varied than the number of journals allow. A new, unified system would enable “content recommendation,” like Netflix or Amazon, to identify research of interest.
  4. The current system is bulky. Papers are often published in at least four locations. There is no systematic way for someone to stay up to date on the current research. The new system would provide the opportunity for a high-quality user interface for researchers to read about developments when they first happen.
  5. The current paradigm accepts research that is roughly 10–50 pages. In most journals, the effective page restrictions are much tighter. However, suppose a graduate student reads a paper and runs a follow-up simulation that gives some small insight into a certain method. Unless this graduate student can expand this idea into a full paper, no one sees the simulation. The new paradigm would allow for users to post follow-up simulations with some code for others to try. In addition to expediting follow-up research, it could also lead to rapid identification of coding errors. Additionally, this site could host academic blogs, which have proved to be a popular medium for several researchers. On the upper page limit, any grand research that exceeds 50 pages often must be split into several papers or published in a monograph. Web-enabled publishing would not have such restrictions.
  6. This system also could allow conferences to have a central web page. Conference speakers could provide links to papers/slides/etc.

The Path Forward

The ASA has convened a panel to discuss the future of academic publishing. There are several researchers with deep concerns as to how a new system might disrupt the current system. However, we do not get to choose if we follow a new path; a step forward is inevitable. We might get to decide what the new path looks like. Some paths are suboptimal. Some paths will be designed for other disciplines. We need to design a system that works for our community.

At the same time, a social network without any people is worthless; the value dramatically increases as more people buy in. Making a separate, second most–popular social network is not worthwhile (see MySpace). If we make the second or third site, we might be forced to follow the path others blaze. For these reasons, we need to build a network that others will want to join, and we need to move fast.

Having a network supported by a large organization such as the ASA or IMS would give credence to a site. This is important. However, for a site to spark, it needs much more. It needs to be easy. It needs to be effective. It needs to add value to people’s lives.

The first step forward should be fast and easy. One option is to build a site on top of arXiv. At first, the site does not need user profiles. It merely links to articles (hosted elsewhere). This puts all the research in one spot. As a second step, there should be a comment thread associated with each linked article. Additionally, there should be a “republish” button that “pushes” the good research to the top of the front page. These are extremely valuable steps forward.

Later, a more formal system can give more authenticity by giving users profiles and letting them connect with fellow researchers. Whether this system should be built in concert with arXiv is not clear. A huge upside is that it would lead to immediate buy-in from a wide range of researchers. One potential downside is that if arXiv owns the system, then the statistics community loses control.

Our current system of publishing was designed for a world with different constraints. We have begun to realize a new system, through academic blogs and homepages, arXiv, and online journals. However, these sources lack unity and are therefore more cumbersome than the limitations of current technology. A central electronic clearinghouse with links, alerts, and discussion boards would reduce some of the burden of finding research and provide a unified place to discuss research. Ultimately, this would lead to an efficient, expedited, and accessible interface for publishing and discussing research that would actively propel lines of inquiry forward. Academia is characterized by creative thought, innovation, and timely applications that benefit society, and the irony of being hamstrung by arcane modalities is striking. It is time to move forward.

Karl Rohe, Assistant Professor
University of Wisconsin-Madison Statistics Department

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)

Comments are closed.