WPbanner5.jpg (10176 bytes)
No. WP- 01-03

The Remarkable Transformation of E-Biomed into PubMed Central[PDF]

Rob Kling*, Joanna Fortuna and Adam King

*Center for Social Informatics
Indiana University
Bloomington, IN 47405


Version 4.41B-2
10/3/2001 [4/4/2002]

This is a working draft: we would appreciate your comments


In 1999, NIH Director Harold Varmus proposed a national biomedical literature server called “E-Biomed.”  E-Biomed reflected the visions of scholarly electronic publishing advocates: it would be fully searchable, free to access, and contain full text versions of both pre-print and post-publication biomedical research articles.  However, in less than a year, the E-Biomed proposal was radically transformed, eliminating the pre-print section, instituting delays between article publication and posting to the archive, and changing the name to “PubMed Central.” This article examines the remarkable transformation of the E-Biomed proposal to PubMed Central by analyzing posts to an online E-Biomed discussion forum created by the U.S. governments’ NIH, and other forums where E-Biomed deliberations took place. We find that the transformation of the E-Biomed proposal into PubMed Central was the result of highly visible and highly influential statements made by publishers and scientific societies against the proposal. We conclude that: 1) scientific societies and the individual scientists they represent do not always have identical interests, especially in regards to scientific e-publishing; 2) stakeholder politics and personal interests reign supreme in policy debates, even in a supposedly status-free online discussion forum; 3) multiple communication forums must be considered in examinations of public policy deliberations.


The scientific publishing system is undergoing many small changes and several major transformations by relying increasingly upon digital media and the Internet. One major development was the creation of an electronic repository of technical reports (called e-prints) for some physics and mathematics fields, operated by the U.S. Department of Defenses’ Los Alamos National Laboratory (LANL). This repository, ArXiv.org, does not charge authors or readers for using it. Authors decide when to post their articles, and most of the posted articles are not peer-reviewed before they are posted (although the vast majority are eventually published in peer-reviewed journals or peer-reviewed conference proceedings). ArXiv.org was initiated in 1991, by April 5, 1998,  72,497 articles were posted on ArXiv.org [1].   The ArXiv.org model has been so successful that some observers suggest that scientists in every field should immediately adopt it (Harnad, 1999)

One scientific field currently venturing into online scholarly publishing is biomedicine. As early as 1995 (Laporte, et al.), observers forecast the imminent “death of biomedical journals” and their replacement with electronically accessible biomedical information servers. Ongoing problems made this “electronic revolution” in biomedical publishing seem inevitable: large lag times between the submission and eventual publication of articles in biomedical journals frustrated researchers. The already high costs of printing, purchasing, and archiving traditional paper biomedical journals were rising well above the rate of inflation. Finally, many researchers felt that the limited range of publications accepted by major biomedical journals was stifling innovative research. Traditional biomedical journals have been described as a “19th Century Compromise” or a “Faustian bargain,” violating the primary goals of biomedical research rather than supporting them (Markovitz, 1998).

In 1999, against this backdrop of concern about the limitations of paper publishing and excitement about electronic alternatives, U.S. governments’ National Institutes of Health (NIH) Director Harold Varmus made a policy proposal that would directly impact the future of biomedical research publishing. The proposal, called "E-Biomed,"   proposed the creation of a central electronic archive for all biomedical research articles. This Web-based, NIH-run archive, would have two sections: a pre-print server, where authors could submit biomedical research papers that had not yet gone through traditional publication channels, and an archive of published peer-reviewed articles. No fees or credentials would be required for access to the E-Biomed archive.

On May 5, 1999, Harold Varmus posted a call for comments regarding the E-Biomed proposal, on the NIH web site.  The scientists who responded to this forum appeared to support the proposal by a 2 to 1 margin.

However, on August 30, NIH officials announced that E-Biomed would be supplanted by a revised proposal called "PubMed Central." The revised proposal differed from the original E-Biomed in more than just name: in particular, PubMed Central would contain no pre-print server that would have enabled researchers to post papers without going through traditional peer-review and editorial processes. Unlike the original E-Biomed proposal, scientific societies and commercial publishers would have central roles in the control and dissemination of content in PubMed Central. PubMed Central would still be free to readers, but publishers would control both the content posted in the archive and the time the articles would be posted. The changes between the E-Biomed proposal and the PubMed Central version ran counter to the “inevitable” outcomes predicted by many electronic publishing enthusiasts. This article examines this surprising transformation in the NIH’s electronic publishing plan.

In order to understand the process by which the E-Biomed proposal was transformed into PubMed Central, we address several key questions:

· Why was E-Biomed, an example of the electronic self-publishing vision, advanced by prominent electronic publishing enthusiasts such     as Stevan Harnad (1992) and Paul Ginsparg (1996), reworked into a form preventing self-publishing, preserving peer review, and
   allowing arbitrary posting delays?

· Why didn’t the final proposal reflect design features that were most supported by the participants in the online forum?

· Who were the most influential players in this transformation process?

We address these questions by tracing the process of transformation from the original May 5 E-Biomed proposal through the August 30 announcement of PubMed Central. While we focus on the online response forum, we will also consider other communication channels and forums used by participants in the E-Biomed dialogue. We conclude by discussing what can be learned from the progression, debate, and transformation of E-Biomed to PubMed Central.

The Contested Transformation of E-Biomed to PubMed Central

We will begin examining this transformation by outlining the chronology of events in which the E-Biomed proposal was envisioned, published, discussed, changed, and ultimately implemented as PubMed Central.

The idea of a pre-print server for biomedical research did not originate with Harold Varmus, the Director of NIH. Nearly a year before Varmus proposed the E-Biomed server, genetics researcher Pat Brown of Stanford University, and David Lipman, director of the National Center for Biotechnology Information (NCBI) at NLM/NIH , discussed the potential of a biological research Web server with their colleagues. During a December 1998 meeting at Cold Spring Harbor Laboratory (Long Island, NY), electronic publishing enthusiast Paul Ginsparg persuaded Brown and Lipman to follow the ArXiv.org model in developing their ideas. At that point, the ArXiv.org pre-print server became the primary conceptual model for the E-Biomed proposal, rooting E-Biomed in the notion that pre-print server models from the field of physics could be exported to other disciplines with a few minor adjustments.

By January 1999, Brown and Lipman completed the rough draft of a proposal for a biomedical pre-print server. Brown sent a letter signed by thirty biomedical scientists to several publishers, challenging their existing publication policies. Among the policies being challenged was the “Ingelfinger Rule” (Ingelfinger, 1969) of the New England Journal of Medicine (NEJM), which defined “prior publication” so broadly that articles posted on pre-print servers were prohibited from being peer-reviewed and published by the journal. Lipman then presented the biomedical pre-print server plan to the Howard Hughes Medical Institute (HHMI) in an attempt to persuade the institute to sponsor the project. HHMI declined.

In February 1999, Brown and Lipman presented their proposal to NIH Director Harold Varmus. Varmus and his staff then restructured the proposal, and on March 4, Varmus attended an NIH budget hearing with the U.S. Congress’ House Appropriations subcommittee. At that meeting, Varmus made his first public mention of his support for the proposal of the biomedical research server. Around the same time, the scientific press started to report on Varmus’ plan to sponsor the biomedical pre-print server project. On April 25, the editorial board of the Proceedings of the National Academy of Sciences (PNAS), the high-prestige journal of the National Academy of Sciences (NAS), received an NIH briefing about the proposal. However, the PNAS board did not offer support for the project at that time. The National Academy of Sciences represents a membership of close to 2,000 scientists. NAS is a prestigious, private, non-profit society, responsible for advising the federal government on scientific matters; NAS also publishes PNAS, a prestigious multidisciplinary journal with a readership base of more than 25,000. Thus, persuading PNAS Editors to support the proposal would have been a major political success for NIH’s E-Biomed proposal.

To stimulate debate on the NIH proposal, on May 5 Varmus posted a draft of the proposal, now called “E-Biomed”, on the NIH Web site. He requested feedback from readers in the form of messages e-mailed to Varmus’ office to be re-posted in a public online forum.  The E-Biomed proposal called for a biomedical research server composed of two sections: one section would feature research that had been reviewed by new or existing editorial boards; the other section would be a repository of non-peer-reviewed (though “screened”) pre-prints posted by individual authors. Authors would retain the copyright to any work posted on the pre-print section of E-Biomed. This arrangement would differ from the traditional publication system, in which the publishers of articles hold copyright rather than authors. The E-Biomed project would be administered by a Governing Board, which would include representatives from “all of the parties concerned – the scientific community (readers and authors), editors, computer scientists, and funding agencies.”  Notably absent from this list of potential stakeholders were scientific societies and commercial publishers.

The messages posted on the NIH site include comments from approximately 225 individuals (mostly academics and MDs) and nearly a dozen statements by officers of scientific societies. The scientific press characterized the early responses as mixed. A number of biomedical associations and scientific societies sent official responses to Varmus, who posted many of them on the E-Biomed discussion archive.  These association responses tended to be critical of the E-Biomed proposal. Comments from those identifying themselves as journal editors, on the other hand, appeared to be divided into two camps – some in favor of the proposal (e.g., The Lancet), others opposed (e.g., NEJM, Journal of the American Medical Association [JAMA]).  Finally, individual readers and authors tended to be strongly in favor of the proposal.

On June 20, an addendum to the E-Biomed proposal was posted on the NIH Web site. This addendum attempted to clarify the original proposal and respond to the questions most frequently raised in the online response forum. The addendum also included a number of significant changes to the proposal. Among these changes were: (1) copyright retention by authors or transfer to publishers upon publication was no longer standardized, but would be decided by individual editorial boards; (2) the title of “Governing Board” for the E-Biomed administrative body was changed to “Advisory Board;” and (3) the majority of the papers on E-Biomed would be submitted through ordinary publication channels and editorial boards rather than being posted directly to the pre-print repository.

In late July or early August, the scientific press (e.g., Science) reported that the name of the NIH proposal had been changed from “E-Biomed” to “E-Biosci.” This name change was due in part to a trademark issue: publisher Mary Ann Liebert had previously reserved the domain name “E-Biomed” for a different project. It also signaled the broadening of the proposal’s scope from biomedical research to include all life sciences. The name of the proposal on the NIH site was not changed.  The scientific press discussed many changes that might be included in the next proposal revision, including: (1) the posting of articles by scientific groups would replace author self-posting; (2) no attempt would be made to influence the standing copyright policies of biomedical journals; and (3) long publisher delays between actual article publication and free archival access to that article would be allowed. Several important scientific societies became more supportive of the proposal as these changes were being discussed.

On August 30, the NIH published another revised version of the proposal.  The name of the archive was changed again, to “PubMed Central.” This name change associated the archive with the successful abstract database (PubMed) already offered by the National Center for Biotechnology Information (NCBI).   Like the earlier versions of the proposal, the PubMed Central proposal did not discuss the cost of the archive, although in his public statements Varmus began to estimate the cost at about $1-3 million a year. The source of funding for the project was not yet formalized, although Varmus suggested that funding would come from page charges paid by authors. PubMed Central would primarily be a peer-reviewed article archive. Varmus and Lipman stressed that authors could still post non-peer-reviewed materials,  but authors would no longer be able to post directly to the server. Only participating publishers, societies, and other approved groups would have the ability to post to PubMed Central. Moreover, the posting of published articles to the server would not be immediate; publishers could delay posting articles for up to a year after their initial publication. (See Table 1 for a breakdown of the differences between the proposals.)

Table 1
Major Shifts from E-Biomed to PubMed Central

May 5 E-Biomed
August 30 PubMed Central
Who posts
Author self-posting   Articles are to be provided by publishers and societies
When posted
 “…even those reports reviewed and listed by editorial boards would be available earlier to the reading public because they would all be posted at the time of acceptance, eliminating the lag time now ascribable to publication on paper.” “Although early deposition offers the greatest benefit to the scientific community, we recognize the concerns of publishers about financial consequences of rapid submission and will welcome content submitted at any time.”
Use of Peer Review
 Some articles are non-peer-reviewed pre-prints, others are peer-reviewed re-prints  No unreviewed materials (unless society or publisher decides to put some up); societies and publishers retain control of peer-review
 “Copyright to reports posted in E-Biomed would be retained by the authors”  “Copyright will reside with the submitting groups”
Board in charge of project management
 “a mechanism for governance (the E-Biomed Governing Board) that involves all of the parties concerned---the scientific community (readers and authors), editors, computer specialists, and funding agencies.”  “an international advisory committee will be constituted by learned societies interested in fostering the development of PubMed Central and the collaborating international repositories.”

Support for the PubMed Central proposal was broader than that for the original E-Biomed proposal. NIH even garnered promises of cooperation from a few biomedical journals. The American Society for Cell Biology agreed to  post, on a trial basis, two months after publication, articles from its journal Molecular Biology of the Cell. The high-prestige European Molecular Biology Organization (EMBO) Journal also agreed to post articles on PubMed Central, albeit six months after publication.

In mid-October PNAS also announced its willingness to participate in PubMed Central.   PNAS participation was a major coup for the proposal, but it was contingent upon several conditions.  PNAS required that only peer-review materials appear on the server: “Participation in PubMed Central is contingent upon its not including “reports that have been screened but not formally peer-reviewed””   This requirement imposed a major redefinition of PubMed Central.  Some observers had expressed hope that one part of the site would host non-peer-reviewed papers, and that over time PubMed Central might evolve into a service resembling ArXiv.org.  These hopes were dashed with the PNAS’ demand that  all materials on the site appearing on PubMed Central must be peer-reviewed.  Also, PNAS articles would appear following a four-week delay, and author charges could not be levied in order to fund the PubMed Central archive.

PubMed Central has been in operation since February 2000.  As of September 2001, the archive (http://PubMedCentral.nih.gov/) includes PDF files, abstracts, full texts, and sometimes “Supplemental Data” and/or “Video Material” for nine journals: Arthritis Research, bmj.com (British Medical Journal), Breast Cancer Research, Bulletin of Medical Library Association, Critical Care, Genome Biology, Molecular Biology of the Cell, Proceedings of the National Academy of Sciences of the United states of America (PNAS), and BioMed Central (BMC), the last containing 44 separate titles (see Table 2).  The BMC journal collection, which deposits immediately to PubMed Central, is expected to grow rapidly.

Table 2
Available Journal Titles on PubMed Central

Available Volumes
  Publication Schedule
Arthritis Research
 “Biological research relevant to arthritis and related autoimmune diseases, including disorders of bone and cartilage.” Vol. 1(1), 1999; Vol. 2(1-6), 2000; Vol. 3(1-4), 2001  Bi-monthly
BMC Titles (BioMed Central)
 44 separate titles available—“research across all areas of biology and medicine.”  Varies by title, but all either 1st and/or 2nd volume, years 2000 and/or 2001.  Annually
British Medical Journal (bmj.com)
 Survey of general medical research, including sections on “Primary Care,” “General Practice,” and “Information in Practice.”  Vol. 316(7131-7149), 1998 through Vol. 323(7303-311), 2001(8 vols., 179 issues)  Weekly
Breast Cancer Research
 “All areas of research into breast cancer.” Vol. 1(1), 1999; Vol. 2(1-6), 2000; Vol. 3 (1-4), 2001  Bi-monthly
Bulletin of the Medical Library Association
 “Aims to advance the practice and research knowledgebase of health sciences librarianship.”  Vol. 88(1-4), 2000;Vol. 89(1-3), 2001  Tri-monthly
Critical Care
“Critical care, intensive care, and emergency medicine.”   Vol. 1(1-3), 1997; Vol. 2(1-2), 1998; Vol. 3(1-6), 1999; Vol. 4(1-6), 2000; Vol. 5(1-4), 2001  Bi-monthly
Genome Biology
“Most important findings in post-genomic biology as they occur.” Vol. 1(1-6), 2000; Vol. 2(1-6), 2001    Bi-monthly
Molecular Biology of the Cell
 “Provide for the exchange of scientific knowledge in the area of cell biology… through the scholarly dissemination of research….”  Vol. 8(10-12), 1997; Vol. 9(1-12), 1998; Vol. 10(1-12), 1999; Vol. 11(1-12), 2000; Vol. 12(1-6), 2001  Monthly
Proceedings of the National Academy of Sciences of the United States of America (PNAS) 
“Physical Sciences,” “Biological Sciences,” and “Social Sciences.” Vol. 90(1-24), 1993; Vol. 91(1-24), 1994; Vol. 92(1-24), 1995; Vol. 93(1-24), 1996; Vol. 94(1-24), 1997; Vol. 95(1-24), 1998; Vol. 96(1-24), 1999; Vol. 97(1-24), 2000; Vol. 98(1-15), 2001  Bi-weekly

Of the current journals, all but one are available both in print and electronically: the BioMed Central titles are available only online.  Moreover, PubMed Central has announced the forthcoming availability of several more journals: Biochemical Journal, Canadian Medical Association Journal, Current Controlled Trials in Cardiovascular Medicine, EMBO Journal, Journal of the American Medical Informatics Association (JAMIA), Journal of Medical Entomology, Nucleic Acids Research, The Plant Cell, Plant Physiology, and Respiratory Research.

Although the bulk of the archive’s publications are peer-reviewed primary research articles, PubMed Central welcomes and includes other relevant works from participating journals, such as editorials, review articles, symposiums, and case studies.  Submissions are required to be in the English language and are accepted from any life sciences journal (including plant sciences) which meets one of the following criteria:

While the majority of the currently available journals have no lag time between original publication and release in PubMed Central, Molecular Biology of the Cell has a delay of two months and PNAS of four weeks.

The E-Biomed controversy seems settled, yet the NIH biomedical research server has continued to come under fire. On January 19, 2000, the Endocrine Society and the American Society of Hematology circulated a draft of a letter to Congressman Ted Porter that expressed concern over PubMed Central’s threat to their membership and revenues. This draft was later signed by several additional scientific societies and publishing organizations.  The future of PubMed Central remains contested.

The E-Biomed Online Forum

The E-Biomed online forum played a central part in the unfolding E-Biomed controversy. We examined the messages in the online forum to assess the variety of positions taken toward the proposal by various biomedical research communities.

The electronic comments forum Varmus created on the NIH site includes 269 e-mail messages sent to Varmus between May 5 and August 17. Most of the comments that Varmus posted to the forum were responses to the original May 5 E-Biomed proposal and the June 20 addendum. Most striking about these responses is the intensity of the opinions and emotions displayed both in favor and in opposition to the proposal.

Opinions ranged from complete acceptance and endorsement-such as:

“I cannot wait for E-Biomed to get started.”
“E-Biomed is exactly what I have been longing for.”
“I support this effort 100%.”
“I applaud your efforts.”
“I fully support your proposal.”
“Excellent idea that's long overdue!”
“I am quite enthusiastic about your E-Biomed proposal.”
to strong opposition:
“I am very concerned …”
“The E-Biomed proposal is unwise and ill-advised.”
“I urge that the E-Biomed concept NOT be implemented.”
“I find myself strongly opposed.”
“This proposal is so horrible that …”
“Among the very worst ideas I have ever heard.”

A typical message included the author’s personal identity and institutional affiliation, and many messages also included the authors’ status as an MD or Ph.D., their membership in scholarly associations, their roles on editorial boards, their years of experience as reviewers, or the number of their subscriptions to journals. Respondents seemed to believe that identifying their stakes was essential to communicating their point of view. Most respondents thoroughly identified themselves, establishing both their interests and their legitimacy as commentators.

Respondents sometimes identified themselves as having multiple roles (e.g., as academics and medical practitioners, as authors and editors) which made coding difficult.  However, we found that of the 224 respondents in the E-Biomed forum, at least 90 were affiliated with academic institutions, more than 20 were medical practitioners without an academic affiliation, and almost 30 were editors of scientific journals (see Table 3).

Table 3
Online Forum Messages by Position and Institutional Affiliation

  # Respondents  # Supportive 
 # Opposed
~ 125 
 ~ 60 
Academics (Authors/Readers) 
90 + 
~ 55 
~ 25 
20 + 
~ 15 
~ 5 
Government Employees
10 + 
Other (Commercial, Consultants, Lobbies, Trusts, etc.)
 ~ 20 
~ 10 
Unidentified Affiliation or Stakes 
~ 20 
+ 7 
25 + 
~ 10 
~ 10 
Society Officials 
Maybe 1 
8 – 9 
Commercial Publisher 

Messages supporting the original E-Biomed proposal outnumbered those against the proposal by about 2 to 1.  Many comments favoring the proposal mentioned the “self-interestedness” of those opposed to it. They often dismissed the concerns presented by E-Biomed opponents, claiming that those concerns were merely a facade for their real concern about the potential loss of revenues from journal publications. These critics were not similarly skeptical about the motives behind responses given in support of their position. Many comments opposing the proposal also made a self-interested argument about their rivals, claiming that supporters of the proposal were only interested in free access to materials and not in supporting the structures that ensure the quality of those materials.

Although two polarized stances towards the E-Biomed proposal quickly developed, the positions taken were not absolute. For instance, even respondents opposing the proposal on other grounds often acknowledged the possible advantages of a single search engine for all biomedical literature. Society officials who were highly critical of the E-Biomed proposal often supported scholarly electronic publishing in general, calling it the “wave of the future” and “inevitable.” Those same officials often stressed that they were already implementing electronic publishing initiatives, and the proposed E-Biomed archive would needlessly duplicate these efforts.

In addition to stating a position for or against the E-Biomed proposal, many messages in the online forum also mentioned broader issues surrounding the electronic publishing of biomedical research. At least half of the  problems most frequently identified in the online forum related to the issue of peer review:

The fear that non-peer-reviewed literature posted on an NIH Web site would be legitimized, and conclusions accepted uncritically by the public and practitioners, caused great opposition to the original proposal. Some were concerned that research that turned out to be unreliable could have been put into application, causing physical harm.  In this, the biomedical site differed from ArXiv.org, whose articles have less direct significance for people’s health or well-being.

Other problems in the online forum frequently cited included:

Some of the issues identified were specific to the E-Biomed proposal, such as: A useful way of thinking about the numerous and contradictory stances seen in the online forum is to group the respondents according to their identifications.  Looking through the electronic comments, we found that the interests displayed in a message related closely to the way in which the writers identified themselves in the message’s opening. In general, respondents identifying themselves in certain roles tended to focus on particular concerns:
The differences between the May 5 E-Biomed proposal and the August 30 PubMed Central proposal (see Table 1) can be described as a shift from one that emphasized authors’ and readers’ interests to one that favored the interests of societies’ and publishers’.  Though both NIH proposals were framed in terms of the interests of the “scientific community,” the composition of the “scientific community” is defined differently in the two proposals. The May 5 E-Biomed proposal defines the scientific community as authors and readers, excluding scientific societies:
a mechanism for governance (the E-Biomed Governing Board) that involves all of the parties concerned---the scientific community (readers and authors), editors, computer specialists, and funding agencies. [emphasis added]
This view of the scientific community is typical of electronic publishing enthusiasts in the scholarly communication literature (Brody, 1996; Odlyzko, 1996), and seemed to be accepted uncritically by many supporters of the original proposal. Scientific societies, on the other hand, were careful to stress their role in the scientific community, and tended to define the scientific community in terms of journal readership and society membership. For example, the officers of the American Society for Pharmacology and Experimental Therapeutics stated:
The scientific community is diverse and should not necessarily speak with a single voice. Individual scientific society disciplines and their respective journals should be celebrated and encouraged to succeed. (ASPET 1999)
Thus, the perception of whether the E-Biomed archive would be a boon or a bane for the scientific community depended to a great degree on just how the term “scientific community” was being defined. Supporters and  opponents of the proposal  tended to characterize the “scientific community” so as to bolster their particular positions.

The Limitations of the E-Biomed Electronic Forum

Because we speculate that Harold Varmus was willing to take seriously the comments he posted to the E-Biomed forum, we would expect that the comments heavily influenced the subsequent shaping of the E-Biomed proposal. However, though only nine of the 269 messages in the E-Biomed forum were written by scientific society officials, subsequent versions of the proposal were much more sensitive to the concerns of those societies than to the concerns of the much more numerous individual respondents. How did such a small minority of the respondents in the online forum have such a disproportionate impact on the subsequent revisions of the proposal?

There are several explanations for this outcome. First, the official statements signed by one or several top officers of a scientific society were presented not as the personal opinions of the signatories but as the official position of the Society’s membership. As such, these official statements implied a unified viewpoint held by every member of the society.   For example, most of the scientific societies that responded online to the E-Biomed proposal belong to the Federation of American Societies for Experimental Biology (FASEB), a coalition of 20 biomedical societies representing more than 66,000 individuals.  Electronic comments by individuals on the NIH site numbered in the hundreds while societies sent fewer than a dozen comments, but each society response seemed to represent the views of thousands or tens of thousands of society members. Given the huge memberships of the societies, even statements expressing the interests of a fraction of the society membership would still represent more “votes” than all of the individual responses in the online E-Biomed forum combined.

Thousands or tens of thousands of individual responses in support of the E-Biomed proposal might have balanced the powerful influence of responses from society officials. However, individual scientists did not respond in the huge numbers necessary to balance the official society responses.

Second, societies could and did engage in a variety of internal activities to increase member consensus on the E-Biomed issue. Societies responding to the E-Biomed proposal were able to gather information from their membership and to influence members’ positions via newsletters, questionnaires, meetings, and society Web sites. These internal mechanisms allowed the unification of society voices and the minimizing of contrary responses from the society membership. Harold Varmus noted this phenomenon, saying:

I have heard less from the scientific community than I am used to, because people in the trenches are used to having their opinions bubble up through their leaders, and [in this instance] some of their leaders have been a little resistant (Science, 1999)
Third, the NIH had political reasons to give greater weight to the challenges issued by scientific societies and journals. FASEB and its member societies are influential lobbyists in the U.S. Congress, particularly in regards to the NIH budget. During the E-Biomed debate, society officials reminded Varmus of this political relationship:
The revenues provided to the Society through the publications program provide support for a number of societal activities, including its scientific meetings. These revenues also help support the Society’s public affairs efforts, which encourages Congress to increase research funding for NIH and other Federal agencies. (American Physiological Society, 1999)
Given the tight relationship between biomedical scientific societies and the NIH budget process, the NIH may have been more attentive to statements about the E-Biomed proposal made by those societies.

Fourth, aspects of the E-Biomed proposal required the cooperation of existing print journals. To make the pre-print archive successful, many biomedical journals would have to alter submission policies excluding articles that have appeared on a pre-print server. Posting re-prints of published articles to the archive would require publisher approval because publishers currently hold copyrights on those materials. The E-Biomed server also required some form of peer-review, a function that would be best accomplished by existing journal editorial boards. The structure of E-Biomed depended critically upon society support much more than it did on the support of individual authors and readers.

Fifth, official society statements in the online forum were very conspicuous because they differed in style and character from comments written by individual respondents. Society representatives generally responded with well-organized, carefully prepared statements, while the responses of individual commentators tended to have more of a spontaneous, unfinished feel. Strictly based on the quality of their presentation, the polished official society statements may have been more influential than the rougher-seeming individual comments.

In summary, the official statements made by a small number of publisher and society officials were louder than the voices of many more individuals. Societies issuing official statements were more influential for the following reasons: they seemed to speak for many more people; they were able to exercise internal control over their membership; their cooperation was critical to E-Biomed’s success; they had political clout based on their role in the Congressional funding process; and  their statements were more organized and polished. To counter these highly effective messages, individual readers and authors needed to speak in much greater numbers, more articulately, and with much greater consensus than they did.

The Interplay of Multiple Forums

The E-Biomed forum on the NIH Web site was the major focus of media attention on the controversy. This focus is reasonable, since the online forum was by far the most publicized and most publicly accessible location in the E-Biomed debate. However, the online forum was neither the only, nor the most important arena for debate and discussion of the E-Biomed proposal. Other forums where debate and discussion took place included official and unofficial face-to-face meetings with Varmus; journal and newspaper reports and editorials; personal correspondence to Varmus and his colleagues; informal conversations between various key players; staff meetings at the NIH; and society newsletters and questionnaires.

Importantly, most of these forums were either inaccessible or asymmetrically accessible to most readers and authors. Face-to-face meetings with Varmus, for instance, were accessible to society representatives (see below) but inaccessible to individual scientists. Furthermore, anyone with a subscription or library access to a journal could read an editorial, but only an editor could write one. To demonstrate the complicated accessibility of important discussion forums, we will discuss the face-to-face meetings between Harold Varmus and society and publisher officials.

Face-to-face meetings with Varmus:

Even before the NIH posted the E-Biomed proposal, Varmus and Lipman made a point of meeting with societies and especially with their publication committees. These meetings were not open to the public, and the press did not cover them in any detail. We know of several face-to-face meetings between Varmus and society representatives, meetings that tended to be antagonistic towards the original proposal.

On April 25, the editorial board of PNAS was briefed about E-Biomed by the NIH, in the hope of gaining support for the project from this prestigious journal.  However, the presentation at the National Academy of Sciences is reported to have produced a “less than enthusiastic” response (Science, 284 (5415), April 30). On June 2, Varmus and Lipman met with the Publications Committee of FASEB.  The question and answer session following Varmus’ presentation was described in a FASEB newsletter,  which stressed that representatives raised the same questions and concerns that had already been presented in societies’ official statements in May. On June 24, Varmus presented his proposal again and listened to concerns at a summit of electronic publishers held at the National Academy of Sciences.  On June 30 Varmus met with science writers, some commercial, and some society publishers at the National Library of Medicine.  A report in Government & Policy stated that “an adversarial tone dominated discussions between Varmus and his audience at both meetings, with Varmus expressing surprise at the vehement reaction coming from some publishers.”  Some of the speeches at the science writers meeting, as reprinted in HMS Beagle, stressed the experience and value added by existing publishers, and held up cost, journal prestige, and quality control issues as the key problems with E-Biomed.  In short, society representatives, publisher representatives, and editors of major journals had privileged access to Varmus through these face-to-face meetings, access that gave them the opportunity to stress their concerns and to reassert their central role in the decision-making process regarding the E-Biomed proposal.

To comprehend the transformation of E-Biomed into PubMed Central, we must consider not only the existence of multiple forums for debate about the E-Biomed proposal, but the interplay between those forums. Communication forums have permeable boundaries that straddle complex interrelationships, and are not singular, insulated sites for discussion and debate.
Our assessment of the impacts of the various discussion forums includes the interplay of these forums with one another. The outcome or existence of a discussion of E-Biomed in one forum frequently had an impact on the discussion in another forum.  Some interchanges influenced others by setting the stage for future interaction in other forums. Societies’ official statements or letters to NIH often mentioned past or future face-to-face meetings with Varmus. The ASBMB response said, “Representatives of the ASBMB would welcome the opportunity to meet with you [Varmus] for further discussion.” The APS response stated, “Thank you for arranging to meet with us and for providing the American Physiological Society with an opportunity to comment on your proposal to create E-Biomed.” Participants in one forum (submitting e-mail commentaries) were able to directly shape the creation or development of another forum, such as arranging face-to-face meetings. Forums influenced each other in two other important ways:

Reinforcing views expressed in other forums:

Comments in the online forum sometimes referred to meetings held by the society membership. In addition, society meetings often included discussions of letters or online responses that had previously been made in the name of the society (see the account of the August 1999 FASEB meeting).  Society meetings mentioned the face-to-face meetings between Varmus and society officials. Another excellent example would be the Arnold Relman (1999) editorial about E-Biomed in the NEJM, which was often cited in comments on the online forum. In this fashion, a comment made in one forum was often quickly communicated to another forum, breaking down the boundaries between forums.

Interpreting the debate in the other forums:

Participants in some key communication forums reported, analyzed, or interpreted the developments of E-Biomed that were discussed in other forums. For example, scientific magazines (especially Science and Nature) reported extensively on Varmus’ E-Biomed talks, his meetings with society and publisher representatives, and the discussions that were taking place on NIH’s E-Biomed online forum. These journalistic accounts provided yet another forum for interpretation and discussion of the E-Biomed proposal. Many biomedical society Web sites also posted editorial or analytical commentary on the E-Biomed debates (see The American Society of Plant Physiologists Web page for an example).

Anticipating the Controversies of E-Biomed

In this article, we note that a few highly influential voices, those of scientific society and publisher representatives, dominated the policy debate regarding E-Biomed. We also note that Harold Varmus, who set up the online E-Biomed forum to gather responses from a broadly defined biomedical community, was surprised that so few individual scientists participated. The forum did not take on the appearance of a pluralistic debate, as forum responses were mostly one-shot official statements about the proposal.

The notion that individual scientists would have the same views as scientific society representatives may seem  naïve. It may seem equally shortsighted to think that individual scientists would have had as much influence on the E-Biomed proposal as scientific societies and publishers. From this point of view, it would have been unrealistic to predict that the online forum would be pluralistic and interactive in nature rather than officious and formal. Whatever the creators of the E-Biomed proposal may have hoped, this line of reasoning goes, they were idealistic if they thought things would develop in any way other than they did.

This position, so clear in hindsight, overlooks the state that the scholarly e-publishing debate was in at the time. If the creators of the E-Biomed forum had examined extant discussions of online scientific e-publishing, they would have found something very different in form and content than the later E-Biomed online discussion forum. To highlight the differences between online e-publishing discussions that proceeded E-Biomed and the E-Biomed online forum, we will compare and contrast the E-Biomed online forum with the precursor most similar to it in structure and topic, the online American Scientist e-publishing discussion.

American Scientist, a research journal published by Sigma Xi, has maintained an online forum since 1998 dedicated to the topic of scientific electronic publishing.    The forum was prompted by an article that proposed that scientific societies should publish their journals in electronic form in a way that is free to readers (Walker, 1998). In response, the electronic forum, moderated by Stevan Harnad, was set up to discuss the theme: "Freeing the Refereed Journal Literature Through Online Self-Archiving".

Table 3
Activity Levels of Electronic Forums Discussing Electronic Publishing

 American Scientist 1998
 American Scientist 1999
 E-Biomed comments 1999
Time period
 Aug 25-Nov 21 
 Feb 10-Dec 23 
 May 5 – Aug 17 
# of messages 
# of participants
# messages by Dr. Publicus
# of participants with more than 2 messages (including Dr. Publicus)

We examined the American Scientist on-line discussion between its inception in 1998 and May of 1999.  In doing so, we hoped to discover whether someone who had read this major and timely discussion of scholarly electronic publishing could have anticipated the powerful and surprising stance the scientific societies would later take against E-Biomed.  During the period that we analyzed, approximately 350 messages on the American Scientist forum (see Table 3). discussed several dozen topics, including the relative costs of paper and e-media, the value and limitations of peer review, citations to e-journals, economic models where the author pays, and so on.

These discussions consisted exclusively of individual opinions, with no society representatives making official statements. Unlike the E-Biomed commentators, who tended to identify themselves as being relevant stakeholders (officials in societies, researchers in the affected disciplines, editors of journals in the field, medical practitioners, etc.), participants in the American Scientist forum typically identified themselves solely as interested observers.
Accompanying this difference in self-identification was a shift in focus, with the American Scientist commentators being much more interested in abstract discussions and E-Biomed respondents focusing on how specific elements of the proposal would or could affect their particular concerns.  The American Scientist forum stressed certain issues (copyright, intellectual property) that received much less interest in the later E-Biomed debate, and downplayed many issues (costs, budget, logistics, funding, implementation, and maintenance) that received much greater interest in the E-Biomed discussion.

The possibility that scientific societies might have a problem with archives organized like arXiv.org or e-Biomed, and thus would oppose these kinds of archives, did not receive any significant attention in the American Scientist debate. Like the electronic publication literature in general, the American Scientist online discussion downplayed societies as major stakeholders in the discussions and decision-making processes regarding electronic publishing initiatives, and assumed societies and individual scientists to have the same stance towards these issues.

Although the American Scientist forum was superficially very similar to the E-Biomed online forum, these debates took quite different developmental paths. We estimate that the American Scientist discussion is quite representative of online discussions of scholarly e-publishing prior to E-Biomed. The American Scientist forum seems much more interactive and less formal than the E-Biomed discussion. It focused on issues that were ignored by the E-Biomed discussion, and ignored issues central to the E-Biomed discussion. Most importantly, the American Scientist forum gave no consideration to a key E-Biomed issue that scientific societies may have reasons to actively oppose e-publishing policies that individual scientists support. These differences lead us to conclude that if E-Biomed’s creators had examined the American Scientist forum or one of the other online discussions of e-publishing at the time, they would have found very little to help them predict the style and substance of the later E-Biomed controversy. The E-Biomed forum’s creators were not naïve or foolish when they failed to anticipate the reactions and influence of the scholarly societies. They were simply dealing with a situation that was very different than earlier literature and discussions of scholarly electronic publishing may have led them to expect.

Limits of the ArXiv.org Model

Early in the E-Biomed controversy, Paul Ginsparg persuaded Brown and Lipman to use his LANL pre-print server “ArXiv.org” as a model for their first E-Biomed proposal. ArXiv.org has been a successful and enduring pre-print server by almost any standard, and its goals paralleled those of the E-Biomed proposal. Why then was the ArXiv.org model ultimately abandoned as the E-Biomed proposal became transformed into PubMed Central?

To understand why the ArXiv.org model was dropped in the PubMed Central plan, we must consider the networks of social ties between the members of the disciplines, the federal government, and the general public. Several important differences can be found between the social locations of ArXiv.org and E-Biomed. First, ArXiv.org was created by a single scientist (Ginsparg) at a single institution (LANL). ArXiv.org was originally a small enterprise and was not formally sponsored by a major federal funding organization. Its creation thus did not cause major public controversies in the physics research communities. E-Biomed, on the other hand, was not created by a single scientist at a single research institution: it was proposed by the Director of the NIH. The source of the E-Biomed proposal gave it an official weight and suggested a legitimate risk to the entire structure of biomedical publishing. As such, the proposal came under much more intense scrutiny than did the creation of ArXiv.org.

Second, when Paul Ginsparg created ArXiv.org, he built on a thirty year tradition of circulating pre-prints in particle physics. Particle physicists tend to see new results first in the form of non-peer-reviewed technical reports that are circulated to laboratory libraries. A central pre-prints electronic server dovetailed neatly with these long-standing practices and relationships between researchers. However, the biomedical research communities do not have this tradition of widely disseminating technical reports prior to peer review: most biomedical discoveries are first reported in major research journals. The biomedical pre-print server proposal was a much more direct challenge to publishing and governance regimes already in place, than ArXiv.org was to the physics communities.

Finally, ArXiv.org is based at Los Alamos National Laboratories, a military weapons laboratory managed by the U.S. Department of Energy and Department of Defense. The NIH is dependent upon scientific societies and publishers for support in Congressional budget hearings that determine the resources and objectives of the NIH. In other words, the NIH is beholden to scientific societies and publishers for funding support in a way that is different from the military governance of LANL. This set of relationships made the proposal (and any NIH proposal) much more sensitive to the interests of biomedical societies than a LANL initiative would be to physics research societies.

In short, the ArXiv.org model was not effective for E-Biomed because ArXiv.org is grounded in one web of financial, political, scientific and technological relationships, and the E-Biomed proposal was generated within a very different web of relationships. Understanding the rich complexity of these social and technological relationship webs, or “Socio-Technical Interaction Networks” (Kling, et al., 2000), is an important step towards understanding the models used for scholarly electronic publishing initiatives.


In this article, we examined the process by which the E-Biomed policy proposal was transformed into the PubMed Central policy, focusing in particular on the varied communication forums in which the E-Biomed proposal was discussed, and ultimately rejected. In the course of our analysis, we made some observations about the process by which stakeholders transform a public policy initiative:

With these considerations in mind, the transformation of E-Biomed to PubMed Central becomes more understandable. E-Biomed’s designers may simply have expected the policy debate to take a different path than it ultimately took. Varmus’ seeming disappointment regarding the small number of favorable responses on the online E-Biomed forum indicates that the proposal’s designers may have expected a major groundswell of grassroots support in the online forum. However, in contrast to this optimistic vision of scientific community consensus, different stakeholders clearly had different perceptions of who constituted the “scientific community,” and of which “interest groups” should have decision-making power for the community. These stakeholder interests precluded the emergence of a unified response from the biomedical research communities.  In the end, professional society representatives and individual scientists found they were not sitting on the same side of the table.

Hilgartner (1995) uses the concept of a “communication regime” to characterize different structures of scientific communication.  He was particularly interested in comparing communication via molecular databases, such as GenBank , with biological journals.  For Hilgartner, a “scientific” journal regime includes:

…not only writers, readers and printing presses, but also the means to persuade people to submit material, to decide what to publish, to acquire funds, to distribute the product, and to manage legal issues such as copyright.  Clearly, the system is deeply embedded in the practices and infrastructure of science and in the wider societal infrastructure of libraries, postal services, and paper mills.
Examining the transformation of E-BioMed to PubMed Central from a ‘communication regime’ viewpoint, we see that significant changes to the bio-medical science journal communication regime existed in the original proposal.  Substantial shifts included those regarding: gatekeeping, the business model, speed of information sharing, mobilization of authors, and the communication infrastructure (see Table 4).

Table 4
Scientific Communication Regimes

E-Biomed draft, May 5 (1999) 
 PubMed Central(2001)
 Editorial Boards, Peer-review  Legitimate authors (.edu or .gov)  Authors  Societies, journals
Business Model
 Subscription, authors may pay page charge to journals  Free to readers and to authors  Free to readers and to authors  Free to readers and to authors (possible page charges to authors by journals)
Copyright ownership
 Journal Author Author Journal
Speed of information sharing  (from time that author could allow access to outside readers) 
Months – years  Days  Days  Months – years
Mobilization of Authors
Prestige of journal  Publication speed, accessibility Publication speed, accessibility   Not Known
Communica-tion Infrastructure
 Postal system  Computer networks Computer networks   Computer networks

A more general conclusion can be drawn from these events. Professionals and researchers commonly refer to the ways that the Internet is “transforming” some aspects of social life. This phrasing makes “the Internet” into a powerful autonomous actor, and is fundamentally incorrect. Rather, as the E-Biomed controversy case illustrates, various groups shape Internet resources and forums to serve their interests. Sometimes these Internet ventures help to transform social relationships and institutional arrangements. In other cases, such as the case of E-Biomed and most electronic journals, these ventures preserve pre-existing institutional arrangements while offering some useful incremental expansion of pre-existing services.


Funding was provided in part by NSF Grant #SBR-9872961 and also with support from SLIS at Indiana University. This article benefited from helpful discussions with, and comments from Phil Agre, Blaise Cronin, Amy Friedlander, Jeff Hart, Robin Peek, Sharon Ross and Suzanne Thorin. In addition, we are indebted to numerous particle physicists and molecular biologists who have discussed their publication and reading practices with us in this and other studies. A short version of this article was presented at the Annual Meeting of the Society for the Social Studies of Science in Vienna, Austria in September 2000.

Appendix A: Research Methods

Archival and documentary (Appendix D). (to be completed)

Note: we sent an early draft of the ms. to Drs. Brown and Lipman for comment, but they did not respond.

Appendix B: Elements of NIH E-Publishing Proposals

 E-Biomed Draft
 E-Biomed Addendum
 PubMed Central
May 5  June 20  Late July – early August August 30
Who posts 
 Authors  Authors Scientific groups and journal publishers Participating publishers and societies
When Posted
Before or immediately at publication      Any time after acceptance – delay after publication up to 1 year (four weeks, two months, 6 months, …)
Peer Review?
1) pre-print repository with screening &2) peer review by editorial boards – either existing or new Still two parts BUT: “We expect that the vast majority of reports are likely to be submitted through editorial boards”  Published reviewed reprints only  Up to publishers, in proposal still both, BUT result soon peer-reviewed only (condition of PNAS) – Varmus & Lipman still want separate repository
Who Screens 
NIH, journals, existing and new editorial boards, and/or 2 experts      Journals, societies, existing editorial boards
Unspecified  Still unspecified   Perhaps $1-3 million annually
Access Fees 
Free Access Free Access Free Access Free Access
Authors Authors or journals, for individual editorial boards to resolve No debate or change With submitting groups
Governing Board of readers, authors, editors, computer specialists, & funding agencies Advisory Board   An international advisory committee will be constituted by learned societies
biomedical May expand Expanded scope Expanded to include life sciences in general


Appendix C:  Society & Publisher Positions vs. Reader and Author Positions

 Reader interests
Author interests
Society & Publisher interests
When posted
As soon as written 
 As soon as written 
Peer-review period + publication processing +PNAS: 4 weeks delay between publication and posting to PubMed Central;EMBO journal: 6 months delayMolecular Biology of the Cell:  2 months delay
Who posts
Readers are interested in most current research à reader support for author self-posting?BUT readers interested in filtering and opposed to info overload à favor gate-keeping Authors tend to be in favor of self-postingBUTReputation and prestige comes from journal publication American Society of Biochemistry and Molecular Biology: “There is also the question of who would be allowed to deposit information. Would, for example, high school and college students or lay people be able to submit their data, thoughts or hypotheses to the repository?”
Pre-prints put up at authors discretion even before peer-review ? 
Of 90 academics who responded with electronic comments to NIH: about 25 were critical of peer review or supportive of pre-prints, about 25 found peer review crucial and did not want to see any unreviewed materials, about 40 did not mention peer review (but of these about 25 were mostly or strongly in favor of the proposal) à this means that readers and authors who responded electronically to NIH were divided on the peer-review issue, with a substantial enough number in favor of online pre-prints or at least not opposed to them
Same as reader interests 
· American Society of Biochemistry and Molecular Biology: “All of the opinions that have been received from the Council, JBC Editorial Board members and members of ASBMB stress the enormous importance of peer review.” 
· American Association of Immunologists: “scientists … do not want to be held to lesser standards.” – speaking on behalf of all scientists, implying all favor peer-reviewed materials only 
· One of the conditions of PNAS participation in PubMed Central: there will be no non-peer-reviewed materials on this serverà many societies and publication committees do not acknowledge the support for pre-prints or non-peer-reviewed materials
Not applicable 
Authors retain copyright 
American Association of Anatomists:  “Copyright for E-Biomed should, in most cases, transfer from E-Biomed to the relevant journal upon acceptance for publication in the relevant journal. Archival information should reside with the journal. The E-Biomed version of a paper should be deleted upon acceptance and publication in the journal.” – note, the AAA was one of few societies in favor of pre-print server, but with these conditions on copyright

 Pre-prints put up at authors discretion even before peer-review ?Of 90 academics who responded with electronic comments to NIH: about 25 were critical of peer review or supportive of pre-prints, about 25 found peer review crucial and did not want to see any unreviewed materials, about 40 did not mention peer review (but of these about 25 were mostly or strongly in favor of the proposal) à this means that readers and authors who responded electronically to NIH were divided on the peer-review issue, with a substantial enough number in favor of online pre-prints or at least not opposed to them · American Society of Biochemistry and Molecular Biology: “All of the opinions that have been received from the Council, JBC Editorial Board members and members of ASBMB stress the enormous importance of peer review.”· American Association of Immunologists: “scientists … do not want to be held to lesser standards.” – speaking on behalf of all scientists, implying all favor peer-reviewed materials only· One of the conditions of PNAS participation in PubMed Central: there will be no non-peer-reviewed materials on this serverà many societies and publication committees do not acknowledge the support for pre-prints or non-peer-reviewed materials
Copyright Not applicable Authors retain copyright American Association of Anatomists: “Copyright for E-Biomed should, in most cases, transfer from E-Biomed to the relevant journal upon acceptance for publication in the relevant journal. Archival information should reside with the journal. The E-Biomed version of a paper should be deleted upon acceptance and publication in the journal.” – note, the AAA was one of few societies in favor of pre-print server, but with these conditions on copyright

Appendix  D: Key Documents About PubMed Central

“PubMed Central: An NIH-Operated Site for Electronic Distribution of Life Sciences Research Reports” 30 August 1999

“Archive of Comments on E-biomed: A Proposal for Electronic Publications in the Biomedical Sciences (May 5, 1999 DRAFT)” May 5 – August 22.   http://www.nih.gov/welcome/director/ebiomed/comment.htm

“Selected citations regarding E-biomed: A Proposal for Electronic Publications in
the Biomedical Sciences” http://www.nih.gov/welcome/director/ebiomed/ebiomedcit.htm

American Society for Pharmacology and Experimental Therapeutics, Public Affairs News Brief No.13. July 16, 1999.

American Scientist. 1998.  September 98-Forum on “Free Internet Access to Traditional Journals”

“Netprints, Clinical Medicine & Health Research.” http://clinmed.netprints.org  (searched on October 2, 2001)

Bloom, Floyd. “Just a Minute Please (an editorial).” Science. 9 July 1999. 285 (5425): p. 197.

Butler, Declan. (1999, January 14) “US biologists propose launch of electronic preprint archive.” Nature.  397: p. 91.

Butler, Declan. “NIH plan brings global electronic journal a step nearer reality.” Nature. 29 April 1999. 398: p. 735.

Delamonte, Tony, Richard Smith, Michael Keller, John Sack, and Bill Witscher. “Netprints: the next phase in the evolution of biomedical publishing.” (an editorial). BMJ. 11 December 1999 319: 1515-1516.

Horton, Richard, David Sharp, and John McConnell. “The Lancet response.” 14 May 1999. http://www.thelancet.com/newlancet/reg/nochange/body.nihdiss_3.html

Jacobson, M. “Biomedical publishing and the Internet – the message of the medium.” (editorial) Journal of Intensive Care Medicine. 1998. 13: 153-154.

Kassirer, Jerome and Marcia Angell. “The New England Journal of Medicine Response.” 4 May 1999.

Macilwaln, Colin. “Mixed Response to NIH’s web journal plan.”  Nature. 6 May 1999. 399: p. 8-9.

Marshall, Eliot. “NIH Weighs Bold Plan for Online Preprint Publishing.” Science. 12 March 1999. 283 (5408): p. 1610- 1611.

Marshall, Eliot. “Varmus Circulates Proposal for NIH-Backed Online Venture.” Science. 30 April 1999. 284 (5415): p. 718.

Marshall, Eliot. “Varmus Defends E-biomed Proposal, Prepares to Push Ahead.” Science. 25 June 1999. 284 (5423): p. 2062-2063.

Marshall, Eliot. “U.S., European Backers Differ on E-biomed Plan.” Science. 16 July 1999. 285 (5426): p. 315.

Marshall, Eliot. “E-biomed Morphs to E-biosci, Focus Shifts to Reviewed Papers.”Science. 6 August 1999. 285 (5429): p. 810 - 811.

Marshall, Eliot. “NIH’s Publishing Venture Ready to Launch.” Science. 3 September 1999. 285 (5433): p. 1466.

Marshall, Eliot. “View From the Top of a Biomedical Empire.” (Science interview). Science. 10 September 1999. 285 (5434): p. 1654 - 1656.

Marshall, Eliot. “Varmus to Leave NIH in December to Run Sloan-Kettering.” Science. 15 October 1999. 286 (5439): p. 382.

Marshall, Eliot. “PNAS to Join PubMed Central – On Condition.” Science. 22 October 1999. 286 (5440): p. 655 – 656.

Marshall, Eliot. “Researchers Plan Free Global Preprint Archive.” Science. 29 October 1999. 286 (5441): p. 887.

Relman, Arnold. “The NIH ‘E-biomed’ Proposal – A Potential Threat to the Evaluation and Orderly Dissemination of New Clinical Studies.” (an editorial). New England Journal of Medicine June 1999. 340: 1828 – 1829.

Ross-Flannigan, N.. 1998.  "The Virtues (and Vices) of Virtual  Colleagues,"  Technology Review, Cambridge, MA: MIT,   (March/April).

Smaglik, Paul. 1999. “Varmus Seeks Societies’ Support for Electronic Journal.” (an interview with Varmus). The Scientist. 7 June. 13 (12): p. 1.

Smaglik, Paul. 1999. “E-Biomed Becomes PubMed Central.” The Scientist. 27 September. 13 (19): p. 8.

Smaglik, Paul. 1999. “NIH Chief to Step Down.” The Scientist. 25 October. 13 (21): p. 25.

Varmus, Harold, David Lipman and Pat Brown. 1999a. “Original Proposal for E-biomed (E-BIOMED: A Proposal for Electronic Publications in the Biomedical Sciences).” National Institutes of Health, Bethesda, MD. (May 5). (Available at:
http://www.nih.gov/welcome/director/pubmedcentral/ebiomedarch.htm )

Varmus, Harold, David Lipman and Pat Brown. 1999b. “Addendum” to  Varmus, Lipman and Brown (1999a) ).” National Institutes of Heath, Bethesda, MD. (June 20). (Available at: http://www.nih.gov/welcome/director/pubmedcentral/ebiomedarch.htm#Addendum

Wingerson, Lois (editor), Harold Varmus, Mary Waltham, Michele Hogan, Karen Hunter, Paul Ginsparg, and Vitek Tracz. 1999. “Journals Online: PubMed Central and Beyond (a debate).”  HMSBeagle: The BioMedNet Magazine (http://www.biomednet.com/hmsbeagle/61/viewpts/overview), Issue 61 (September 3).


Bailey, Charles W. Jr. 2000. Scholarly Electronic Publishing Bibliography.Version 34: December 1. University Libraries, University of Houston, Houston, TX at: http://info.lib.uh.edu/sepb/sepb.html

Brody, Herb. 1996. “Wired Science” Technology Review October. Donovan, Bernard. "Learned Societies and Electronic Publishing." Learned Publishing 11 (April 1998): 91-107

Ginsparg, Paul. 1996. “Winners and Losers in the Global Research Village” In Shaw, Dennis and Moore, Howard (eds.). Electronic Publishing in Science, Proceedings of the Joint ICSU/Press/UNESCO Expert Conference. ICSU Press, 83-88.

Harnad, Stevan. 1992 “Interactive Publishing: Extending American Physical Society’s Discipline-Specific Model for Electronic Publishing” Serials Review, Special Issue on Economics Models for Electronic Publishing, 58-61.

Harnad, Stevan. (1999) “Free at Last: The Future of Peer-Reviewed Journals” D-Lib Magazine 5, no. 12

Hilgartner, Stephen. “Biomolecular Databases: New Regimes for Biology?” Science Communication, Vol. 17 No. 2,December 1995, 240-263 Sage Publications, Inc.

Ingelfinger, Franz. “Definition of ‘sole contribution.’” New England Journal of Medicine 1969; 281:676-7

Kling, Rob and Roberta Lamb (1996). "Analyzing Visions of Electronic Publishing and Digital Libraries."  In Newby, G. B. & Peek, R. M. (eds.).
Scholarly Publishing:  The Electronic Frontier. Cambridge Ma: The MIT  Press. [not in text yet]

Kling, Rob and Geoffrey McKim. (2000). "Not Just A Matter Of Time: Field Differences And The Shaping Of  Electronic Media In Supporting Scientific Communication” Journal of the American Society for Information Science 51(14) . [not in text yet]

Kling, Rob, Geoffrey McKim, Joanna Fortuna and Adam King. 2000. A Bit More To IT: Scientific Communication Forums as Socio-Technical Interaction Networks. CSI WP.

LaPorte, Ronald, Eric Marler, Shunichi Akazawa, Francois Sauer, Carlos Gamboa, Chris Shenton, Caryle Glosser, Anthony Villasenor, and

Malcolm Maclure. “The Death of Biomedical Journals.” BMJ. 27 May 1995. 310: 1387-1390

Markovitz, Barry. “Electronic Journals: Time for a New Paradigm.” Journal of Intensive Care Medicine. 1998. 13: 158-167.

Odlyzko, A. M. (1996). “Tragic Loss or Good Riddance? The Impending Demise of Traditional Scholarly Journals”.  In R. Peek and G. Newby (eds.), Scholarly Publishing: The Electronic Frontier (pp. 91?102). Cambridge, MA: MIT Press.

Peek Robin and Greg. Newby (eds.). 1996.  Scholarly Publishing: The Electronic Frontier. Cambridge, MA: MIT Press.

Rheingold, Howard.  "Electronic Democracy."  Whole Earth Review, Summer 1991, pp. 4-13. . [not in text yet]

Thomas, Timothy. 1998. Archives in a New Paradigm of Scientific Publishing: Physical Review Online Archives (PROLA). D-Lib Magazine. (May) (Available at: http://www.dlib.org:80/dlib/may98/05thomas.html ) . [not in text yet]

Walker, Thomas J. 1998. “Free Internet Access to Traditional Journals.” American Scientist. (September-October), 86(5)