Opening Up Communication : Assessing Open Access Practices in the Communication Studies Discipline

INTRODUCTION Open access (OA) citation effect studies have looked at a number of disciplines but not yet the field of communication studies. This study researched how communication studies fare with the open access citation effect, as well as whether researchers follow their journal deposit policies. METHOD The study tracked 920 articles published in 2011 and 2012 from 10 journals and then searched for citations and an OA version using the program Publish or Perish. Deposit policies of each of the journals were gathered from SHERPA/RoMEO and used to evaluate OA versions.  RESULTS From the sample, 42 percent had OA versions available. Of those OA articles, 363 appeared to violate publisher deposit policies by depositing the version of record, but the study failed to identify post-print versions for 87 percent of the total sample for the journals that allowed it. All articles with an OA version had a median of 17 citations, compared to only nine citations for non-OA articles. DISCUSSION & CONCLUSION The citation averages, which are statistically significant, show a positive correlation between OA and the number of citations. The study also shows communication studies researchers are taking part in open access but perhaps without the full understanding of their publisher’s policies.


INTRODUCTION
Some library science scholars have theorized that one of the benefits of making articles open access (OA) is an increase in the number of times the scholarship, on average, is cited (Antelman, 2004).Doty (2013) coined the term "OA citation effect" to describe the relationship between access and citations.Open access advocates find this measure valuable as it may show the greater impact that OA articles have over other scholarly articles that remain behind a toll-access (TA) gateway.To support this theory, a number of studies have compared the intra-journal citation rates for published articles made OA via deposit into a repository with the citation rates for articles that remain behind a paywall.
The initial focus on access and citation in the literature was on the natural sciences, where OA has seen much buy-in, but more recent findings show that differences in the citation rate might depend on the discipline or field of an article (Antelman, 2004;Hajjem, Harnad, & Gingras, 2005;Norris, Oppenheim, & Rowland, 2008).Several studies have branched out from the natural sciences into various social science fields (Antelman, 2004;Norris, Oppenheim, & Rowland, ., 2008;Atchison & Bull, 2015), but so far none have examined citation rates for OA articles in the discipline of communication studies, which encompasses a broad range of subdisciplines, including communication theory, health communication, mass media, interpersonal communication, political communication, and performance studies (National Communication Association, 2015).The National Communication Association defines the discipline as studying "how people use messages to generate meanings within and across various contexts, cultures, channels, and media," (2015, para.1).In researching the OA citation effect on communication studies, this study will seek to answer the following questions: 1. Do OA communication studies articles have higher citation rates than TA articles from the same journal?2. What is the difference, if any, between citation rates for OA and TA communication articles?3. Are authors or depositors following publisher guidelines for making articles available openly?

OA Effect and the Social Sciences
A number of studies have investigated citations of green OA articles, which is when a work has been deposited in an open repository or website (Suber, 2015), with many of them finding a positive correlation (SPARC Europe, 2016).Although most initial research studies into the open access citation effect (OACE) looked at the sciences (Metcalfe, 2006;Moed, 2006;Eysenbach, 2006;Henneken et al., 2006;Davis & Fromerth, 2006), a number have since looked at the OACE on various social science disciplines.Antelman, in one of the first OACE studies, looked specifically at citation rates in both the sciences and social sciences: mathematics, electrical and electronic engineering, political science, and philosophy (2004).Antelman studied 2,017 articles from the four fields and found that OA articles in each of the four disciplines did see higher citation rates than TA articles in the same fields but that the rates differed, ranging from as low as a 45 percent increase for philosophy to a 91 percent increase for mathematics (2004).
Other studies have also looked at the OACE on the social sciences, including the work by Norris et al. in ecology, applied mathematics, sociology, and economics.Like Antelman's findings, OA articles in all four fields had higher citation averages than TA articles (Norris et al., 2008).However, the results again show differences among each discipline, ranging from a 44 percent (ecology) increase to 88 percent (sociology) (Norris et al., 2008).The authors noted the need to realize disciplines would show different rates, saying "it is evident that the level of OA is subject dependent and that within these subjects there are different levels of authorship and citation practices, thereby making it difficult to explain the cause of any OA citation advantage," (Norris et al., 2008(Norris et al., , p. 1970)).
In a more recent study of the OACE in the social sciences, Atchison and Bull looked at the citation rates of OA and TA articles in the political science discipline.They found that OA articles had a significantly higher mean number of citations than TA articles, noting "this holds across the data sample as well as within each of the included journals; therefore, OA publication results in a clear and significant citation advantage," (Atchison & Bull, 2015, p. 5).The study also analyzed whether articles made OA adhered to the publisher's deposit policies, determining publisher policies by referring to the SHERPA/RoMEO database.They found that many articles are made OA despite being published in a journal with a policy that forbids the practice (Atchison & Bull, 2015).Another study by Laakso and Lindman used a similar process to track which versions of TA articles in the information science discipline were green OA and found that the publisher's version was the most common, again often seemingly in violation of the journal's copyright policy (2016).Noting the variety of websites where these deposited articles live, Laakso and Lindman took the process a step further by also analyzing the types of websites where they discovered these articles (2016).The authors argue that for green OA to survive, authors must respect the deposit policies of their publishers (Laakso & Lindman, 2016).
Other studies have also shown a connection, including one large study that looked at a number of fields.Hajjem et al. found that OA articles in sociology, political science, economics, education, law, business and management all saw more citations than TA articles, ranging from increases of 49 percent (economics) to 172 percent (sociology) (2005).Another study on agricultural articles showed OA versions had a mean citation rate of 5.7, versus three for TA articles (Kousha & Abdoli, 2010).Finally, Donovan, Watson, and Osborne (2015) showed that OA correlated to a 53 percent increase in citations for law articles.

OA Citation Effect Practices
Harnad and Brody were two of the first to argue that researchers cannot compare citation rates of gold OA articles, meaning those published in an OA journal, to those published in a TA journal (2004).Instead, they advocated for a different methodology, since followed by many of the OACE studies.Harnad and Brody argued that researchers must compare articles published in the same journals by looking at those made OA through a deposit in a repository or a website vs. those articles that remain TA (2004).By doing this, they argue, researchers can help factor out other possible variables resulting from the articles appearing in different journals (Harnad & Brody, 2004).
While these OACE studies have focused on green OA articles, the vast majority of OACE studies appear to have not focused on another distinction, that between between gratis and libre OA.The 2002 Budapest Open Access Initiative (BOAI) definition of OA stresses the need for open access articles to be both free and licensed for reuse in such a way that the only requirement for reuse should be attribution (2002).However, Suber since has posited two broad categories of OA: libre and gratis (2008).He defines libre OA similarly to the BOAI definition (2008).On the other hand, gratis OA is when an author has made a work freely available but has not licensed it for reuse (Suber, 2008).Researchers appear to have focused on gratis OA, looking only at whether an article is available for free online (Wohlrabe & Birkmeier, 2014;Antelman, 2004;Atchison & Bull, 2015;Norris et al., 2008;Kousha & Abdoli, 2010).
Many of the studies on the OACE, especially the initial ones, have relied on Web of Science to collect citation counts (Antelman, 2004;Xia, Myers & Wilhoite, 2011;Kousha & Abdoli, 2010;Norris et al., 2008) and Google to find OA versions of articles.However, de Winter, Zapdoor, and Dodou (2014) showed that when searching for citations for 52 articles, only 6.8 percent of the citations found in Web of Science could not be found in Google Scholar, whereas 57 percent of the citations found in Google Scholar could not be found in Web of Science."This study shows that Google Scholar now covers a large share of the scientific literature, suggesting that Google Scholar is an invaluable tool for conducting literature research," ( de Winter, Zapdoor, & Dodou, 2014, p. 1,562).De Winter et al. also showed that although Web of Science remains strong in the hard sciences, Google Scholar is better for the social sciences (2014), which include communication studies.
Some OACE researchers have begun to look at whether using Google Scholar can be just as, if not more, effective for collecting both citation counts and OA article versions.Norris et al. tested Google Scholar, along with Google, OpenDOAR, and OAIster, and found that both Google and Google Scholar found a large majority (84 percent) of the OA versions of articles being searched (2008).In their look at the OACE in political science, Atchison and Bull combined the use of Google Scholar with the program Publish or Perish (2015).
The program, which uses Google Scholar and Microsoft Academic Search to come up with citation counts, was created to help tenure-track faculty search for citation counts for their articles (Herzig, 2015) and was also used by Xia to track citations in his study of library science journals (2012).Atchison and Bull used the program to conduct a single search that both returned citation counts and indicated whether an OA version was available (2015).
Some people have criticized the methodology of extant OACE studies (Kurtz et al., 2005).
One particular concern is that authors are choosing to make only their best work OA (self-selection bias), which could account for increased citations counts for these articles.However, Xia and Nakanishi (2012) studied articles from the top 10 and bottom 10 anthropological journals, as indexed by Journal Citation Reports, and found no evidence that more articles from the top 10 journals were OA than those from the bottom 10.Xia and Nakanishi found that the evidence ruled against self-selection bias: "…in other words, an author's decision to choose open access publishing disregards the prestige of their publication; and the self-selection theory is not supported at this step of the analysis," (2012, p. 48).Gargouri et al. (2010) found that articles deposited under an OA mandate did not see fewer citation counts than articles voluntarily deposited.The authors argued that this pro-vided more evidence against self-selection bias, as authors might select only their best work when depositing voluntarily.Another criticism of OACE research is that OA articles could gain more citations through early access, as authors could post them to a subject repository such as arXiv before publishing them in a journal (Kurtz et al., 2005).However, Xia, Myers, and Wilhoite note that only a few fields have a strong preprint culture in place-specifically physics and economics-and argue that the common practice of posting an article's final, published version online shows early access is not a problem for most fields (2011).

OA and Communication Studies
At least one communication studies association, the International Association for Media and Communication Research (IAMCR), has shown strong support for OA, including converting all of its journals to OA and mandating its publication committee to "promote open access to IAMCR publication activities," (2015, para.5).IAMCR also specifically included OA as one of six principals in a charter it authored on research activity, noting that "the universal free exchange among researchers of intellectual work should be regarded as being of critical importance to maintaining democratic order," (n.d., p. 1).Pooley argues that communication studies researchers should have a more vested interest in OA publishing as "our familiarity with the changing modalities of communication, moreover, make us good candidates for publishing-format alternatives to the printed page and the PDF… We should be among the experimenters," (2016).He also argues that the discipline's interdisciplinary nature benefits researchers taking part in OA as the discipline is not dominated by a few top, core journals, leaving researchers seeking tenure more freedom to publish in up and coming journals (Pooley, 2016).
Although no study has yet looked at the OACE in communication studies articles using the preferred methodology of Harnad and Brody, other studies have looked at the relationship between OA and the communication studies discipline.Husain and Nazim found that there were 106 gold OA communication journals in the field, which has seen a recent growth spurt in the 2000s (2013).Another study found that articles in OA communication studies journals had similar citation practices as TA communication journals, which the author argues shows they have a similar level of quality (Poor, 2009).
One study attempted to measure the OACE for communication studies, although it did so by comparing one gold OA journal to one TA journal (Zhang, 2006), the methodology dismissed by Harnad and Brody (2004).Zhang found that articles in the OA journal enjoyed almost twice as many average citation counts as those in the TA journal, noting that "such evidence suggests that the access mode may be the variable affecting the web citations counts to journal articles, with OA articles receiving more web citations," (2006, p. 154).This resulted in 11 journals, including one-Journal of Computer-Mediated Communication -that is a gold OA journal.Including JCMC in the overall analysis introduced very little change to the average citation count for OA articles and no difference at all to the median citation count.However, as JCMC is a gold OA journal and thus does not match the other TA journals in the study, the author decided to not include the journal, leaving 10 journals in the study.

METHODS
All articles-except editorials, letters, and reviews-published in 2011 and 2012 from those journals were used for the study sample.By selecting Journal Citation Report's highest-ranked journals for this study, any concern of perceived article quality or lack thereof should be limited, although this is a subjective issue that can never totally be accounted for.
Articles published in the years 2011 and 2012 were used for two reasons: to factor out an article's age as the reason for any change in citation counts and to allow adequate time for articles to establish stable citation counts.
Data collection took place in late January through early March 2015 and tracked each article's journal of publication, number of authors, page length, first author's country of residence, available OA, type of OA version available (preprint, postprint, publisher's version), deposit policies for each journal at the time of data collection, and the number of citations for each article.
Most of the data-the journal the article appeared in, number of authors, page length and country of first author-were collected from each journal's website.The study then used the program Publish or Perish to count citations and find whether an article had been made OA.This was done specifically by searching each article title (and, when needed, journal title and author name[s]) in Publish or Perish, which then returned a list of results that included citations.When Publish or Perish had two entries for the same article, the first was used as this was the method employed by Atchison and Bull (2015), and this method consistently preferred the entry with the most citations.
The author then followed the article title link from the Publish or Perish results page, which opened up Google Scholar search results for that article in a web browser.Although prior OACE studies have used Web of Science to collect citation counts, evidence from de Winter et al. (2014) shows that Google Scholar can be equally effective in this area, which is why it was used.Each version of an article was then examined to determine 1) whether that version was OA, and if so 2) was it a preprint, postprint, or publisher's version.If an article was not specifically listed as a preprint, postprint, or version of record, the study determined its status by looking for certain visual clues.For instance, an article that appeared to be a text document with no journal title and no other design details was considered a preprint.An article that had been given a journal's design and formatting but lacked a volume and issue number was considered a postprint.An article that had all such elements was considered the publisher's version.
Finally, a journal's deposit policy was collected from SHERPA/RoMEO, a site that tracks deposit policies for many scholarly journals and publishers, by searching each journal title.
Although the policies often have various nuances, and institutions and authors can have their own separate agreements with publishers, this study tracked which versions were allowed by policy in a not-for-profit repository and whether an author was allowed to retain the copyright.The data was then analyzed with the statistics program STATA, using a Wilcoxon-Mann-Whitney test (instead of a t-test) because the data was not normally distributed.
A second round of data collection took place in July 2015, similar to the methodology of Laakso and Lindman (2016), to determine whether any of the OA articles had been deposited on an author's individual site, whether on their university page or on a social research site such as ResearchGate.netor Academia.edu.This was done in order to help narrow which OA articles were more likely than not to have been deposited by an author or at the direction of an author.It cannot be assumed that the author was responsible for depositing an article and thus was responsible for following or breaking a journal's deposit policy.For example, several studies show faculty authors often do not actively take part in depositing their works into institutional repositories, with library staff more likely to take a role in not only depositing but also overseeing copyright issues (Yang & Li, 2015).However, an article posted on a researcher's personal site was assumed to be more likely deposited by or at the direction of the author.This step of the study was done by running the OA articles through Publish or Perish again and viewing each OA version on its website.If it was not clear what type of website the article was on, the article was not counted as being posted on an author's individual page.
There are several limitations to this methodology.Because Journal Citation Reports does not track every journal, there could well be other top-quality journals that are not included in this study.In addition, journal quality is subjective; Kurmis notes that biases can influence impact factors and that impact factors should not form the basis for judging the quality of a journal or authors (2003).Thus, the use of Journal Citation Reports in this study should not be understood as a proxy metric for assessing journal quality.Articles appearing in journals with a high impact factor might receive more citations simply because they appear in that journal, not because they are the best articles in the field.Another limitation involves using Google Scholar to search for OA versions and citation counts.Although studies have shown Google Scholar finds more OA articles than Web of Science, it might not find every OA version or citation and it could double count citations.That would mean this study might not find all OA versions of and citations to the articles in the study sample.Finally, this report also relied on SHERPA/RoMEO for journal deposit policies, and thus any errors SHERPA/ RoMEO would include about these journals would then affect the results.

RESULTS
From the 10 journals, 920 total articles were included in the study sample.Of these, 388 were OA, or about 42 percent (see Table 1).The publisher's version was most the most common OA version available, comprising 266 of the articles (see Table 2).The method for deciding whether an article's version was the preprint, postprint or publisher's version was not perfect, as distinguishing between preprints and postprints is not always clear.However, as publisher policies will show, this is not an issue of major concern as the vast majority allow both the pre-and postprint versions to be deposited online.

PUBLISHER POLICIES
All but one of the journals included in this study have a policy allowing authors to make the postprint version OA, and the remaining journal allows them to make the preprint version OA.However, none of them (unless an author has negotiated directly with the publisher to make an article OA) allows for posting the publisher's version.This suggests that up to 263 articles in the sample may have improperly made the publisher's version OA (two included Creative Commons licenses on the publisher's site, indicating approval from the publisher).
Because of various restrictions that many of the journals place on OA postprint deposit (such as limiting deposit to not-for-profit web servers), some of these could also be improperly posted, but this evaluation was outside the scope of this study.However, 85 percent of the articles that came from the nine journals that allowed deposit of the postprint have not had the postprint version made OA (see Table 3).

OA ARTICLES AND WEBSITES
The second round of data collection determined that 227 of the articles were posted on a personal page (59 percent of OA articles), with 74 percent of those consisting of the publisher's version.Deposit on a personal website could indicate the authors or their assistants deposited the work, versus a librarian or other repository worker.This number does not include 18 articles where the nature of the deposit site could not be determined and seven articles that could no longer be found freely available on Google Scholar during the second search.

OA ARTICLES AND CITATIONS
Statistical analysis of the OA articles showed that they averaged about twice as many citations as the TA articles did (see Table 4).The study also looked at the data for each individual journal.Nine had higher citation counts for OA articles than TA articles, with Public Communication being the only journal to show lower citation counts for OA (see Table 5).This evidence follows other OACE studies (Antelman, 2004;Norris et al., 2008;Atchison & Bull, 2015;Kousha & Abdoli, 2010;Donovan et al., 2015) in presenting the results in terms of the average.However, as extreme numbers can skew the average, this study also includes median citation counts to counteract this and help give a clearer picture of the data.

DISCUSSION
Because the citation counts do not appear to be normally distributed, determined both by a view of a histogram of citation counts and a skewness of 10, it was determined a Wilcoxon-Mann-Whitney test would be a better choice for data analysis than a two-sample unpaired t-test.However, both tests returned a p value of <.001, below the level of 0.05, meaning there is a significant statistical correlation between an article being OA and its citation rate.This would appear to support previous research showing a connection between the two factors.Of the individual journals, two returned a p value greater than 0.05, meaning the results are not statistically significant.This includes Public Communication, the only journal that returned a higher citation average for TA articles than OA articles, and Human Communication Research.The remaining eight journals, all of which had higher citation averages for OA articles, returned a p value lower than .05,meaning these numbers are also statistically significant.
The results also appear to support evidence from other studies that researchers are making their articles OA without the apparent approval of their publishers (Atchison & Bull, 2015;Laakso & Lindman, 2016).Just under three-quarters of the OA articles had a publisher's version posted, when none of the publisher's policies reviewed through the study allowed that version to be made OA.At the same time, most articles versions identified in the study were not postprints, as a large majority of authors in the sample did not make this version OA (for the 85 percent of articles for which posting of the postprint was allowed by publishers).This could be considered evidence of authors' lack of general knowledge or lack of concern regarding open access and copyright.Although it is impossible to tell without further inquiry who was responsible for making the articles OA, it can be inferred that those posted on an author's personal site-including the 69 percent that were the publisher's version-were likely put up by the author or at least at the author's direction.This would appear to show the authors' willingness, then, to make these articles freely available, if not their understanding of journal deposit policies.More mediation by librarians to advocate for open access, contract negotiations, and licensing could help improve these numbers in compliance with existing journal deposit policies and approved exemptions from journal policies.Some studies show that authors are likely unaware of both the idea of open access and the ways in which the publishing contracts they sign affect their copyrights.In one study, 90 percent of faculty say they sign their copyright contract with a publisher as-is, instead of trying to negotiate for friendly deposit rights (Charbonneau & McGlone, 2013).Another Malaysian study found that 70 percent of researchers said they knew what OA was but could not give a full and correct definition (Abrizah, 2009).
The difference between the number of preprints, postprints, and publisher's versions also suggests that it's unlikely that early access plays a role in these increased citation counts, per the argument by Xia, Myers, and Wilhoite (2011).Both preprints and postprints made up a minority of the openly available articles, with less than a fifth of the OA articles having the preprint available online and less than a third of the OA articles having the postprint available.The publisher's version was the most common version available at almost three quarters.

CONCLUSION
Open access means that anyone can read an article, regardless of affiliation.It makes some sense, then, that more people would read OA articles and thus that they would likely garner increased citations, especially as the serials crisis continues to force academic libraries to cancel journal subscriptions, particularly those in developing countries (Igwe, Oyewo, & Yusuf, 2013).Previous studies have shown some support for this idea in general.This study provides more evidence, especially as it pertains to the discipline of communication studies.
The OA articles, analyzed as a whole, showed an average number of citations (28.4) twice as many as the average for TA articles (14.3).Although not all journals saw such results individually, eight out of 10 did show on average more citations for OA articles than for their TA versions.
This study also indicates that communication studies researchers are interested in at least one major tenet of open access, as almost half of the articles were freely available online.
This compares favorably with other social science disciplines that have been studied.Various studies have found OA articles to make up a broad percentage of study samples, including 14 percent (Kousha & Abdoli, 2010), 22 percent (Donovan & Watson, 2011;Davis, 2011), 30 percent (Xia & Nakanishi, 2012), 55 percent (Xia et al., 2011), and80 percent (Wohlrabe &Birkmeier, 2014).Although not matching the highest percentages of some studies, this study's finding of 42 percent of articles made OA shows similar buy-in to other social science disciplines.
At the same time, this study raises questions about just how much communication studies researchers know about open access and policies supporting open access.More than half of the articles studied were not OA at all, although almost all of the included journals have established OA-friendly policies for the postprint version of an article.This, combined with the fact that so many researchers appear to violate journal policy by depositing the publisher's version, indicates a possible lack of awareness or concern on the part of many authors.Further study on why authors make these decisions could help shed light on just how much deposit practice was based on misunderstanding of copyright and OA, and how much stemmed from a disregard for copyright.However, unless or until publishers make their policies more intelligible to their authors and put processes in place that better facilitate posting preprint or postprint versions, it is also likely that librarians and university administrators will need to do more work to educate faculty members about these issues.This and other studies have also chosen to examine the OACE only on gratis OA articles, when the original (and narrower) definition of OA also includes licensing the articles for reuse (i.e.libre OA).Future studies can look at whether the distinction between gratis vs. libre OA has any further correlation with regard to citations.Finally, future studies could also compare the articles in the study samples with OA mandates by the institutions of the authors to determine what, if any, effect these had on OA rates.
This study followed the example of previous OACE studies and used Journal Citation Reports to identify top journals in communication studies.Journals that ranked in the top 20 in the JCR annually over a five-year period from 2008 to 2013 were used.Although most OACE studies have focused on the top journals in just one year of Journal Citation Reports results, this study follows Atchison and Bull's assertion that journals that consistently rank in the top 20 are more likely to have a steady number of citations, vs. a journal possibly experiencing an outlier year (2015, p. 4).

Table 2 .
Breakdown of OA articles by article version.(Note: One article could have multiple versions.)

Table 3 .
Policy for each journal as to highest level of article allowed to be deposited.(*Policy allows authors to retain copyright.^Only counts articles when highest form allowed has been made OA.)

Table 4 .
Average and median citation counts for OA articles and TA articles.

Table 5 .
Average and median citation counts for OA and TA articles by journal.