Measuring Cost per Use of Library-Funded Open Access Article Processing Charges: Examination and Implications of One Method

Hampson, C., & Stregger, E. (2017). Measuring Cost per Use of Library-Funded Open Access Article Processing Charges: Examination and Implications of One Method. Journal of Librarianship and Scholarly Communication, 5(General Issue), eP2182. https://doi.org/10.7710/2162-3309.2182 Measuring Cost per Use of Library-Funded Open Access Article Processing Charges: Examination and Implications of One Method


INTRODUCTION
As one means of providing open access (OA) support to institutional authors, some academic libraries set up an open access fund to pay OA article processing charges (APCs) on behalf of institutional authors.The Scholarly Publishing and Academic Resources Coalition (SPARC) defines an OA fund as: "a fund set aside by an institution specifically to cover costs for researchers who publish in journals with article processing charges" (Campus open access funds, 2016).In this case, the OA journal charges the author an APC to publish their article in the journal.The library receives requests from their institution's authors to pay such APCs from the library's OA fund.The library's OA fund policy may include various limits, for example a price cap or requirement that authors exhaust other funding options first, such as grants.Not all open access journals charge APCs and, among those that do, the cost of the APC varies substantially (Directory of Open Access Journals, 2017).Libraries began creating OA funds by 2005 (Open access funds in action, 2016).Monies for institutional OA funds may come from the library only or other campus units may contribute, such as university administration, research offices or individual departments (McMillan, O'Brien, & Young, 2016).In some cases, libraries support their OA fund using money from the library's collections budget (Yates et al., 2016).In this situation, the OA fund is one of many demands competing for limited funding to pay for scholarly publishing.In times of financial constraint, libraries may consider stopping their OA fund in order to continue support for other collections items (Yates et al., 2016).Libraries need ways to assess OA funds, not as a service to authors that is paid from the collections budget, but in relation to other collections expenditures, i.e. as a different way to pay for scholarly publishing.
Libraries typically assess the various purchases they make from their collections budgets to inform decisions of where best to invest funds on behalf of their institution.In mak-ing these assessments for traditional models of scholarly publishing of academic journals, libraries often calculate a cost per use as one factor to consider.In this case, the cost per use is calculated as the annual cost of the subscription (either of the individual journal title or the journal package) divided by "use," typically measured as the number of PDF downloads plus the number of HTML article views.To ensure comparable definitions of what constitutes "use," libraries ask publishers and vendors to report usage using the standards developed by Project COUNTER ("Project COUNTER," n.d.)Standard COUNTER journal reports provide usage details for the specific institution, over a specific period of time, for the associated journal title(s).For example, the COUNTER JR1 journal report provides the number of PDF downloads and HTML article views for each journal title listed, over a given period of time.COUNTER JR1 GOA (gold open access) reports the usage of gold OA articles in the listed journal titles.Both reports provide usage at the journal level rather than article level.GOA is used to allow libraries to factor out the usage of OA articles published in otherwise subscription-access journals (hybrid journals).These standardized reports provide usage metrics for libraries to measure a cost per use to evaluate their institution's use against their library's associated annual charges for the traditional subscription payment model for scholarly journals.
In the case of OA APCs, the cost is a one-time payment for perpetual use, rather than an annual payment for a limited subscription period.It is also a payment for an individual article, rather than an entire journal or package.In addition, the APC payment covers not only the institutional use of the library paying the fee, but global use.Since APCs then pay for global, perpetual use of individual articles, current standard COUNTER reports do not provide a measure of usage for OA APCs.How then might libraries measure an analogous cost per use for OA fund payments as they review and make the often difficult decisions of where to best apply their limited collections funding?Since usage at the article level is required for an APC cost per use measure, article level usage metrics provide a potential source for such data.Article level metrics (ALMs) track various types of activity at the level of the individual article.These metrics include traditional types of measures such as citations, as well as article views and downloads.Newer, non-traditional measures of activity at the article level include social media and similar web activity such as mentions on blogs, Twitter, activity in Mendeley, etc., however their application to library assessment of collections expenditures is not established.Views and downloads, on the other hand, are well-established as assessment measures in libraries but at the journal title level.Application of article views and downloads to study cost per use of open access APC payments has been lacking.The development of article level metrics now enables this topic to be investigated.This article presents and examines a method to measure the cost per use of sets of OA APCs, using article level downloads and views, and considers the current limitations of such a method.The purpose of this examination is to explore one measure for assessing an OA fund, funded from the library's collection budget, comparable to an established method for assessing traditional-model collections purchases.This paper therefore focuses on article views and downloads, rather than on other article level metrics.The purpose of this method is not to assess the value of individual funded articles, as these are expected to vary; this paper does not intend to propose that libraries should select the articles they fund based on an anticipated cost per use for that article.Neither does this article intend to compare specific APC models or publishers; the data used does not support such a comparison.As funding OA publication continues to evolve, processing individual OA APCs as micropayments may continue or may be supplanted in whole or in part by other, macro-level payment models, such as offsetting hybrid journal subscription payments based on amount of APCs paid to the same journal(s).This paper considers APC-model payments only.Different funding models may require different methods for assessment.
An initial question that may arise is: why attempt to measure cost per use of OA APCs?As mentioned, cost per use is only one point of consideration that libraries refer to when making decisions about where to direct collections funds.Libraries choose to fund OA APCs for various reasons, including to support campus authors or to support and promote open access (Yates et al., 2016).Financial efficiency is not necessarily the primary consideration, particularly in an environment of necessary experimentation of OA publishing models and pricing.In fact, financial efficiency is not necessarily the primary consideration for libraries as they evaluate traditional subscription model purchases either.For example, if a particular purchase is less heavily used and therefore relatively expensive in terms of cost per use, but is required for specific researchers or specific fields of study, a library may choose to retain it despite its high relative cost.However, libraries paying for OA publishing will seek at some point to review the value for money of the various expenditures they make.Calls for assessment of OA funds and questions about return on investment (ROI) are already beginning to arise (McMillan, O'Brien, & Young, 2016;Yates et al., 2016).Cost per use is one factor that can inform librarians as they consider their support for OA funds, particularly when being weighed against other collections purchases.Therefore, a method to measure cost per use for APCs is needed.
There are different models by which open access APCs are levied by publishers, such as article processing charges alone, discounted APCs based on institutional membership payment, or APC credits available to authors from institutions subscribing to the journal.This article discusses two examples to illustrate the development of an APC cost per use measure: (1) article processing charge alone (PLOS), and (2) membership with APC discount (BioMed Central).Each example illustrates the calculation of cost per use for one year's OA fund payments.While the available data used for this study is derived in part from actual examples, the results should not be taken as a fully accurate or representative cost per use of OA APCs generally speaking.Neither does the data, as it was available, constitute a valid comparison between the two publishers.The examples presented in this article are essentially illustrative.

LITERATURE REVIEW
The potential for libraries to use article level downloads and views as one way to assess OA APC payments has received limited attention in the literature.In an article in 2004, Holmström discussed the concept of using lifetime downloads by year of publication to measure return on investment for journal articles (2004b).As an example, he estimates the use of 1998 articles that form usage in subsequent years and presents such total usage over time as the ROI.(In this case, the ROI would be for the cost of the 1998 payment).In an earlier article the same year (2004a), he also proposed a method to measure a "cost per reading" (as opposed to cost per use) for articles.This cost per reading was based on the cost of the institution's one-year membership for full APC coverage for BioMed Central.This cost was divided by: that year's institutional PDF downloads and full-text article views, minus a factor to estimate the percentage of downloads that do not represent actual reading.His study was from the perspective the institution only therefore his calculations did not include usage outside of the institution, including by non-paying readers of the open access content.In the same article, he mentions a measure discussed by BioMed Central to relate "the article-processing fee to the number of times an author's specific article is downloaded, read or cited" (2004a, p. 7).This measure includes a broader range of usage than that typically used by libraries, which include views and downloads but not citation counts when measuring cost per use.More recently, Glushko, Hampson, Moore and Yates (2016) posed the idea of measuring cost per use for OA funds using the cost of the APC and the article level usage.This paper develops and expands on this idea.
A number of recent articles discuss the cost of open access APC payments made by libraries or other institutions.Kingsley (2014) outlined a wide variety of models of OA publishing, including both fully OA and hybrid journals, and discussed difficulties involved in gathering data on the amount collectively paid for APCs by grant funders, authors, libraries and their institutions.She noted that an APC publishing model shifts the cost of publishing towards high-research output institutions, a point studied in detail previously by Walters (2007).Reporting on part of a Jisc study related to OA APCs in UK higher education institutions, Lawson (2015) discussed the need for standardized data on APC payments and mentioned a template Jisc created with the goal of collecting data on APC payments from around the world.In a Jisc report, Shamash also argued for the importance of gathering common types of data regarding APCs, and for making such open as a way to assess APC expenditures and inform negotiations of offsetting agreements (2016, p. 21).
Studies of the amount paid at a national level for APCs have been conducted.Pinfield, Salter and Bath (2015) researched the cost of APCs, subscriptions and administration costs, to attempt to calculate the "total cost of publication" for UK higher education institutions.Building on the notion, Gray (2015) investigated the cost of additional other publication charges, such as page fees, color fees and submission charges, for their inclusion in the total cost of publication.Jahn and Tullney (2016) studied the national level of OA publishing fees paid by German universities and research organizations, with comparison to Austria and the UK, noting the high proportion of German publications in fully open access journals as opposed to hybrid journals (consistent with Germany's largest research funder's policy) and the higher APCs for hybrid journals.
Consistent with Walters, the University of California's "Pay it Forward" project concluded that, for research-intensive institutions in North America, the cost of paying APCs for their institution's research articles would be more than the cost of their library journal subscriptions (Smith et al., 2016).This project proposed grant funding as an additional source, and author-controlled funds as a way to influence market prices.Morrison (2013), considering library expenditures on scholarly journals globally, concluded that these funds are more than the cost for a fully open access journal system.Björk and Solomon (2014), for example, proposed ways research funders could contribute to paying APCs and also influence the OA APC market.
These studies appropriately focus on the macro level costs of funding scholarly journal publishing.A macro level solution may be preferred by the institutions involved in administering the payments, if it is fiscally transparent and administratively efficient compared to managing numerous micropayments.However, while the wider level solutions develop, individual libraries consider the options practically available to them for how to allocate their funds today.
Library assessment of OA funds is currently in its early stages, with several recent publications identifying or attempting to address a need for OA fund assessment.A 2016 survey of Association of Research Libraries (ARL) member institutions and OA funds identified a need for further research on a number of topics, including how to assess OA APC funds (McMillan et al., 2016, p. 7).The same year, a report from a study of OA funds in Canada (Yates et al., 2016) recommended using the measures collected by the SPARC's OA Funds in Action reports ("Open access funds in action," 2016), as well as the number of requests denied due to lack of funds.For reasons of practicality, it recommended tracking of altmetrics and citations only when specifically needed.This report recommended obtaining qualitative data from funded authors.It also outlined several libraries' decisions resulting from reconsideration of their OA funds, primarily initiated due to financial constraints.Qualitative assessment was undertaken by Beaubien, Garrison, and Way (2016).They surveyed authors who had received grants from their library's OA fund, to assess whether the fund was meeting its intended purposes.They also used similar quantitative measures to SPARC's reports, including number of authors, number of journals, author's department, and number of applications.In addition to the SPARC measures, Glushko et al. (2016) suggested several additional measures to report on distribution among publishers and journals.Consideration of APC cost and related usage has not been highlighted.
Much of the examination of article level metrics to date has focused on them as a means to measure or predict research impact, particularly in relation to citations.Early studies used log files from platforms to find article level activity.Pinkowitz, for example, noted the potential of article downloads for measuring research impact correlating them positively with citations (2002).However, a number of articles about ALMs outline some context and potential expectations for measuring usage associated with APC payments.Moed studied the relationship of article downloads and later citations, including the converse effect of increased downloads of articles after they had been cited ( 2005).This finding indicates that usage of a given article may be subject to increases at unpredictable points.In an opinion piece, Harnad and Brody made the point that measurement at the article level, not the journal level, is necessary to compare citation statistics on OA versus non-OA articles ( 2004), underscoring the value of metrics at the article level.Several studies of article level usage found that open access articles have higher levels of usage than non-OA articles.Two different randomized controlled trials found OA articles had higher levels of downloads, and a greater number of addresses downloading them, than non-OA articles (Davis, 2011;Davis, Lewenstein, Simon, Booth, & Connolly, 2008).Two other studies found that usage of OA articles is higher not only initially after publication but continuing over time (Wang, Liu, Mao, & Fang, 2015;Wang, Mao, Xu, & Zhang, 2014).From these studies, it can be anticipated that usage counts for APC funded articles will be relatively high compared to traditional-model publishing, particularly when considered cumulatively.Considered alongside research done by Huntington, Nicholas, Jamali, and Tenopir (2006) that found an article usage decline two to three years after publication and on through to eight to nine years after publication, it is evident that elapsed time will be an important factor in gathering usage data for OA articles.
More directly addressing library assessment, in an article focusing primarily on altmetrics, Galligan and Dyas-Correia noted the potential for Mendeley's data on usage, among other data, to complement COUNTER statistics in libraries' collection management considerations (2013, p. 58).Besides pointing to article level usage as a counterpart to COUNTER statistics, this statement points to sources of usage in addition to the publisher platform, particularly important when considering open access content which can be freely distributed.

METHODS
To illustrate the concept and examine the potential and the limitations of using article level usage metrics to calculate a cost per use of APCs, this article considers examples from two different publishing models: APC-only (PLOS) and membership plus APC (BioMed Central).The examples are from journals where OA APCs have been in use for many years, both publishers becoming established in the early 2000's ("BioMed Central," 2017; "Who We Are | PLOS," n.d.).Both platforms measure article activity and are COUNTER-compliant ("BioMed Central | Technical standards," n.d.; "Public Library of Science (PLOS)," n.d.).
The two platforms count "views" differently and the cost per use calculations in this paper therefore include both abstract-only views and full text article views.In the case of PLOS, the abstract and full text views are not counted separately, being on the same page.Abstracts are normally excluded from COUNTER reports ("COUNTER code of practice for e-resources.Release 4. Appendix D.," 2015).The BioMed Central data used for this study includes a separate count for abstract views, as these were available on the publisher's previous platform.However, they are combined with downloads and full text views in the usage calculations, for consistency.Abstracts and articles occur on the same page in other major platforms currently, such as PubMed Central (PMC), SpringerLink, ScienceDirect, Wiley Online Library, and the new BioMed Central platform.Therefore, views are presented similarly on several major platforms.The inclusion of abstract-only views may not affect comparison between such platforms, however libraries still need to consider this factor in their interpretation of the resulting cost per use calculations, as including abstract views as usage creates higher levels of use than platforms following a more precise application of COUNTER.
To allow time for usage to accumulate, statistics were gathered for articles published three years previously.This amount does not represent all anticipated lifetime usage, but it is consistent with Huntington et al.'s findings of a significant decline in usage two to three years after publication (2006, p. 1850).Usage includes PDF downloads, article views, and abstract views.Prices are in United States dollars.As previously mentioned, since the purpose of this article is to explore the method using available data, the costs per use calculated below should be taken as primarily illustrative rather than fully accurate and representative.

Example 1. Article Processing Charge Only: PLOS
PLOS is a major OA publisher using the APC-only model.In this example, the formula for calculating cost per use of individual article APCs would be the APC price divided by the cumulative global use of the associated article.However, an OA fund would be assessed not at the level of individual articles but by the aggregate cost and aggregate use, since libraries are interested in assessing the overall fund, not each individual article.Since libraries normally assess yearly costs, the formula for assessing the cost per use for one year's funded APCs would be calculated as the aggregate total cost of one year's funded APCs divided by the aggregate cumulative global use of the associated articles, or: Aggregate APC cost/Aggregate cumulative use PLOS has been making article level metrics available since 2009 ("About | ALM Reports," n.d.).Usage ("article accesses") can be viewed by searching individual articles.All available article level metrics can be accessed and exported using PLOS' ALM Reports tool ("ALM Reports," n.d.).To illustrate this example for the purpose of this paper, sets of articles published in 2013 in PLOS ONE (591 articles from a period of July 1-7, 2013) and PLOS Biology (165 articles from January 1 to December 31, 2013) were downloaded using ALM Reports.A large number of articles is used so as to give a reasonable representation of what a cost per use might be, even though this number of articles exceeds the annual capacity typical of North American library OA funds ("Open access funds in action," 2016).The examples from PLOS are intended only to illustrate the calculations; neither example is a particular library's set of articles.
ALM Reports provides views and PDF downloads from both PLOS and PubMed Central ("Track impact with ALMs | PLOS," n.d.).Usage data from each platform is presented and combined to form total use.Because the cost of the APC was not included among the data available in the report, the current 2016 advertised APC prices for PLOS ONE and for PLOS Biology ("Publication fees | PLOS," n.d.) were used as the APC for all articles in each journal.The APC is then divided by the use to give a cost per use after three years.

Example 2. Membership Plus Article Processing Charge: BioMed Central
In the BioMed Central (BMC) model of OA publishing, a library pays for an annual membership that provides institutional authors a discount on their APC.The amount of discount is based on the type of membership chosen.The APC may be paid directly by the authors, without further funding by the library.In this case, the cost to the library is that of the membership only.The usage would be that of all articles published under that year's membership.The library's cost per use would then be expressed as:

Membership cost/Cumulative global use of the associated articles
If the library also paid APCs, the library's cost per use would be calculated as: (Membership cost + APCs paid)/Cumulative global use of the associated articles Note that, in both cases, the formula represents the cost per use for the library's contribution (the library's cost per use).Unless the library paid the APC for all the articles published in the year under this model, it does not represent the complete cost per use of these articles, only the library's share of that amount.However, the same is true of traditional cost per use calculations: they are the library's cost per use, exclusive of institutional authors' costs (e.g.author-paid APCs in hybrid OA publications; or submission, page or color fees).
In this example, the set used was composed of the articles published in 2012 under the University of Saskatchewan's "supporter" membership.The data was provided by the publisher in 2015.Articles published in 2012 were chosen, to represent three years of usage.In reality, the APC would be paid at some point between submission date and actual publication date.Since this date is not known, publication date was used.For simplification, the institution's membership fee paid in 2012 was used as the membership for the 2012 calendar year, rather than prorating based on membership period.The APCs are the actual, discounted APCs paid for these articles.

PLOS
Tables 1 and 2 provide example results for each PLOS journal.PMC use is provided separately from PLOS use to illustrate examples of the level of use occurring on one source beyond the original publisher's platform.A range of examples is provided here, illustrating various possible costs per use.The analysis includes noting the highest, lowest, median, and average costs per use in each set.A library might perform similar analysis of their funded articles to discover whether their overall cost per use results are affected by extreme cases.However, to assess an OA fund, a library would not assess the cost per use of the individual articles.Instead, a library would primarily use the "aggregate" cost per use, that is, the aggregate APC cost and aggregate cumulative use of a year's funded articles.Of course, unlike these examples, a library's list of actual APC payments made under an APC-only model would typically be from more than one publisher.

BioMed Central
Table 3 shows results from the two possible cost per use scenarios noted above.The first scenario is for membership payment only, with usage from all articles published under one year's membership.In the second scenario, when including the APC payments, the assumption was made that the library paid the membership and all 17 APCs.The membership fee was not averaged among the articles but treated as a lump sum, demonstrating the cost per use calculation for the library's 2012 OA fund payments, i.e., the assessment of the OA fund, not the cost per use for any particular article.Note that in this example, usage is from the BMC platform only; PubMed Central usage is not included.Abstract views account for 18% of total use.
Several limitations should be noted regarding the data used in this study.As stated previously, for the PLOS examples the current advertised APCs were used as the APC price for all articles in each journal.This assumption does not take into account any potential APC waivers or any price changes since 2013.It therefore may represent a high estimate of the actual cost for these sets of articles.In the BMC example, PubMed Central usage was not available, therefore usage for the BMC articles is anticipated to be low compared to the PLOS examples.Additionally, the BMC APCs and membership fee are from 2012, rather than today's price as in the PLOS examples.For both of these reasons, it is therefore not possible to compare cost per use between these two publishers based on the above calculations.Furthermore, the BMC sample (17 publications), while sufficient to illustrate the cost per use model for this paper's purpose, is too small a number to be statistically representative of the cost per use of BMC publications generally.

Current Limitations Related to the Method
The examples demonstrate a method to calculate cost per use for OA funds that is simple in principle.However, several current limitations make it less simple in practice to obtain and collect the necessary data.
The Nature of Open Access.One limitation of calculating usage statistics for open access articles is the nature of open access itself.OA articles can be stored in multiple places including institutional repositories, ResearchGate, Mendeley, ingestion by other platforms such as aggregators, and freely shared by individuals.Counting downloads and views from the original publisher's platform is unlikely to account for all use.Therefore, it is important to keep in mind that the usage statistics from the publisher's platform likely under-represent actual total usage.This fact must then be taken into account when considering a calculation of cost per use based on data from a particular platform; the resulting cost per use is likely somewhat high.Obtaining precise usage statistics would require gathering and combining comparable statistics from all locations where the article is downloadable or viewable.Currently this data is not possible to obtain.However, to some degree, this challenge is also true for traditional subscription journals; articles published as green open access can also be held in a similar range of possible locations and use from these locations is not included in COUNTER statistics drawn from publisher platforms.Further studies comparing the amount of use of OA articles from the publisher's platform versus from other locations would inform considerations of how much use additional usage may exist.For example, the PubMed Central usage was 18% of the use in the PLOS ONE data sample used in this study.In PLOS Biology, it was 7%.However, while use counts of OA articles from publisher platforms may not be fully precise this does not mean they cannot be informative, recognizing they are to some degree incomplete.
Accessibility of Data.Not all platforms currently make article download and view data easily accessible.For example, this data is not displayed currently by PubMed Central, Wiley Online Library, or ScienceDirect.However, the data may be being captured even if not displayed.For example, PubMed Central provides article-level use data to PLOS.Platforms that show "most viewed" articles presumably base such measures on ALM data.This data may not be directly accessible by the library but it may be possible to request it from the publisher.Making article level metrics data available and readily accessible is important for all those interested in the data, including for library OA fund assessment.
Gathering the Data.If the data is not otherwise available from the publisher, it must be gathered article by article by the library.This may be possible for small numbers of articles though it would be impractical at any large scale.Ideally, usage would automatically be complied from multiple, primary platforms and repositories, based on DOI.This information could then be combined with APC paid in the library's records.Tools such as PLOS' ALM Reports make a full set of ALM data easier to obtain.However, even using the ALM Report, a library cannot download data on just the articles it paid for or identify these by a particular element in the result set.For example, author institutional affiliation does not necessarily align with who paid the APC.Articles may have multiple authors, from different institutions, or the APC may have been paid by grant or departmental funds and not from library funds.
Ease of gathering the data for APC payments is important, as usage would need to be regathered if the same articles are assessed again after more time has elapsed.This situation would occur, for example, if the library chose to re-assess its OA fund over a longer period to capture a fuller measure of usage, since usage is ongoing.Until gathering the data becomes easier, the library would ultimately be better served to establish a review period of a certain number of years and not re-assess the same payments even though usage continued to accumulate on those articles.For example, a library may use a review period of three years after publication for disciplines where that period was expected to represent the main use of the article, and then not further review those same articles.
COUNTER Definitions.It is important that article level usage metrics are standardized and comparable between platforms, both for OA and traditional purchase models.Platforms will still need to use COUNTER definitions.For platforms where abstract view statistics are no longer separable from downloads and full text views, libraries may consider revising and lowering their acceptable cost per use expectations compared to their former criteria.
Additional Considerations in the Membership Model.In the membership model, the number of articles submitted in a year varies.The price of the membership is therefore distributed over more or fewer articles in any given year.This effect should be kept in mind when considering the resulting cost per use.Also in this model, if the article is submitted directly by the author, the library may not be aware that the article was submitted and therefore not have a record of which articles' use to include.In the case of BioMed Central, the publisher can supply this information.However, currently the library cannot directly generate a re-port of the articles submitted under their annual membership, along with their cumulative use.
Time for Usage to Accumulate.A further limitation to this approach to cost per use calculation is timeliness of the data.Usage requires time to accumulate.To measure an actual amount for perpetual use is not possible, but a sufficient amount of time must be allowed to gather meaningful usage.This factor must be taken into consideration when reviewing cost per use.The length of time to wait may vary for different disciplines and would need to be established by the library for their institution.

Implications
The known effect of increased downloads for OA articles (Davis, 2011;Wang et al., 2015) means article level use counts will be higher than non-OA articles when calculating cost per use.Unlike the traditional publishing model where many institutions pay each for their own institution's use, for OA fund payments one institution pays for all use, including their own and that of other institutions and individuals.Additional use from such altruistic access is a new factor for libraries to consider when weighing the value of OA payments.Altruistic access is an inherent purpose of OA; unlike traditional library assessment, it is not only benefit to the local institution being weighed.Libraries will need to develop new benchmarks for what constitutes an acceptable cost per use for their institutions in cases where the usage and benefit of altruistic access is included.
The aggregate costs per use in each of the examples, after three years of usage, is: $0.35, $0.22, and $0.41, with the lowest cost per use in these samples approaching zero ($0.01).Three years may not yet represent all cumulative lifetime usage, therefore it is possible the cost per use for these samples would be lower if considered over a longer period.For traditional-model publishing, librarians have reported using much higher costs per use as thresholds at which they consider an expenditure for cancellation.For example, Currie and Morris report using $40 for cancellation and also used $20 or less for retention (2017).Scott and Eva found librarians describing thresholds of $20-25, or $5 at a large institution, for seriously considering cancellation (2017, p. 12).Others report such consideration at thresholds in the range of $10-$20 (Nash & McElfresh, 2016;Pedersen, Arcand, & Forbis, 2014) and $30 for one health sciences library (Fought, 2014).Comparing between very low costs per use, even costs per use near $0.00, becomes less useful to librarians.For example, the difference of six cents between two of the samples used in this paper may not represent a significant difference.If open access expenditures consistently result in very low costs per use, cost per use calculations may not be helpful for comparing between OA expenditure choices as the differences may not be significant.If that is the case, as libraries consider where to place their funding support, assessment between different options for OA expenditures would rely on criteria other than cost per use.However, considering the costs per use that libraries indicate as cancellation thresholds, low costs per use for OA funds may be one helpful point of consideration when assessing the OA fund relative to more traditional types of collection budget expenditures as the differences may be much larger.
The specific examples in this study suggest that OA APCs may compare favorably to traditional publishing when considering value for money based on cost per use.However, the data in this study should not be interpreted as a verification of such an argument, as this study was not designed to answer that question, nor can it do so given the limitations on the data.This paper was designed to present and illustrate a method.Further study would be necessary to verify or refute this possibility.

CONCLUSION
Article processing charges are a completely different model of paying for scholarly publications than traditional publishing models.As long as OA APC payments co-exist with payments for traditional-model publishing in the libraries' collection budgets, a comparable assessment method for APC payments is needed to inform libraries' decisions when distributing funds.The one-time cost of OA APCs should be measured against global, cumulative usage of the particular articles over time rather than the institutional, annual, journal title-level usage provided in standard COUNTER reports.Article level metrics provide potential for obtaining an indicative, though incomplete, measure of usage at the article level.This usage can be totaled and measured against the aggregate cost of the corresponding articles' total APC payments to calculate an aggregate cost per use, to assess a year's OA fund expenditure.Sufficient time must be allowed to elapse before gathering usage.Limitations exist to obtaining fully precise usage data for open access publications, due in part to the nature of open access and to the current accessibility of the data and the process to gather it.Increased, ready access to the relevant data is needed, including on more platforms.The ability to automatically gather and combine usage data from the primary locations where the articles are stored (i.e. the main platforms and institutional repositories), would make usage easier to gather and the data more complete.
Making article level usage data more accessible allows libraries to make better-informed decisions about directing their budget between traditional publishing and open access APCs.
Libraries will need to develop new benchmarks for acceptable cost per use where use includes altruistic access.If further research determines that OA APCs typically result in very low costs per use, assessment between different OA expenditures would need to be based on other considerations, though APC costs per use may still be useful for comparison with traditionalmodel subscriptions.The examples suggest that APCs may compare favorably to traditional publishing when compared based on cost per use, but further research is needed to determine if this possibility is true for OA APCs generally; this paper was not designed to address that question.Ultimately, cost per use remains only one factor libraries consider when making decisions about where to apply their collections funds.

Table 1 .
Cost per Use for 591 Articles Published in PLOS ONE 2013

Table 2 .
Cost per Use for 165 Articles Published in PLOS Biology, 2013

Table 3 .
Cost per Use for 17 BioMed Central Articles Plus Supporter Membership, 2012