Scholarly Communication Librarians ’ Relationship with Research Impact Indicators : An Analysis of a National Survey of Academic Librarians in the United States

Miles, R.A., Konkiel, S., & Sutton, S. (2018). Scholarly Communication Librarians’ Relationship with Research Impact Indicators: An Analysis of a National Survey of Academic Librarians in the United States. Journal of Librarianship and Scholarly Communication, 6(General Issue), eP2212. https://doi.org/10.7710/2162-3309.2212 Scholarly Communication Librarians’ Relationship with Research Impact Indicators: An Analysis of a National Survey of Academic Librarians in the United States


INTRODUCTION
Since the 1990s, scholarly communication has seen a dramatic shift away from an environment of print journals and books toward diverse, web-native research outputs, such as journal articles, eBooks, individual book chapters, data sets, source code, conference presentations, conference proceedings, and other nontraditional online outputs.As a result, an expanding and complex suite of metrics has evolved to measure research's many online indicators of impact, attention, and influence (often found in data from social media, news outlets, public policy, and other online sources, in addition to traditional citation data).While traditional, citation-based metrics, also known as bibliometrics, continue to provide valuable insights on the scholarly influence of research, a new class of indicator called altmetrics can highlight the online, nontraditional impacts of research in academia and beyond.
Within this changing environment, many academic libraries in the United States offer scholarly communication support services.Not only do librarians offer support for publishing, archiving, and sharing scholarship, but many also often provide support to faculty and administrators regarding the appropriate uses of metrics for assessing research impact.Sometimes, this support can take the form of helping faculty find and include impact metrics in their tenure and promotion dossiers and grant applications.Administrators also request support when compiling evidence of research impacts for their college, department, or institution.
To date, most studies on how librarians understand and use qualitative and quantitative measures of research impact (collectively referred to in this article as research impact indicators) tend to focus heavily on bibliometrics or, in the case of altmetrics, trend heavily toward the hypothetical.Librarians' use of research impact indicators often intersects with researchers' use of these metrics, with librarians providing essential support services and expertise.This study aims to explore librarians' self-reported perceptions and uses of indicators of research impact.
In this article, we share insights on a recent survey1 aimed at assessing the levels of familiarity with and uses of research impact indicators among the U.S. academic library community in universities with high levels of research activity.First, we describe the current known state of scholarly communication librarianship in the United States and the use of research impact indicators among librarians worldwide.We then describe the results of a national survey administered to academic librarians in the United States at Carnegie-classified "R1" (research-intensive) universities.We compare the responses of librarians who reported having scholarly communication support job duties to the larger population of respondents who did not report these duties.We then draw conclusions about the following research questions: • Are librarians with scholarly communication support duties more familiar with research impact indicators?
• Are librarians with scholarly communication support duties more likely to use research impact indicators?
• Do librarians with scholarly communication support duties have different reasons for using these indicators in their everyday practices and in their professional advancement?
• Are academic librarians at R1 institutions more likely to address citation-based research impact indicators, such as the JIF and citation counts, during one-on-one consultations with faculty?
• Are academic librarians at R1 institutions more likely to address research impact indicators during faculty consultations when addressing concerns regarding publishing issues or when addressing concerns related to tenure, promotion, and grants?

Scholarly Communication and Research Assessment Services
Scholarly communication is a diverse and thriving field in academic librarianship.Services in this field range from the highly technical to the more publicly engaging; scholarly communication librarians' skills also fall on this wide spectrum and can be a hybrid of both (NASIG, 2017).As early as 1993, librarians have been identified as being essential to leading changes in scholarly communication in the internet era (Lynch, 1993).Libraries and librarians have led the way in establishing a worldwide network of nonprofit institutional repositories (Bosc & Harnad, 2005;Lynch, 2003) and open access publishing infrastructure (Bosc & Harnad, 2005); offering consultation services and classes on publishing, open access, research impact indicators, research data management, and other scholarly communication issues (Corrall, Kennan, & Afzal, 2013;Lewis, Sarli, & Suiter, 2015); and funding and supporting massive infrastructures for digital preservation (Molinaro, 2016) and text and data mining (York, 2009).
Scholarly communication services are also staffed in a variety of ways.In some libraries, scholarly communication outreach is done primarily by liaison librarians (Malenfant, 2010;Radom, Feltner-Reichert, & Stringer-Stanback, 2012).One recent survey of ARL libraries found that 25% of library-based scholarly communication initiatives are managed by scholarly communication departments or offices, 23% by informal teams, 30% by individuals, and 16% by committees, with 6% of libraries reporting that "leadership was not associated with any single individual or group" (Radom et al., 2012).Scholarly communication librarians are often responsible for outreach on research impact indicators (Lewis et al., 2015;Miles, Sutton, & Konkiel, 2016); however, librarians of many backgrounds also have been known to do similar outreach (Lewis et al., 2015;Salisbury & Speer, 2016).
Within academia, citations are understood to be a useful way to understand the scholarly, long-term influence of research (Cronin, 1984;Leydesdorff, Bornmann, Comins, & Milojević, 2016).As such, some researchers use citation counts and citation-based metrics like the journal impact factor (JIF), "a measure of the frequency with which the 'average article' in a journal has been cited in a particular year or period" (Garfield, 1994), in job applications, promotion and tenure dossiers, grant applications, and other professional advancement scenarios (Abbott et al., 2010;Brand, 2013).However, the potential for misuse and misinterpretation of these metrics by researchers-in particular, the application of journal-level metrics to describe the influence of individual articles or to summarize a researcher's career-long impact-has caused librarians to advocate for their roles as experts in evaluating research impact (Cameron, 2005;Corrall et al., 2013;Nordfeldt, 2015).

Citation-Based Metrics
Researchers and university administrators use research impact metrics, most notably traditional citation-based metrics, for a variety of reasons, including for the purposes of assessing job applicants, for funding decisions, or for their own individual career advancement (Abbott et al., 2010), although a growing number of university administrators claim the use of qualitative measures, such as recommendation letters, as decisive measures of researchers' impact in their fields (Abbott et al., 2010;Zare, 2012).Originally heavily used in the sciences, in recent years research impact metrics have also begun to be applied to the arts, humanities, social sciences, and professional disciplines (Wilsdon et al., 2015).Although several studies have examined the use of research impact indicators by researchers, faculty members, and university administrators, most of these studies focused on citation-based metrics and not necessarily on how academic librarians use a diverse set of research impact indicators (Corrall, Kennan, & Afzal, n.d.;Malone & Burke, 2016).
In addition to tenure and promotion guidelines, faculty often struggle to obtain highly competitive grants.Such grants may require that faculty report authorship in prestigious journals and/or citation-based metrics affiliated with their publications (Carpenter, Cone, & Sarli, 2014;Piwowar, 2013).Thus, while research impact metrics may not always be reported in tenure and promotion dossiers, grants and grant renewals are likely reported in dossiers and generally require some form of evidence of impact.
Much debate and controversy has emerged in recent years related to the misuse of citation-based metrics.Most notably, the San Francisco Declaration on Research Assessment (DORA) calls for scientists, university administrators, and funders to discontinue using the JIF to assess the quality and impact of a researcher's work, especially in hiring, promotion, and funding decisions (ASCB, 2012).
In addition, citation-based metrics can offer an indicator of a scholar's merit in conjunction with other research assessment methods, but the use of these citation-based metrics has probable bias tendencies (O'Meara, 2014), and the "Matthew Effect"2 suggests a recycling of citations among advantaged scholars (Rushforth & de Rijcke, 2015).Many of these citation-based methods and research assessment methods are based largely on prestige (Stephan, 2012), and often, other forms of evidence are marginalized or overlooked (O'Meara, 2014).

Usage Statistics
Usage statistics are another class of metrics that are often used both in libraries and among researchers to understand the influence and reach of research.The term usage statistics often refers to the number of page views and downloads that a research output receives online.
Other types of usage statistics include search queries, clicks, and requests for access (Konkiel, 2013).Librarians often use usage statistics to evaluate journals' use (Welker, 2012), but usage statistics can be applied at the article level as well.The Journal Usage Factor (JUF), a journal-level aggregate metric based on article download data and provided by COUNTER (a nonprofit that "provides the Code of Practice that enables publishers and vendors to report usage of their electronic resources in a consistent way" 3 ), was proposed as an alternative to the JIF, especially for collection development purposes like making purchasing, retention, and cancellation decisions (Shepherd, 2007).However, the JUF is not yet offered by publishers nor library vendors that aggregate other usage statistics.

Altmetrics
Altmetrics is a class of indicators and related qualitative data that measures the volume and nature of attention that an article, book, or other research output receives online.There is no agreed-upon definition for altmetrics. 4Altmetrics is generally understood to be a group of ever-changing indicators sourced from the social web that showcase interest in research from diverse stakeholder groups.Social media, news articles, peer reviews, and policy documents are types of sources where one can find altmetrics data.Applying not only to articles and books, but also to software, data sets, presentations, videos, and other nontraditional scholarly outputs, altmetrics can help researchers and librarians understand how communities outside of academia respond to current research, and how often research is being discussed online overall.
Altmetrics data is often produced immediately, whereas citation counts can take months or years to accumulate.Despite the advantages of altmetrics, there is some hesitation among the academic community in using and relying on social media in their professional activities (Weller, 2015) often due to perceived "silliness" of online engagement activities (Colquhoun & Plested, 2014) or concerns about gaming for social media metrics (Holmberg, 2014).Therefore, it can be difficult to convince faculty of the usefulness of such indicators.
In addition, there is a push to move in the direction of more robust research assessment for grant application requirements.While no known grant applications exist that require or encourage the inclusion of altmetrics, there are methods that applicants can employ for including such nontraditional measures of impact (Carpenter et al., 2014).Certain research grant funding agencies, such as the National Science Foundation, are moving in the direction of evaluating research impact in a more multifaceted way with a variety of metrics and qualitative measures (Cozzens, 1997;Engel-Cox, Van Houten, Phelps, & Rose, 2008), but as of yet, there are no known policies that require or encourage the inclusion of altmetrics (Piwowar, 2013).
Librarians have taken an interest in altmetrics since not long after the term was coined.
There is not yet information on how widely known altmetrics is in general among academic librarians worldwide, but a few national-scale studies have been published to date.Awareness of altmetrics among Spanish academic librarians hovers around 50%, with few respondents able to confirm whether or not their library offers altmetrics training for researchers (González-Fernández-Villavicencio, Domínguez-Aroca, Calderón-Rehecho, & García-Hernández, 2015).In a survey administered to university libraries throughout Sweden, LIS scholars and doctoral students responded that altmetrics tools are fun but not useful for research evaluation; surveyed librarians responded that altmetrics were complementary to traditional citation-based metrics (Nordfeldt, 2015).In the United States, a small study of academic librarians found that there is "a dearth of knowledge among academic librarians in Oklahoma about altmetrics tools" (Malone & Burke, 2016).
In reality, academic libraries tend to provide research impact metric consultation services or documentation for researchers, who compile these metrics themselves (González-Fernández-Villavicencio et al., 2015;Lewis et al., 2015;Nordfeldt, 2015).In the United States, 79% of respondents to an Association of Research Libraries (ARL) survey who offer library-based, scholarly output assessment training say they include altmetrics in their sessions, and reportedly most respondents provide research impact assessment services to their patrons (Lewis et al., 2015).In Australia, one study found that of the 36 university libraries that provide written guidance on research impact metrics, "24 (67%) mentioned altmetric indicators; 12 (33%) libraries did not provide information about altmetrics" (Mamtora & Haddow, 2015).
In terms of other library-specific applications of altmetrics beyond institutional repositories, there are only a handful of instances of librarians using altmetrics to supplement their collection management processes (Konkiel, 2016), track the influence of digital libraries (Williams, 2015), or manage library-based journals (Deliyannides, 2016).Several LIS scholars have suggested that altmetrics will influence the changing role of the academic librarian (Galligan & Dyas-Correia, 2013;Konkiel, 2013;Konkiel & Scherer, 2013), but more study is needed to consider the current practices among academic librarians in the United States in their application of research impact indicators.The current body of research on the use of research impact indicators in academic libraries is limited and contains gaps; thus, our research team set out to answer questions related to how and why academic librarians are using research impact indicators in their daily practice and, in the case of tenure-track librarians, for their own professional advancement.

Population Sample
Our study analyzed data from a national survey conducted during 2015, which was administered to 13,436 full-time academic librarians from 150 institutions included in the 2013 National Center for Education Statistics (NCES) list of Carnegie-classified R1 institutions in the United States.Due to limited resources and time constraints in collecting email addresses, this study's focus was limited to academic librarians at R1 universities.Furthermore, our research team deemed that responses from academic librarians at R1 institutions would offer valuable insight into how universities with the highest levels of research activity in the United States use research impact indicators.

Survey Distribution
Email addresses were collected manually in order to filter out certain positions not considered professional in nature (e.g., student workers, graduate assistants, and library assistants), and the survey was designed to eliminate responses from outside of the United States, as well as part-time, retired, and other non-full-time respondents.The Qualtrics-administered survey was sent out three times over a two-week period in 2015, resulting in a 5.3% response rate (n=707, valid responses counted only).

Survey Design
The survey consisted of 31 questions that included questions aimed at determining respondents' demographics (including job duties, years on the job, and other characteristics); their levels of familiarity with various research impact indicators; how respondents used these metrics in their own professional advancement scenarios (e.g., when applying for jobs, going up for tenure, etc.); and how respondents used these metrics in the context of their library job duties.
For questions that asked about familiarity with research impact indicators, respondents were asked to rate their familiarity on a Likert scale between one and five, with one indicating "I know nothing" and five indicating "I'm an expert."In addition, respondents were asked to rate their frequency of use and discussion of various metrics on a Likert scale, with one indicating "Never" and five indicating "Very Often."Journal of Librarianship and Scholarly Communication The survey addressed familiarity and usage of the following types of research impact metrics and related data:

Author h-index
Reflects an author's total number of publications that have received at least as many citations each (e.g., a researcher with an h-index of 25 has authored at least 25 journal articles that have received 25 citations).

Expert peer reviews
Post-publication peer reviews (e.g., F1000, Publons, resource reviews published in library journals).A type of altmetrics.
Qualitative measures of impact "Who's saying what about research?"(e.g., context in a citation from a scholarly article or a public policy document).
The study also sought to identify the extent to which respondents supported researchers in their use and understanding of various metrics, including • Helping faculty understand impact indicators • Conducting one-on-one consultations with faculty members to address tenure, promotion, and grants • Conducting one-on-one consultations with faculty members to address publishing issues • Compiling evidence of research impact for a faculty member or administrator Other types of research impact metric usage were addressed in the survey, such as using metrics for collection development or information literacy sessions.Some of these results have been reported elsewhere (Sutton, Miles, & Konkiel, 2017).

Data Analysis
The resulting data from the Likert-scale responses provided us with categorical data, and thus nonparametric tests like the chi-square test of independence were used to identify relationships between variables.Statistically significant relationships, when identified, helped us understand answers to our research questions, such as whether having scholarly communication support duties affects levels of familiarity and frequencies of use, and whether the type of issues addressed during faculty consultations affected the likelihood of certain research impact indicators being addressed.
In addition to scholarly communication, collection development, assessment, reference, and instruction duties could also be selected on the survey.Multiple answers could be selected for these regularly reported duties, which meant that the duties could not be compared to one another; thus, these variables were not mutually exclusive.Instead, in order to meet the assumptions of the chi square test of independence, those who reported scholarly communication duties were compared to those who did not report those duties.
Data analysis was conducted using Excel and SPSS.Data was initially imported from Qualtrics into SPSS, and then transferred from SPSS to Excel.Excel was used to produce data visualizations to spot trends in the data.When the data visualizations in Excel indicated significant results, then the chi-square test of independence was conducted in SPSS on subsets of data to determine statistically significant relationships, if any.When the tests showed statistically significant relationships, these results supported our inferred assumptions based on the descriptive data and data visualizations produced by Excel.
These results helped us determine, with statistical certainty, the relationships between those who reported scholarly communication support duties and those who did not report them regarding familiarity levels with different research impact indicators and the likelihood of using the JIF for specific purposes.The results also helped us to determine the relationships between two types of faculty consultations when addressing specific concerns (publishing issues versus tenure, promotion, and grants) regarding the likelihood of addressing specific research impact indicators.The descriptive data helped us to determine the nature of the relationships as either negatively or positively correlative; although the statistically significant results indicate that the relationships did not likely occur due to chance, the direction of the correlative relationships was determined using data visualizations and cross-tabulation comparisons with percentages.

Demographics
The pool of respondents consisted of academic librarians from R1 institutions.Over 80% of the survey respondents had been in the job for more than five years, with over a third having more than 20 years' experience.Less than one percent had less than one year of experience as an academic librarian.Overall, the respondents were considered to be experienced academic librarians.
Respondents were asked to choose from a list of regular duties and occasional duties, with regular duties defined as performing the duty once a month or more and occasional duties defined as performing the duty once every other month or less.The results presented here include an analysis of respondents' regularly performed duties. 5Occasionally performed duties are not addressed in this paper.Respondents could select more than one regular duty.Thirtynine percent of respondents reported regularly performing scholarly communication support duties.

Familiarity with Research Impact Indicators
Respondents were asked to report their level of familiarity with the JIF, citation counts, usage statistics, and altmetrics.Respondents were more likely to select "5 -I'm an expert" when assessing their familiarity of both citation counts and usage statistics compared to their selfreported familiarity with JIF and altmetrics (95% CIs [12.93, 19.07] and [12.01, 17.99], respectively).In addition, respondents were more likely to select "5 -I'm an expert" for JIFs than altmetrics (95% CIs [8.47, 13.53] and [2.35, 5.65], respectively) (Figure 1).

Regular Scholarly Communication Support Duties and Familiarity
Overall, librarians who reported regularly performing scholarly communication support duties had more familiarity with journal impact factors, citation counts, usage statistics, and altmetrics compared to those who did not report regularly performing these duties.Respondents with regular scholarly communication support duties replied "5 -I'm an expert" for all four categories more often than those who did not report those duties (Table 2).

Use of Metrics
This section will review our respondents' answers to questions related to their use of research impact indicators and present data analyses of these responses; such responses include the reasons why academic librarians use the JIF, the frequency with which certain indicators are addressed during one-on-one consultations with faculty, and the use of research impact indicators by academic librarians for their own professional advancement.

Use of the Journal Impact Factor
We asked our survey participants why they chose to use the JIF, and by far, the most popular response was "to evaluate the quality of a journal."The least popular response was "to determine which journals to read."The other responses included "to decide whether or not to purchase a journal subscription," "to recommend journals to faculty or students," and "to determine where to publish."The responses were limited to multiple-choice options and did not include a text box for additional responses (Figure 2).
The descriptive data related to uses of the JIF by respondents who reported performing regular scholarly communication support duties was compared to those who did not report regularly performing these duties.Overall, respondents with regular scholarly communication duties were more likely to use the JIF for the given reasons than respondents who did not report these duties (Figure 2 and Table 3).

Addressing Research Impact for Publishing Issues
Participants in the study provided responses for how frequently they address certain indicators of research impact with faculty members during one-on-one consultations concerning publishing issues.A five-point Likert scale was used to measure the respondents' level of use of six different measures of research impact in these consultations: the JIF, citation counts,  Have you ever used journal impact factors for any of the following purposes?(all respondents, n = 707) h-index, qualitative measures, altmetrics, and expert peer reviews in this situation, with "1 -Never," and "5 -Very Often."We used the chi-square test of independence to test for statistically significant relationships among the six research impact measures.

Addressing Research Impact for Tenure, Promotion, and Grants
Respondents also indicated how frequently they address certain indicators of research impact with faculty members during one-on-one consultations on the topic of measuring and understanding research impact for tenure, promotion, and grants.Overall, all the research impact indicators were addressed more often during these consultations than during the consultations concerning publishing issues.The relationships between the variables for the two types of consultations will be addressed in the next section.
As with the consultations concerning publishing issues, there was no statistically significant relationship between addressing the JIF and addressing citation counts during one-on-one consultations with faculty related to measuring and understanding research impact for tenure, promotion, and grants (χ 2 (4, n = 566) = 2.007, p = .734),indicating that both the JIF and citation counts are given similar degrees of attention and emphasis during one-on-one consultations with faculty.

Comparing Consultations
There were several statistically significant relationships between the two types of one-on-one consultations with faculty.These affect the likelihood of discussing certain types of research impact indicators, which depended on whether respondents addressed issues concerning publishing or issues concerning tenure, promotion, and grants.

Use of Metrics for Professional Advancement
Respondents were also asked about their use of indicators of research impact for their own professional advancement.When asked about inclusion of these indicators in their tenure and promotion dossiers, respondents could select any of the following: JIF, citation counts, qualitative measures of impact, usage statistics, and altmetrics.Two thirds of respondents (60%) answered that they included citation counts in their dossiers, and almost three quarters (69.23%) also responded that they use citation counts as an indicator of research impact when evaluating a colleague's work for tenure and promotion.In addition, the inclusion of any indicator of research impact is more prevalent when evaluating a colleague's dossier than when compiling evidence for one's own dossier.8 There were also differences in respondents' previous use of research impact indicators in their professional advancement and their intention to use indicators of research impact in the future.The percentage of respondents who answered that they intended to include research impact indicators in their tenure and promotion dossiers was far greater than those who had previously included those same indicators in their dossiers, especially for altmetrics. 9 Respondents were also asked about which research impact indicators they track for their own publications and were allowed to select multiple answers.We identified differences between respondents who reported performing regular scholarly communication support duties and those who did not in the type of research impact indicators they tracked.By far, those who reported scholarly communication support duties were far more likely to track any of the listed research impact indicators (Figure 3).Among all our respondents, the most commonly tracked research impact indicator was citation counts, followed by downloads and page view counts, qualitative evidence of impact, JIF, and altmetrics, respectively.
Finally, respondents also indicated reasons for tracking indicators of research impact.Again, they were allowed to select multiple answers.

Familiarity with Research Impact Indicators
Based on this study's findings and the descriptive data, academic librarians at Carnegieclassified R1 institutions are likely to have much more familiarity with the JIF than altmetrics as measures of research impact; they are also far more likely to be familiar with citation counts and usage statistics than the JIF and altmetrics (Figure 1).Our results also show that academic librarians at R1 institutions with regular scholarly communication support duties are more likely to have statistically significantly higher levels of familiarity with all four of these metrics than those who do not report scholarly communication support duties (Table 2).

Reasons for Using the JIF
Our data indicates that librarians who report regular scholarly communication support duties are more likely to use the JIF than those who do not report those duties for the listed  purposes, 10 with "evaluating the quality of a journal," "recommending journals to faculty or students," and "determining where to publish" the most popular answers, respectively.This data may simply indicate that librarians who are not in the field of scholarly communication are less familiar with the JIF and thus are less likely to use the JIF for any of the listed purposes.It could also indicate that librarians in the field of scholarly communication are more likely to use the JIF for the listed purposes, possibly because they are more aware of current trends and methods in academia for using the JIF.
Our survey did not ask respondents if they used other methods beyond the JIF for any of the listed purposes.For example, 64.29% of respondents who reported scholarly communication support duties answered that they use the JIF to evaluate the quality of a journal, but it is unclear if they use additional methods to evaluate the journal's quality, so these results should be interpreted with care.
Nevertheless, many of these reported purposes are still concerning.For example, more than half (52.04%) of respondents with scholarly communication support duties responded that they use the JIF to "recommend journals to faculty or students."The JIF was originally intended for use as a bibliographic discovery tool for scholars (Garfield, 2006), and later it was suggested that it be used as a tool for making collection development decisions in libraries (Garfield, 1999).In recent decades in academia, it has become a tool to indicate a journal's value and quality, thus acting as a swift method for evaluating the quality of research, but it is also a controversial one when used exclusively for evaluating an individual article or its author (Kurmis, 2003;Seglen, 1997;Smith, 2006).

Addressing Research Impact Indicators with Faculty
Our data indicates that all survey respondents address the JIF and citation counts far more often than the h-index, qualitative measures, altmetrics, and expert peer reviews during one-on-one consultations with faculty members.While altmetrics was not addressed nearly as frequently as the JIF or citation counts, it was addressed as frequently, if not more than, both the h-index and qualitative measures.Expert peer reviews were the least likely type of indicator to be addressed during consultations with faculty.
The type of consultation affected the frequency with which certain research impact indicators were addressed.Respondents were much more likely to discuss the JIF, citation counts, h-index, and qualitative measures of research impact when addressing faculty concerns re-lated to tenure, promotion, and grants versus addressing concerns related solely to publishing; citation counts had the most significantly statistical difference, followed by the JIF and h-index (citation counts, JIF, and h-index: p < .01);and qualitative measures had a smaller statistically significant difference (p < .05).In addition, when discussing expert peer reviews or altmetrics, the type of consultation made no statistically significant difference on the likelihood of discussing them.
The latter finding indicates that issues concerning tenure, promotion, and grants do not yet influence faculty members' interest in altmetrics or expert peer reviews, which suggests that tenure and promotion guidelines and grant applications do not yet recommend or require the inclusion of altmetrics or expert peer reviews.The opposite is true for citation counts, JIF, author h-index, and to some extent, qualitative measures, which suggests that there are explicit criteria requirements in tenure and promotion guidelines and in grant applications for the inclusion of citation counts, JIF, and the h-index.

Influence of Tenure and Promotion Guidelines on Publishing Behavior at R1 Institutions
These results may provide a distant snapshot of the influence of the processes that govern tenure, promotion, and grant applications on faculty publishing behavior and on academic librarians' engagement with faculty.Unfortunately, the survey did not ask respondents who initially mentioned the topic of these research impact indicators-the librarian or the faculty member.Therefore, it is not clear whether the survey respondents introduced specific research impact indicator(s) before or during the consultations, if the faculty member asked about them initially, or a combination of both.
Specific requirements in tenure and promotion guidelines, such as a requirement to publish in a high IF journal, may explain the frequency differences in addressing research impact indicators between the two types of consultations. 11Tenure and promotion guidelines and their requirements are likely the driving force behind the interest in research impact indicators among university faculty (Abbott et al., 2010), which is further supported by our findings.
Though no studies have analyzed massive amounts of tenure and promotion guidelines, it is doubtful that many guidelines address the inclusion of research impact indicators beyond citation-based metrics.Further research is needed to investigate the prevalence of specific types of research impact indicators and other forms of evidence included in tenure and promotion guidelines.For instance, it would be useful to know the trends across universities and disciplines that have one-dimensional requirements for attaining tenure and promotion, such as a specific number of publications, publication in high IF journals, and/or publication in specified journals.

Controversies and Suggestions for Further Research
Evaluating an individual article and its author based solely on a journal-based metric, like the JIF, is currently a hotly debated topic in academic and scientific communities (e.g., the Declaration on Research Assessment [DORA]), yet at least one study shows that most scientists perceive the JIF as important or useful in the evaluation of scholarly performance (Buela-Casal & Zych, 2012).However, most bibliometricians agree that the JIF should not be used to evaluate individual articles and their authors, at least not on a practical level, though there is some evidence to suggest that it can be used to assess an individual article on a statistical level (Waltman & Traag, 2017).
Unfortunately, the topic of how research evaluation influences actual academic and scientific processes has largely been neglected so far (Rushforth & de Rijcke, 2015).Research in this area could reveal whether the evaluation and assessment processes that largely govern academic careers influence scientists and other researchers' work.Here are questions worth considering for future research: • Does a specific requirement, such as publishing in a high IF journal to attain tenure and promotion, incentivize researchers to change the focus or topic of their research?If so, does this compromise or improve scholarly and scientific communication?
• Alternatively, do such requirements motivate researchers to work harder toward their goals?

Librarian Professional Advancement
Respondents differed on their likelihood to use research impact indicators for professional advancement, such as in tenure and promotion dossiers.The likelihood depended on a number of variables, including their intention to include them, their past inclusion of them, and whether they used them to evaluate themselves or their colleagues.
Our respondents were more likely to use research impact indicators to evaluate a colleague's dossier than to include them in their own dossier for tenure and promotion.For example, 47.86% of respondents indicated that they would use the JIF when evaluating a colleague's dossier, while 35.65% of respondents indicated that they included the JIF as evidence of research impact in their own dossiers.For altmetrics, 17.09% indicated they would use them when evaluating their colleague's dossier, while 8.70% would include them in their own dossier.This finding could simply illustrate that tenured faculty librarians are more knowledgeable of the tenure and promotion process and actively challenge tenure-track faculty librarians to boost their academic performance. 12n addition, respondents' intentions differed from their past actions.Respondents were more likely to intend to include research impact indicators in their dossiers than to have previously included them.For example, a much greater percentage of respondents answered that they intended to include altmetrics in their dossiers (45.10%), while only a small percentage indicated that they had previously included them (8.70%).This may indicate a growing interest in altmetrics among academic librarians.Alternatively, this finding may reflect motivations and ambition among tenure-track academic librarians to demonstrate research impact.Altmetrics is still a relatively new player in the field of research impact indicators; therefore, it makes sense that fewer respondents had previously included altmetrics in their tenure and promotion dossiers. 13  Finally, respondents who reported regular scholarly communication support duties were more likely to track research impact indicators than those who did not report these duties, suggesting that librarians with regular scholarly communication duties are more likely to use these indicators for their own professional advancement in addition to their regular job duties.It is not surprising that librarians with scholarly communication duties are more likely to track research impact indicators in their own published research.After all, this group of respondents is significantly more familiar with research impact indicators and is more likely to use them in their daily practices.

Limitations
Respondents were allowed to select multiple regular duties on the survey, and since the chisquare test assumes mutually exclusive variables, regular duties could not be compared to one another.Thus, we compared respondents who reported performing regular scholarly communication support duties to those who did not report regularly performing those du-ties.While these comparisons resulted in the identification of several statistically significant relationships, it cannot be asserted with complete certainty that not reporting regularly performing scholarly communication support duties is truly indicative of not having regular scholarly communication support duties.
In many of the survey questions, respondents could select one of five options to indicate their frequency of use or level of familiarity on a Likert-scale, which resulted in categorical variables.Categorical variables allow for the identification of statistically significant relationships, not the nature of those relationships, so the direction of the correlative relationships was inferred using cross-tabulation comparisons with percentages and stacked bar graphs.As a result, we were unable to determine the direction of these correlative relationships with statistical certainty.Finally, the survey did not seek to identify respondents' motivation for their use of research impact indicators.For example, we were able to determine what research impact indicators were addressed during faculty consultations, but we were unable to ascertain who initially requested the consultation, who brought up the specific research impact indicators, or any additional reason(s) (beyond the type of consultations) for addressing the specified indicators.It is possible that faculty members requested to discuss citation counts and the JIF, for example, rather than respondents bringing them to the faculty members' attention during consultations.It would also be worth investigating faculty members and administrators' underlying motivations for learning about specific indicators and assessing research; the results do suggest that tenure and promotion are major external motivators for assessing research and specifically for learning about the JIF and citation counts.

CONCLUSION
One of the core areas of the field of scholarly communication is research assessment and impact metrics (NASIG, 2017); this area demands consistent training and expertise as it continues to shift with the changing scholarly publishing and scholarly communication landscape.The trends in our data indicate that scholarly communication librarians are using their cultivated expertise in research assessment and impact metrics to assist faculty members and to enhance their own tenure and promotion dossiers.
Our data also indicate that librarians with scholarly communication support duties are using the JIF and other citation-based metrics for traditional purposes, such as for evaluating the quality of a journal; they also frequently discuss these metrics with faculty members during one-on-one consultations.While citation-based metrics are valuable when used responsibly, they can be harmful to the academic community when they are misused or misinterpreted.For example, using the author h-index as the only method to assess a scholar's merit can be a particularly misleading evaluation method.The h-index disadvantages early-career academics and academics in disciplines with lower citation rates (e.g., the humanities).After all, as Gadd (2018) notes, "metrics aren't bad in and of themselves, it's what we do with them that can make them dangerous." Making the transition to a healthier research assessment model requires that academic librarians continue to educate themselves on the vast number of quantitative and qualitative measures of research impact while also practicing how to use them appropriately in different contexts and settings.Some of our results indicate that our respondents may be willing to use specific research impact indicators to make or recommend overly simplistic assessments.
While research impact indicators can help in the research assessment process, using them as a proxy for actual influence or impact on the public or even on scientific communities can be misleading.
Librarian education and training may be key to promoting the responsible use of research impact indicators in American research institutions, but librarians in this field must take action to make this happen.Research impact workshops, 14 conferences, 15 open online courses, 16 and informative books 17 have started to emerge in the United States.Increasing educational opportunities and other collaborative efforts in library communities could eventually change how academics approach research assessment and potentially shift the entire field of scholarly communication to a more evolved system.

Figure 3 .
Figure 3. Types of research impact indicators tracked for publications

Research Impact Metric Reported regular scholarly communication duties Did not report regular scholarly communication support duties
Figure 1.Familiarity with indicators of research impact

Table 3 .
Reasons for using journal impact factors The majority of respondents (78.20%) revealed that they track this information "to satisfy [their] own curiosity."Almost half use indicators of research impact for annual performance reviews while about 43% use research impact indicators for tenure and promotion.Only about 20% use indicators of research impact to gain administrative support for work or research, and about 5 % use this information for grant applications.
(Phan, Hardesty, & Hug, 2014)imited to academic librarians at R1 institutions, or those institutions with the highest levels of research activity.While this study offers valuable insight into academic librarians' familiarity and practices with research impact indicators at R1 institutions, librarians from other types of institutions were not included in our sample.It would be useful in a future research project to determine familiarity levels and uses of research impact indicators among academic librarians at R2 and R3 institutions, community colleges, and other types of research institutions and special libraries, such as health and law libraries.Further research is needed.Another limitation is the relatively small sample size combined with our inability to compare our responses to librarian demographic information compiled by the National Center for Education Statistics (NCES)(Phan, Hardesty, & Hug, 2014), which would have helped us confirm the extent to which our respondents mirrored the field.An obvious limitation to the generalizability of this study is that the sample contained primarily librarians working in the United States.Due to differing academic cultures in nations worldwide, it is possible that academic librarians in other countries may have entirely different levels of familiarity with research impact metrics and may use them in different ways-or not at all-in their own professional advancement scenarios.Further research is needed here as well.In addition, there is no current data available regarding the number of librarians with scholarly communication support duties at R1 institutions.If we had sampled a population beyond those at R1 institutions, there is also no general data available regarding the number of librarians in the field of scholarly communication.It would also be useful to know where scholarly communication librarians are most likely to work.Based solely on levels of research activity, R1 and/or the Association of Research Libraries (ARL) member institutions probably have the highest number of such positions.
Figure 2a.Frequency of Addressing Research Impact Indicators: Comparison between One-on-One Consultations with Faculty concerning Publishing Issues and Consultations concerning Tenure, Promotions, and Grants Previous Inclusion of versus the Intent to Include Research Impact Indicators in Librarian Tenure and Promotion Dossiers