How librarians use bibliometrics to support leadership

What federal libraries can learn from NOAA

NOAA Central Library
NOAA Central Library in Silver Spring, Maryland. Photo taken by Sarah Davis & Jamie Roberts.

As federal libraries continually search for ways to demonstrate their value, bibliometrics has emerged as a way to quantitatively assess the impact of federally funded and produced research. Bibliometrics is essentially the statistical analysis of written publications, such as books, reports and journal articles.

Bibliometrics practitioners—increasingly, librarians—apply repeatable methodologies, such as citation analysis, to answer questions such as:  where should the organization invest in research, who is producing high-impact research, and where is collaboration occurring? For federal agencies that fund research efforts, these are important data points to take into consideration when making grant decisions.

In a recent webinar, LAC Group librarians at the National Oceanic and Atmospheric Administration (NOAA) discussed how bibliometrics and the use of Clarivate Analytics’ InCites product is making it possible to add a quantitative dimension to research decisions and provide additional value to individual researchers.

InCites is an analytics tool that incorporates the Web of Science (WoS) Core Collection of citations. With over 1.4 billion citations from 20,000+ journals, WoS is a curated and comprehensive index that allows users to provide accurate benchmarking and analyses on references made to journal articles, conferences and books.

Bibliometrics is growing as a competency for librarians and is enabling them to play a more strategic role in funding agencies. Particularly as budgets shrink and traditional librarian roles have changed dramatically, the use of citation analysis to inform funding decisions is a growing area of value for federal libraries.

The growth of bibliometrics at NOAA

Sarah Davis and Jamie Roberts are two members of the bibliometrics team at the NOAA Central Library in Silver Spring, Maryland. The Central Library is one of 20 NOAA libraries and has a unique role in providing services to all the branches as well as to other federal agencies and the public at large.

The team got its start in 2012 with a mandate to identify and report on all articles authored by NOAA researchers. It is mainly focused on the institution and looks at what NOAA is doing to contribute to other areas of scholarship and academic dialogue.

At first, the work was painstakingly manual as the team personally extracted and reviewed every article citation in the WoS that was authored or funded by NOAA. They would add their own metadata to the citations, and then use the data to produce annual and quarterly reports to senior staff. The reports focused on certain factors, such as a researcher’s or team’s productivity and collaboration. This work also yielded a public bibliography with a searchable list of articles. Over time, the bibliography was enhanced to enable portfolio reviews, metrics and data visualizations for presentations.  

Publication metrics for NOAA authored peer-reviewed journal articles published in Fiscal Year 2017. All data from Web of Science and accurate as of January 26, 2018. Graph created by Sarah Davis & Jamie Roberts.

The process laid out above is still in use by the team today and will not be discontinued anytime soon.  However, the group also uses InCites as an end-to-end analytics tool for a more automated approach to the WoS data set and other citation indexes. InCites enables the team to perform large-scale analysis and calculations on citations, as well as compare NOAA’s output to that of other agencies and parse the data according to various factors, including their in-house metadata. This has allowed the team to do similar projects without the effort of looking at every article and provide services at a faster turnaround rate for more customers within NOAA.

Providing value with reports and visualizations

A significant benefit of using InCites is that NOAA can rapidly and reliably produce regular reports that use the same parameters, for example, annual or quarterly reports on research impact and achievements. It can be challenging to compare NOAA’s research output to that of other agencies, but a former researcher had done just that in an annual publication. Unfortunately, the methodology for that was not well-documented.

Using capabilities in InCites, Davis and Roberts were able to re-create the annual report and preserve the data collection and analysis steps for future reproducibility. The report now serves as a benchmark for NOAA’s achievements when compared to other highly-ranked research organizations, such as the National Institutes of Health. They are also able to contribute meaningful and consistent data and visualizations to higher-level authorities for broader inter-agency compilations.

In addition to being able to quickly and reliably produce citation and impact reports for the top brass, NOAA librarians can provide this as a service to individual researchers. Using Microsoft Excel and PowerPoint, Davis and Roberts can export data from InCites and transform it into readable and colorful presentation templates that give researchers an at-a-glance look at their citations and publications.  

Using bibliometrics to strengthen the library’s mission

Federal libraries continue to struggle with shrinking budgets and with demonstrating their value to skeptical lawmakers who only see dollar signs. Bibliometric analysis allows libraries to add quantitative color to their stories of contribution and achievement. Making the data come alive with infographics, charts and other visualizations allows viewers to understand what they are seeing and draw the right conclusions. Being able to inform funding decisions with bibliometric services like citation analysis is a strategic differentiator for any federal library.

Audrey McKay

Audrey McKay

Client Engagement Manager at LAC Group.