The recent emergence of generative AI has many in academia scrambling to keep up. Chris Belter from the NIH Eunice Kennedy Shriver National Institute of Child Health and Human Development will join us as the Keynote speaker at the 2024 NAPUG conference to talk about generative AI and research impact. As more students and researchers apply these tools to their writing and research activities, questions about their accuracy and ethical use become more important. This is especially true in the field of research impact, where generative AI could transform the ways that we measure and describe the impact that research has on science and society. In this talk, I will discuss a project in which we used generative AI to generate impact stories for selected research projects and then validated those stories using the projects’ known research outcomes. Our results indicate significant weaknesses and potential biases in the current technology, suggesting that generative AI is not yet ready to perform these kinds of tasks. I will then close with some reflections on our work so far and on the potential future of generative AI in research impact studies.
Chris Belter is the lead analyst and evaluator for the Division of Extramural Research at the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), a division of the US National Institutes of Health. At NICHD, he designs and conducts analyses to inform the Institute’s strategic planning and funding decisions, provides expert advice on the use of data and analytics in those decisions, and coordinates analytical activities across the Division. Prior to joining NICHD, he co-created and led bibliometric service programs at the NIH Library and the Central Library of the National Oceanic and Atmospheric Administration. Chris has a master’s degree in library science from the University of Maryland and a BA in religion from Shenandoah University.