The SFB 1187 “Media of Cooperation” is pleased to welcome Kirill Maslinsky (INALCO, Paris) for a guest lecture on the methodological promises and pitfalls of using large language models in empirical research. The lecture continues the SFB’s ongoing discussion of AI-based research methods, including the Synthetic Methods series and the Critical Data Schools on AI methods. It looks at a question that becomes urgent once LLM agents are used to analyse whole corpora: when do the numbers they produce actually support the claims we want to make?

Abstract

Modern LLMs have made it possible to automate content analysis tasks with a much higher level of semantic and contextual understanding than earlier NLP technologies. By analogy with Web 2.0, this transition may be labelled Content Analysis 2.0. The recent surge of agentic workflows has made these capabilities applicable at scale to real corpora. Isn’t this the dream of mixed-methods research, where qualitative understanding translates into quantitative results with only modest time and budget requirements?

The problem with this scaling is that in the naive setting, where we simply instruct agents and collect results, we risk running estimation without identification. Estimation is the procedure that produces the numbers of interest that characterise the content of a corpus: proportions, trends, and effect sizes. Identification is the argument that the data at hand can support the claim we wish to make. The apparent fluency of LLM-generated results masks the lack of identification for some of the claims they produce. In this lecture, I will discuss methods to ensure the identification of text-based claims in the context of agentic workflows.

About the speaker

Kirill Maslinsky is a quantitative researcher in the humanities and a research engineer at INALCO (Institut national des langues et civilisations orientales) in Paris. His background is cross-disciplinary: he trained in linguistics and literary studies and holds a PhD in cultural anthropology. His current work focuses on computational literary studies, including a quantitative history of Soviet children’s book publishing, and he contributes to the ERC project “Theory of Tone”. His methodological toolkit combines causal inference, Bayesian statistical modelling, and agentic workflows adapted for text analysis. He is committed to open data and open research practices and to building digital research infrastructure.

The lecture will be held in English and is open to all SFB members, students, and interested guests. It takes place on-site with a hybrid option. Please register via email to info@sfb1187.uni-siegen.de to receive the Zoom link.