This hands-on two-day online workshop equips PhD students with the practical skills to apply AI-assisted text analysis in R using LLMs, combining applied coding with critical reflection on methodological and ethical research standards.
What Is This Course About?
This short two-day, online workshop is an intensive, hands-on seminar designed specifically for PhD students interested in applying AI-assisted text analysis using R. It provides practical training in working with both open-source and closed large language models (LLMs) for tasks such as classification and scaling, while emphasizing reproducible and transparent research workflows. Throughout the workshop, participants will engage in applied coding exercises while also reflecting on the methodological and ethical implications of integrating LLMs into research.
Learning Goals
After this course you will:
- Be able to identify and apply suitable AI-assisted text analysis techniques to address your own research questions using R
- Know how to work with both open-source and closed large language models (LLMs) for tasks such as text classification and scaling, and integrate them effectively into your research workflow
- Understand key principles of reproducibility, validation, and ethical use of AI tools in academic research, and apply them to your own projects
- Be familiar with key R tools and packages for AI-assisted text analysis (such as the quallmer R package), and confident in implementing them through hands-on coding exercises
Recommended Readings for the Course
- Alizadeh, M., Kubli, M., Samei, Z., Dehghani, S., Zahedivafa, M., Bermeo, J. D., Korobeynikova, M., & Gilardi, F. (2025). Open-source LLMs for text annotation: A practical guide for model setting and fine-tuning. Journal of Computational Social Science, 8(1), 1–25.
- Benoit, K., De Marchi, S., Laver, C., Laver, M., & Ma, J. (2026). Beyond text-as-data to natural language understanding: Qualitative analysis of political texts using LLMs. American Journal of Political Science.
- Maerz, Seraphine F., Schafer Dean G., and Carsten Q. Schneider. 2026. “Listening to Leaders: Illiberal Speech as a Symptome of Democratic Decline", accepted for publication in Comparative Political Studies.
- Halterman, A., & Keith, K. A. (2025). Codebook LLMs: Evaluating LLMs as measurement tools for political science concepts. Political Analysis, 1–17.
- Le Mens, G., & Gallego, A. (2025). Positioning political texts with large language models by asking and averaging. Political Analysis, 1–9.
Assignments for the Course
1 coding challenges to be completed in class
Schedule
Pre-Course Preparation (available 3 weeks before course start)
• Asynchronous materials (approx. 90 minutes total):
o Set-up instructions for R, RStudio, and required packages
o Pre-reading on the use of LLMs for political text analysis
o Students are expected to complete these materials before Day 1.
o Materials and readings will be made available on Moodle.
Day 1: Foundations of AI-assisted text analysis in R
• 90’ synchronous lab
o Introduction to text analysis in R and workshop overview
o Hands-on session: Getting started with text analysis in R
• 45’ office hour
o Live Q&A and troubleshooting set-up in R
• 90’ asynchronous materials (flexible, but before group learning)
o Introduction to LLMs: Types, capabilities, and limitations
• 60’ independent small group learning (with TA)
o Reflection on ethical considerations and methodological implications of working with LLMs
• 30’ office hour (with TA)
o Q&A and troubleshooting set-up in R
Day 2: Applied LLMs, text classification, and reproducibility
• 90’ synchronous lab
o LLM API and local set-up (e.g., openai, Ollama) in R
o Hands-on session: Running classification tasks with different LLMs
• 45’ office hour
o Live Q&A and help with LLM API/local setup
• 90’ asynchronous materials (flexible, but before group learning)
o Validating and comparing model outputs
o Building reproducible Quarto workflows for AI-assisted text analysis
• 60’ independent small group learning (with TA)
o Coding challenge: classification task with closed and open-source LLMs in a reproducible workflow
• 30’ office hour (with TA)
o Q&A and feedback on the coding challenge
In addition, students should plan for sufficient time to read the assigned literature in advance.
Who Is Your Instructor?
Seraphine F. Maerz is a political scientist and Senior Lecturer at the University of Melbourne, specializing in computational social science and quantitative methods. Her research focuses on democracy, authoritarianism, and political communication, with a methodological emphasis on text analysis and the application of large language models in political research. See her GoogleScholar profile for recent publications. She is co-founder of QuantLab and co-developer of quallmer. More information and regular updates about her work can be found on her website at seraphinem.github.io. Connect with her via LinkedIn.
While having R skills is an advantage, this workshop is designed to be accessible for beginners, offering a supportive environment to help you build your knowledge from the ground up. Participants will need to have an up-to-date version of R and RStudio installed on their computers. The workshop covers the (optional) application of closed (and fee-based) large language models as well as open-source alternatives (for free).
Dr Seraphine Maerz
Lecturer komex
