An Ai-Assisted Self-Building Protest Database

Studying the effects and causes of protests requires valid measurement of a seemingly objective fact: the number of participants which signals relevance and saliency. Past efforts aiming to create accurate datasets have thus usually relied on qualitative analysis of multiple sources and triangulation, weighing expert knowledge about the trustworthiness and stance on particular issues of sources. While of great quality, these datasets are often limited in scope, cover only short periods, and do not cover the most recent or ongoing protest movements – which makes it difficult to study, for example, demographic shifts across movements, long-term trends or provide timely analysis of emerging events. To tackle this problem, we suggest a pipeline that combines web-scraping, AI-based data extraction and a novel Bayesian framework for source evaluation inspired by multi-coder annotation: using generative AI, we extract information about all protest events from all news articles about these events and evaluate the source reliability by comparison. We demonstrate our approach using a novel dataset of 24,822 articles on protests collected from 48 German-speaking newspapers since April 2025. Across events, this allows us to estimate media-level confidence and bias scores that can be used to weight numbers in other reports by the medium. This automated approach enables continuous, scalable data collection while maintaining quality control through systematic source evaluation, making it possible to rapidly respond to new protest events and identify broad patterns such as changing participant demographics that require extensive cross-event data.