1. médialab Sciences Po
  2. Productions
  3. Political Bias in Large Language Models Is a Curve, Not a Point

Political Bias in Large Language Models Is a Curve, Not a Point

Paul Bouchaud, Pedro Ramaciotti

Audits of political bias in language models typically report a point estimate on a left-right axis. That number is unstable: rewording a question, changing the answer format, or changing who appears to be asking relocates a model across the spectrum, so the estimate measures the prompt as much as the model. A chatbot produces political speech in reply to a user, which makes the measurable object a relation rather than a trait: how endorsement varies with the leaning of the stance a user brings. We scale stances on policy items from a nationally representative U.S. survey onto the left-right axis, pose each in the register of real user prompts, and record how four frontier models (GPT-5.6 Luna, Gemini 3.6 Flash, Claude Sonnet 5, Grok 4.3) reply. The resulting endorsement curve -share of agreement as a function of a stance's leaning-distinguishes two quantities that traditional audits fuse: where on the left-right axis a model's agreement peaks, and how far from that peak it keeps agreeing. Two models with the same point estimate can treat users in opposite ways -one validating a single part of the spectrum while resisting the other, one declining to side with anyone-a difference a point conceals and a curve reveals.