1. médialab Sciences Po
  2. News
  3. Hiring Post-doctoral Fellow to work on biases in LLMs within a deliberative context

Hiring Post-doctoral Fellow to work on biases in LLMs within a deliberative context

Sciences Po's médialab is hiring a post-doc in computational social sciences to work on the identification and characterization of biases in LLMs within a deliberative context.

Notice, Job

Deadline:

About the project

This position is part of a research project within the Digital Commons program funded by BPI in partnership with ISIR (CNRS, Sorbonne University) and make.org. Numerous international academic and industrial partners complete the consortium (Hugging Face, Genci, Mozilla, Berkman Klein Center, etc.) of the "democratic commons."

The "Democratic Commons" program brings together dozens of researchers and engineers over a two-year period in sociology, political science, natural language processing, and engineering to develop and freely share an evaluation model for AI biases applied to democratic debate spaces. The objective is to characterize these biases in this specific context, provide an evaluation model, and work on training LLMs that adhere to the stated democratic principles.

Beyond the demographic biases already well-documented in the literature, we will pay particular attention to the fidelity of language models to the opinions expressed by contributors on deliberative platforms. In such a context, large language models promise increased and simplified participation from a greater number of citizens. Conversational bots can, indeed, facilitate participation by summarizing lengthy debates, reformulating citizens' contributions if they are misaligned, or simply translating them to include contributors speaking different languages. However, these applications—given their application context—also call for great vigilance. The project specifically aims to guard against potential biases that these "translation" operations might have on participation and the quality of democratic debate by precisely characterizing them and proposing tools to measure them.

The successful candidate will join an interdisciplinary team of researchers, engineers, and designers at the médialab, working in connection with a team of political scientists at CEVIPOF. They will naturally collaborate closely with the engineers and computer scientists of the project consortium.

Responsibilities

The recruited post-doc will be responsible for setting up a protocol for the characterization and measurement of democratic biases in the context of LLM-based applications developed to improve the quality of online debates. In collaboration with the political scientists engaged in the project and the computer scientists at ISIR (three PhD students working on summarization, translation, and reformulation tasks), the post-doc will need to design a benchmark for the evaluation of democratic biases. Specifically, we aim to guarantee the maintenance of pluralism in expressed opinions when debates are summarized (thereby preventing minority opinions from being marginalized). Similarly, we will seek to measure the extent to which users' expressed opinions are faithfully conveyed by the AI after rephrasing or translation. Interaction data between conversational bots and users (qualified) will be provided by make.org, which plans to deploy AI-assisted deliberative spaces in four different countries and three languages to vary the socio-cultural contexts.

Experiences and skills required

The ideal candidate should have a solid understanding of machine learning algorithms and natural language processing, as well as a strong background in social sciences. Prior experience in the evaluation or audit of algorithmic systems is desirable but not mandatory. We are looking for hybrid profiles in computational social sciences. This could include candidates with a PhD in disciplines such as computer science, applied mathematics, or physics, with a notable interest in social sciences. We also welcome applications from PhDs in sociology, political science, or economics with proven experience in "data science."

Work environment

The successful candidate will be integrated into SciencesPo's médialab, located in the 7th arrondissement of Paris (remote work possible one or two days per week) and will work with its team of researchers, engineers, and designers.

The position is open starting from October 2024, for a duration of 24 months. The proposed gross salary between €3200 and  €3700/month according to experience. The position includes comprehensive health coverage, 40 days of paid vacation per year, 5 RTT days, 50% coverage of public transportation costs, and meal vouchers.

To apply

Please send a CV and a cover letter detailing your relevant experiences and skills for the position, along with a link to your recent scientific articles (published or in progress) to the following addresses before July 23. Interviews will follow for a position start in the fall (starting from October 1, 2024):