Auditing the audits: evaluating methodologies for social media recommender system audits

Through a simulated Twitter-like platform designed to optimize user engagement and grounded in authentic behavioral data, this study evaluates methodologies for auditing social media recommender systems. Our analysis focuses on the impact of key parameters in sock-puppet audits, the number of friends and session length, on audit outcomes. Additionally, we investigate the algorithmic amplification of political content across different levels of granularity, segmenting users based on political leanings and considering multiple political dimensions beyond declared affiliations. Our findings underscore the necessity of employing realistic parameter settings in audits and highlight the importance of nuanced political segmentation. Amid increasing regulatory scrutiny, this research contributes to enhancing methodologies for auditing social media platforms.