Evaluating Comment-Section Conditions and Perceived News Credibility: an Instructional Randomized-Experiment Simulation in a Nigerian University Context

Abstract

Online news is increasingly consumed in interfaces where professional journalism is displayed alongside user-generated comments, creating a setting in which social cues can shape evaluations of the article itself. This study employed a single-factor, between-subjects randomized experimental design to examine whether different comment-section conditions influence perceived news credibility. The study involved 90 undergraduate participants aged 18–25 years in a Nigerian university context, allocated equally to one of three conditions: a neutral news article followed by six civil, relevant comments and a moderation notice; the same article followed by six uncivil comments that also included irrelevant or spam-like content and a non-moderation notice; or the article presented without a comment section. Perceived credibility was measured using five 7-point items, which demonstrated high internal consistency (α = .94). The analysis revealed a statistically significant difference in perceived news credibility across the three conditions, F(2, 87) = 40.49, p < .001, η² = .48. Mean credibility was highest in the moderated-civil condition (M = 5.51, SD = 0.74), followed by the no-comments condition (M = 4.65, SD = 0.89), and the unmoderated-uncivil condition (M = 3.59, SD = 0.85). Tukey post-hoc comparisons indicated statistically significant differences between all three pairs of conditions. Among participants exposed to comments, perceived comment civility was associated with a significant indirect effect of experimental condition on perceived credibility (indirect effect = 1.38, 95% bootstrap CI [0.63, 2.10]). Because moderation status, civility, relevance, and spam-like content were bundled within the comment conditions, the findings represent differences between composite comment-section environments rather than the isolated causal effect of moderation alone. The findings are consistent with heuristic-cue accounts of online credibility and demonstrate that the social characteristics of comment sections can influence how audiences evaluate the credibility of online news.