National and international large-scale literacy assessments play an indelible role in impacting educational policy and curriculum across the globe. In fact, the value attached to these measures persists regardless of questions raised about the influence of the nature of such tests, characteristics and requirements of test items, and the ways in which demographic characteristics such as English learner status, gender, ethnicity, socioeconomic status (SES), and disability impact student literacy performance. This study examines the effect of self-reported language spoken at home and the language in which reading literacy was assessed, on standardized assessment scores in PISA reading literacy. Results indicate a large cross-country variation in the effect of language status on reading literacy as positive and significant for most countries, with higher student performance on reading literacy assessment when students were tested in their native language. The study acknowledges limitations based on language variables that largely affect students’ language self-identification or self-reporting on the PISA reading literacy assessment measure. By extension, policy-makers across countries are invited to interrogate the impact of such challenges on youth’s literacy performance, and literacy researchers, to further examine the impact of these assessments on the representation of youth’s literacies.