School-based assessments (SBAs) were introduced to improve the validity and authenticity of public examinations across the globe. However, associated with this addition were issues related to teacher involvement, lack of effective material support, and student competence. There were also issues related to reliability and validity in SBAs which led to the conclusion that while reliability is a pre-condition for validity, it is also in tension with it. This paper assesses the extent to which this paradox is evident in SBAs by examining the measures taken by the Caribbean Examinations Council (CXC) to ensure the reliability and validity of its SBAs. The paper also discusses which of the qualities should be paramount and why. It concludes that the paradoxical relationship between reliability and validity is evident in SBAs. Additionally, it argues that validity should be paramount in SBAs because reliability is a pre-condition for validity, albeit a critical one.