This study compares Brazil's expert-driven journal evaluations with citational measures to see which better explains journal quality in political science between 2010 and 2014.

📚 What Was Compared

• CAPES' Qualis evaluation system (expert-driven rankings) using data provided by CAPES for 2010–2014
• Citational indicators used for comparison:
• SCImago Journal Rank (SJR)
• SCImago h-index
• Google Scholar Metrics h5-index and h5-median (5-year window)
• SNIP from CWTS Journal Indicators (Scopus)

📊 How The Comparison Was Done

• Pairwise correlations assessed the relationship between Qualis ranks and multiple citational indices.
• Ordered logistic regressions examined which factors predict a journal's next Qualis grade, with key covariates including:
• a journal's past Qualis score
• its citational rankings (SJR, h-index, h5 measures, SNIP)
• country of publication, publication language, and whether the journal has a social science focus

🔑 Key Findings

• There is a positive but weak correlation between citational indicators and Qualis evaluations.
• A journal's past Qualis score is the strongest predictor of its subsequent Qualis grade in ordered logit models.
• Once past Qualis score is controlled for, citational rankings generally do not influence Qualis grades, with one exception: SJR had an effect in the 2013–2014 evaluation cycle.
• Country of publication, language, and social science focus do not significantly affect Qualis scores when other factors are held constant.

🌍 Why It Matters

• These results highlight path dependence in expert-driven evaluation systems: past reputational judgments strongly shape future assessments.
• Reliance on expert panels (as in Qualis/CAPES) limits the influence of citation-based metrics, which has implications for evaluation transparency, cross-national comparability, and incentives for journals and scholars.