SoylentNews

takyon [soylentnews.org] writes:

Warning to academics: Hyphens in paper titles harm citation counts and journal impact factors [www.hku.hk]

According to the latest research results, the presence of simple hyphens in the titles of academic papers adversely affects the citation statistics, regardless of the quality of the articles. The phenomenon applies to all major subject areas. Thus, citation counts and journal impact factors, commonly used for professorial evaluations in universities worldwide, are unreliable.

[...] Professor [T.H.] Tse and team members proposed an innovative method named "metamorphic robustness testing" to verify Scopus and Web of Science. The in-depth study uncovered robustness defects in both systems that might produce erroneous citation counts for papers with hyphens in the titles, so that the journal impact factors subsequently computed are problematic.

Back in 2015, Letchford and colleagues conducted a large-scale study on Scopus, and found that papers with shorter titles tended to be cited more than those with longer titles. Their results were widely reported in international media including Science and Nature.

On the contrary, Professor Tse and the present team find that it is actually the number of hyphens in the title that serves as the more dominating factor for citation counts. Usually, the number of hyphens is correlated to a paper's title length, thus giving the misinterpretation that citation counts depend on title length.

[...] "Our results question the common belief by the academia, governments, and funding bodies that citation counts are a reliable measure of the contributions and significance of papers. In fact, they can be distorted simply by the presence of hyphens in article titles, which has no bearing on the quality of research. Similarly, our results also challenge the validity of journal impact factors," said Professor Tse. "These surprising results are of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. They are applicable to all faculties in any university," he added.

Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors [ieee.org] (open, DOI: 10.1109/TSE.2019.2915065) (DX [doi.org])

Older research: The advantage of short paper titles [royalsocietypublishing.org] (open, DOI: 10.1098/rsos.150266) (DX [doi.org])

Original Submission