| |
 |
| Paper: |
Metrics of Research Impact in Astronomy |
| Volume: |
0, Metrics of Research Impact in Astronomy |
| Page: |
7 |
| Authors: |
John Kormendy |
| Abstract: |
Summary of key conclusions:
1. The unique contribution of this book is to calibrate the interpretation of metrics
derived from the SAO/NASA Astrophysics Data System using mean judgments
provided by 22 experienced astronomers about the career impact of 510 faculty
researchers at 17 highly-ranked universities world-wide.
2. It is more important to write high-impact papers than to write many papers.
3. Paper counts tell us little about research impact. If the aim is to rank candidates
for a resource such as a job, then counting papers is not a good strategy.
4. Citation counts are the most reliable impact indicator for non-big-team people. If
candidates who work substantially in teams of > 30 people are included in a search,
then the most reliable metrics are normalized citation counts.
5. Eight metrics are calibrated: citations of all papers, citations of refereed papers,
citations normalized by number of authors, tori index=twice-normalized citations,
first-author citations, first-author citations received in 2013 – 2017, I100 index,
and paper reads. Two metrics are analyzed with two different techniques each to
demonstrate that conclusions are robust. Different combinations of metrics work
best for different cohorts.
6. Instrumentalists are measured essentially as well as people in other fields, if metrics
are chosen carefully.
7. “Research impact quotient” riq is an age-independent measure of research quality.
It is a useful complement to career-integrated metrics such as citation counts.
8. Hirsch (2005) index H correlates so well with the square root of citation numbers
that, for most people, it provides no additional information. However, outliers to
this correlation who have unusually small H for their citation numbers are people
who have made outstandingly important contributions (e. g., Nobel Prize winners).
9. Metrics measured early in a career (after 10 years of post-PhD ramp-up) can be
used to predict impact later in that career. This is shown for three metrics, citations
of refereed papers, normalized citations, and citations of first-author papers.
10. All metrics are probabilistic in the sense that a small fraction of people are not
well measured. Correlations of impact with individual metrics have scatter that is
quantified here, plus almost every correlation has a few major outliers. Therefore:
11. Metric combinations measure impact more accurately than does any single metric.
12. Standards for tenure-stream jobs are shown in Chapters 7.3 and 7.4 (publications)
and Chapter 10 (citations). Chapter 14 summarizes advice for young scientists. |
|
|
|
|
|
