Imagine 2016 - Symposium Abstract - How Journals Judge Innovation

In an era of near instantaneous on-line communication, and enormous data repositories, some have questioned whether or not traditional biomedical journals are necessary. In fact, some have heralded their rapid demise.  Journals are not needed to aggregate scientific reports or to distribute them. However, most agree that a journal’s important role is a filter function. Currently, humans are required to judge what is missing from an aggregation of information on a topic.  The key to effective human filtering is leveraging “a fabric of trusted individuals thru a social network to validate and prioritize specific information in a discipline”. Journals such as ours use a large number of peer reviewers to critically appraise manuscripts that are submitted.  The editors however, don’t really count the votes, rather they use peer reviewers as consultants to inform their judgements on submitted manuscripts.

Radiology is the scientific journal of the Radiologic Society of North America. Our scope is broad, and we seek to publish the best of the submissions regarding imaging related, radiation oncology and medical physics research worldwide. Each year from nearly 3,000 submissions we publish approximately 380 manuscripts of original scientific investigation.  Currently 2/3 of our submissions are from outside of North America, as are approximately 50% of the manuscripts ultimately published. When evaluating these submissions we try to identify manuscripts that are novel, important, and/ or interesting and informative.  In our lexicon, novel manuscripts add new information, provide new concepts, describe new technology, define diagnostic or therapeutic approaches and potentially resolve existing controversies. Hypothesis generating research, although preliminary is an important part of the manuscripts that we publish.  We also seek to publish manuscripts that we think are important.  Important manuscripts have the potential to change practice, to help us better understand biology or technology, and they generate new hypotheses that stimulate further research.  We also publish manuscripts that we deem interesting and informative.  These may have limited novelty yet add considerably to our available information or have conclusions that help provide direction in areas of controversy.

Also central to our assessment of the quality of a submission is the scientific validity of the methods and conclusions in answering the question posed by the experiments. Another critical factor that must be determined is the ability to generalize from the results that have been reported. There is increased concern globally that much of the research submitted and published cannot be reproduced. These concerns have led to more stringent requirements for data sharing among investigators.  To deal with problems of scientific validity and inability to reproduce reported results, reporting standards or guidelines are desirable. These reporting guidelines can help identify the presence and nature of bias, identify methodologic problems such as sample size or inappropriate analysis. They also help ensure that the description of the methods is adequate to reproduce the study. These guidelines, however, do not ensure that a study is novel, or important, or even interesting.

We are in an era of burgeoning technology and in the ability to store and analyze vast amounts of information. These developments occur as the background to a number of important trends in innovation.  After decades of reductionist biomedical science, systems biology has moved to the forefront.  The inter-relationships of different organs in disease processes are being recognized as increasingly important. The so-called “omics”, namely genomics, proteomics and metabolomics are providing important data on human physiology in health and disease.  Moreover, enhanced computational tools have made possible more quantitative approaches to imaging science in radiology and pathology.  Finally, volume and whole body imaging at very high spatial resolution with temporal resolution on the order of seconds or sub-second resolution is feasible and increasingly important. These trends are combining and form what is now called integrated diagnostics, a comprehensive informatics approach to analyzing disparate data to determine diagnosis and prognosis.  Examples of these will be further illustrated during my presentation.