Ability of Serum Glial Fibrillary Acidic Protein, Ubiquitin C-Terminal Hydrolase-L1, and S100B To Differentiate
Normal and Abnormal Head Computed Tomography Findings in Patients with Suspected Mild or Moderate
Traumatic Brain Injury
Robert D. Welch,1Syed I. Ayaz,1Lawrence M. Lewis,2Johan Unden,3James Y. Chen,4Valerie H. Mika,1 Ben Saville,5Joseph A. Tyndall,6 Marshall Nash,7Andras Buki,8Pal Barzo,9Dallas Hack,10Frank C. Tortella,11
Kara Schmid,12 Ronald L. Hayes,13 Arastoo Vossough,14Stephen T. Sweriduk,15and Jeffrey J. Bazarian16
Abstract
Head computed tomography (CT) imaging is still a commonly obtained diagnostic test for patients with minor head injury despite availability of clinical decision rules to guide imaging use and recommendations to reduce radiation exposure resulting from unnecessary imaging. This prospective multicenter observational study of 251 patients with suspected mild to moderate traumatic brain injury (TBI) evaluated three serum biomarkers’ (glial fibrillary acidic protein [GFAP], ubiquitin C-terminal hydrolase-L1 [UCH-L1] and S100B measured within 6 h of injury) ability to differentiate CT negative and CT positive findings. Of the 251 patients, 60.2% were male and 225 (89.6%) had a presenting Glasgow Coma Scale score of 15. A positive head CT (intracranial injury) was found in 36 (14.3%). UCH-L1 was 100% sensitive and 39% specific at a cutoff value>40 pg/mL. To retain 100% sensitivity, GFAP was 0% specific (cutoff value 0 pg/mL) and S100B had a specificity of only 2% (cutoff value 30 pg/mL). All three biomarkers had similar values for areas under the receiver operator characteristic curve: 0.79 (95% confidence interval; 0.70–0.88) for GFAP, 0.80 (0.71–0.89) for UCH-L1, and 0.75 (0.65–0.85) for S100B. Neither GFAP nor UCH-L1 curve values differed significantly from S100B (p=0.21 and p=0.77, respectively). In our patient cohort, UCH-L1 outperformed GFAP and S100B when the goal was to reduce CT use without sacrificing sensitivity. UCH-L1 values<40 pg/mL could potentially have aided in eliminating 83 of the 215 negative CT scans. These results require replication in other studies before the test is used in actual clinical practice.
1Department of Emergency Medicine, Wayne State University, Detroit, Michigan.
2Department of Emergency Medicine, Washington University, St. Louis, Missouri.
3Department of Anesthesiology and Intensive Care, Clinical Sciences in Malmo, Malmo, Sweden.
4Department of Radiology, University of California San Diego Health System, San Diego, California.
5Berry Consultants, Austin, Texas.
6Department of Emergency Medicine, University of Florida, Gainesville, Florida.
7Neurostudies.net, Decatur, Georgia.
8Department of Neurosurgery, Pecs University, Pecs, Hungary.
9Department of Neurosurgery, University of Szeged, Szeged, Hungary.
10U.S. Army Medical Research and Materiel Command, Fort Detrick, Maryland.
11Applied Neurobiology, and12Center for Military Psychiatry and Neuroscience Walter Reed Army Institute of Research, Silver Spring, Maryland.
13Center of Innovative Research, Banyan Biomarkers, Inc., Alachua, Florida.
14Department of Radiology, University of Pennsylvania, Philadelphia, Pennsylvania.
15Department of Radiology, Shields Health Care Group, Brockton, Massachusetts.
16Department of Emergency Medicine, University of Rochester School of Medicine, Rochester, New York.
ªRobert D. Welch, et al. 2015; Published by Mary Ann Liebert, Inc. This Open Access article is distributed under the terms of the Creative Commons Attribution Noncommercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits any noncommercial use, distribution, and repro- duction in any medium, provided the original author(s) and the source are credited.
JOURNAL OF NEUROTRAUMA 33:203–214 (January 15, 2016) Mary Ann Liebert, Inc.
DOI: 10.1089/neu.2015.4149
203
Introduction
C
omputed tomography(CT) is a frequently used diagnostic test for patients with mild and moderate traumatic brain injury (TBI), but it is considered to be unnecessary for many patients.1,2 Overuse of CT for evaluating patients with mild TBI (mTBI) has continued despite the availability of clinical decision rules,3,4in- creasing awareness of potential hazards of ionizing radiation ex- posure,5,6 and some professional society’s recommendations to reduce CT imaging.7This has led to a search for an easily obtained and objective test that can, in the early post-injury period and with high sensitivity, predict which patients with mTBI will have an acute intracranial lesion found on head CT. A highly sensitive test, even with only moderate specificity, would provide clinicians with a new tool that would be useful in reducing CT use for patients with suspected mTBI.Over the last two decades, there has been increasing interest in several serum proteins that could potentially predict the presence of a brain injury from head trauma. These proteins are released as a result of blunt or rapid deceleration forces to the head that cause injury to the neurons and supporting glial cells. The most widely studied biomarker for this purpose is S100B,8–15a protein that is released after astroglial injury.16It has been suggested that a S100B biomarker level below a threshold level can safely eliminate the need to obtain a CT scan in patients with mTBI.17
The Scandinavian Neurotrauma Committee 2013 guidelines for the management of minimal, mild, and moderate head injuries in adults included the use of S100B as a blood sample derived screening test for CT-proven traumatic intracranial hemorrhage (ICH).17In the United States, however, this has not yet been widely accepted and has not seen broad clinical use. The American College of Emergency Physicians (ACEP) clinical policy for neuroimaging and decision making for adults with mTBI in 2008 gave a ‘‘Level C’’
recommendation for the use of S100B as a screening tool to assess the need for CT imaging in patients with mTBI and no other extra- cranial injuries,7but the 2012 Eastern Association for the Surgery of Trauma guidelines stated that there was not enough evidence to warrant reliance on S100B to determine the need for head CT.18
More recently, other biomarkers have begun to receive more attention for a potential role in detecting brain injury. These include glial fibrillary acidic protein (GFAP) and ubiquitin carboxyl- terminal hydrolase-L1 (UCH-L1). GFAP is a protein expressed in astroglia cells, but unlike S100B, GFAP is not found in significant amounts in extracerebral cells.16 Previous studies have demon- strated that elevated levels of GFAP in patients with mTBI are associated with abnormal findings on imaging of the brain,12can
predict the need for surgical intervention,19and are able to dif- ferentiate between patients in motor vehicle crashes with ortho- pedic injuries and those who had mTBI.19 McMahon and associates20recently reported that elevated levels of GFAP mea- sured within the first 24 h after injury could help reduce unneces- sary CT scans without sacrificing sensitivity in cohorts including patients with mild, moderate, and severe TBI and may outperform S100B for the detection of intracranial lesions on CT for patients with mild TBI.13
UCH-L1 is a deubiquitinase present in neurons.14,21Elevated serum levels of this protein have been correlated with brain injury and clinical outcome.22There is some evidence for GFAP but only little evidence for UCH-L1 regarding the early diagnostic utility of these biomarkers in patients with mTBI, and these studies include only small numbers of patients with head CT positive acute intra- cranial lesions.23
Given the limitations of S100B as an ideal single biomarker for pre-CT screening of patients with mTBI, it was postulated that other biomarkers, such as GFAP and UCH-L1, could be useful in screening for acute intracranial lesions among patients with mTBI.
Therefore, the goal of this pilot study was to conduct the first evaluation of the early (within 6 h of injury) diagnostic character- istics of GFAP and UCH-L1 to exclude the presence of acute in- tracranial lesions on head CT in adult patients with mTBI. We also compared the diagnostic characteristics of UCH-L1 and GFAP with S100B. These results will be used to derive potential threshold values for screening patients with mTBI to determine the need for a head CT scan.
Methods
Patients and study procedures
This prospective multicenter observational study included pa- tients 18–80 years of age who were evaluated and treated at one of seven study site hospital emergency departments (EDs) for a blunt closed head injury and potential mild to moderate TBI. The hos- pitals were composed of Level 1 and 2 trauma centers, a nontrauma center, and included both U.S. and European sites (Table 1). Eli- gible patients were those with an initial Glasgow Coma Scale (GCS) score of 9–15 who underwent emergency head CT scan for evaluation of the head injury as deemed necessary by the attending ED physician.
No formal rules for obtaining a CT were used so as to best reflect current practice in the United States or Europe, but all sites were aware of and considered available clinical decision rules to guide the need for brain imaging. Included patients presented within 4 h of injury, completed the required CT scan as part of routine care,
Table1. Description of Study Site Hospitals
Study site Location Annual ED volume Designation
Washington University, Barnes-Jewish Hospital St. Louis, MO 95,000 Level I
University of Florida, Shands Hospital Gainesville, FL 66,000 Level I
Gwinnett Medical Center–Lawrenceville Atlanta, GA 100,000 Level II
Dekalb Medical–North Decatur Atlanta, GA 92,000 Non-trauma
Wayne State–Detroit Receiving Hospital Detroit, MI 90,000 Level I
University of Pe´cs Medical Center Pecs, Hungary 25,000O Level 1*
Albert Szent-Gyo¨rgyi Medical Center–University of Szeged Szeged, Hungary 90,000 Level 1*
ED, emergency department.
OThe ED volume is>25,000/year, and the facility cares for more than 2000 neurotrauma cases/year.
*This is equivalent to a Level 1 facility in the United States.
204 WELCH ET AL.
and had blood drawn for analysis within 6 h of injury. Table 2 details the full inclusion and exclusion criteria.
Institutional Review Board approval was obtained at each study site, and each site also obtained approval by the ethics board of the U.S. Army Medical Research and Materiel Command (USAMRMC) Office of Research Protections (ORP) Human Research Protection Office (HRPO) Department of Defense. Baseline patient data col- lected included demographics, medical history, substance use, GCS scores, and circumstances related to and the mechanism of injury. This was used to describe the study patient’s general characteristics only.
Serum sampling and handling
Blood samples were collected at time of study enrollment and every 6 hours up to the time of discharge (either ED or hospital) or up to 24 h (maximum of five samples during index visit). Patients who were seen at follow-up (Day 35–5 days) had another sample obtained when feasible. Blood samples were processed and the resulting serum was stored at-80C and then shipped on dry ice to a central repository for storage until time of testing as per a pre- defined specimen handling procedure.
Serum analysis for GFAP, UCH-L1, and S100B
Serum samples were analyzed for UCH-L1 and GFAP concen- tration by the Banyan enzyme-linked immunosorbent assay at a later time by technicians blinded to clinical data and CT results. The assay was performed as follows: the test sample was added to the well of a 96-well microtiter plate that had been coated with a capture antibody specific to the antigen of interest (UCH-L1 or GFAP).
Following a set incubation period during which any antigen present in the sample binds to the capture antibody, unbound material was removed by washing with buffer. Horseradish peroxidase (HRP)-
conjugated detection antibody was then added to the well, binding to a second site on the antigen. Following a set incubation period, excess detection antibody was removed by washing with buffer.
A chemiluminescent substrate was then added to the well. The HRP enzyme catalyzes a specific reaction with the chemilumi- nescent substrate, which produces light that was detected with a 96-well plate-based luminometer. The amount of light generated is proportional to the amount of detection antibody and thus UCH-L1 or GFAP in the test sample and was compared with light generated from wells containing known antigen concentrations representing a calibration curve. Samples were tested in duplicate, and high and low positive controls were included with each plate. The lower limit of quantification (LLOQ) and lower limit of detection (LLOD) for UCH-LI are£30 pg/ml and£10 pg/mL and for GFAP
£30 pg/ml and£20 pg/mL, respectively.
S100B concentrations were determined using an electro- chemiluminescence immunoassay designed forin-vitrodiagnostic testing (Roche, Cobas 6000). Results are given as S-S100 inlg/L with the standard normal reference intervals of 0.00–0.09lg/L.
Although various cutoff values have been proposed, for the pur- poses of this study, an S100B‡0.10lg/L (100 pg/mL) was con- sidered to be abnormal and could indicate a traumatic abnormality on head CT.17,24
Assay results were not available to the treating clinician and were not used to guide treatment. For this main analysis, we only considered results for the first sample collected within 6 h of injury. For all bio- markers, results were reported in picograms/milliliter (pg/mL). Be- cause other studies used different units of measurement—we provided the units in the original referenced manuscripts and the equivalent in pg/mL for the above S100B sample measurement section and in the following results and discussion section where needed.
Head CT scans
Each subject’s head CT images were reviewed by an indepen- dent committee consisting of three blinded board-certified neuro- radiologists. The neuroradiologists determined whether a CT scan was positive—defined as the presence of an acute trauma-related intracranial lesion. Table 3 lists the criteria for what was deemed a positive finding. A procedure outlining the criteria and process to be followed for scoring the CT scans was developed before the reading and interpretation of any of the CT images. Two of the neuroradiologists who had no access to any other clinical or labo- ratory data, except subject age and sex, reviewed all of the study subjects’ CT scans. Any discrepancy with respect to CT-positive or Table2. Inclusion/Exclusion Criteria
Inclusion criteria
The subject was‡18 years of age and no more than 80 years of age.
Acceleration or deceleration closed injury to the head that was either self-reported or witnessed.
Presented to an emergency department (ED) within 4 h of injury.
An initial Glasgow Coma Scale score of 9–15 in the ED performed by the Principal Investigator (PI) or trained study personnel.
ED workup included a head computed tomography (CT) scan (based on standard practice and/or decision rules).
Informed consent was obtained from the subject or his or her legal representative; oral consent for the initial blood draw and/or deferred consent to 24 h was allowed for patients who were unable to consent at initial evaluation or exception from the informed consent requirement by use of ‘‘community consent’’ if approved by an Institutional Review Board.
The PI deemed the subject to be an appropriate study candidate.
Exclusion criteria
Participation in another clinical study that may affect the results of either study.
Time of injury was not able to be accurately determined.
Head CT not done as part of clinical emergency care.
Primary diagnosis of ischemic or hemorrhagic infarct.
Not available for 35 day follow-up visit.
Venipuncture not feasible.
Blood donation within 1 week of screening.
The subject was otherwise determined medically unsuitable for study participation.
Table3. Definition of Acute Intracranial Lesion Acute intracranial lesion is defined as any trauma induced or
related finding. Acute lesions may include the following and the number of each finding:
N Extra-axial lesions
Acute epidural hematoma (EPH) 3
Acute subdural hematoma (SDH) 21
Cortical contusion 10
Ventricular compression 3
Ventricular trapping 0
Brain herniation 0
Intraventricular hemorrhage 1
Hydrocephalus 1
Subarachnoid hemorrhage 27
Petechial hemorrhagic or bland sheer injury 0
Brain edema 0
Post-traumatic ischemia 0
Intracerebral hematoma 0
Dural venous sinus injury and/or thrombosis 0
GFAP, UCHL-1, AND S100B TO CLASSIFY HEAD CT FINDINGS 205
CT-negative was adjudicated by a third blinded radiologist and, in those instances, was the final interpretation. Interrater reliability between the two primary radiologists was determined using the Cohen Kappa statistic.
Outcomes
For this study, the primary clinical outcome of interest was the results of the head CT scan (positive/negative) among patients with the first blood sample drawn within 6 h of injury.
Data analysis and reporting
A descriptive analysis for all subjects was performed. Patient groups (CT positive and CT negative) were described using pro- portions and means or medians where appropriate. For data analyses, any biomarker value below the detectible limit was coded as 0 pg/mL.
The exception to this was for the scatter plots that used a log linear y-axis for which a value of 0.01 pg/mL was used (0 pg/mL is not a valid value using the log scale). The Spearman correlation coef- ficients were calculated for each biomarker pairing. Logistic re- gression was used to determine area under the receiver operator characteristic (AUROC) curve for all three biomarkers indepen- dently and also for a single model with all three biomarkers col- lectively. Probability output from the logistic regression models and graphical displays were used to evaluate clinically relevant cutoff values for positive and negative values for GFAP, UCH- L1, and S100B.25A nonparametric approach was used to com- pare the correlated ROC curves without adjustments for pairwise comparisons.26
Sensitivity and specificity were calculated for the primary and secondary outcomes. Because there would be little acceptance of missed injury on CT scan, our goal was to maximize sensitivity to 100% at the expense of specificity. This allowed for determination of the number of CT scans that could potentially be safely elimi- nated when screening using each biomarker. We also examined threshold values that would result in sensitivity‡95% and‡90% for comparison of specificity.25For GFAP and UCH-L1, we calculated individual sensitivity and specificity values. We also implemented a combined biomarker strategy: if either GFAP was‡100 or UCH-
L1‡40 (chosen based on our data for optimal UCH-L1 sensitivity), the test was considered positive. For the reporting of S100B, we used the Scandinavian recommendation of>0.10lg/L (100 pg/mL) for patients injured within 6 h of serum sampling as the standard cutoff value.17
A sensitivity analysis (in this instance meaning varying the as- sumptions of the diagnostic tests) that examined patients with an initial sample obtained within 4 h (ACEP guidelines) of injury was performed to evaluate for diagnostic capability of the biomarkers at times closer to injury and times that may be more favorable to the standard S100B values.7Because the cut points for UCH-L1 and GFAP were optimized to our data, we also re-examined the AUROC curves and graphical data to determine whether a different cut point value for S100B would improve the diagnostic capability of that test for our data. This was done so not to ‘‘favor’’ the new diagnostic markers (GFAP and UCH-L1) over S100B.
There were no formal hypotheses for this study, and it was not designed to provide a definitive assessment of the studied bio- markers’ test characteristics. The study was, rather, descriptive and exploratory in nature, and the results will be used as an adjunct to help derive biomarker cutoff values for a future validation trial. The analysis included proportions with associated exact Clopper- Pearson 95% confidence intervals (CIs), means with standard de- viation (SD), and medians with 25th and 75th percentiles where appropriate. Thepvalues were only used for comparing AUROC curve data.
Recommended guidelines for reporting results of diagnostic tests were followed.27,28These results are not to be considered as evidence to reject a null hypothesis. All data analyses were per- formed using SAS9.4 and R 3.1.0.
Results
The study enrolled 290 patients; 26 were initially excluded for a variety of reasons, leaving a total of 264 patients who had data for at least one blood sample and the required CT results. Excluded were 13 more subjects because the initial blood sample draw was past the 6-h time window, leaving a study sample size of 251 patients for this main analysis. Of those, only 231 subjects had an adequate sample volume for the analysis of all three markers (20 patients did
FIG. 1. Flow diagram describing excluded and included patients with mild to moderate traumatic brain injury. CT, computed tomography; UCH-L1, ubiquitin carboxyl-terminal hydrolase-L; GFAP, glial fibrillary acidic protein.
206 WELCH ET AL.
not have enough blood sample volume to test S100B). Figure 1 outlines the study patient selection process.
The mean age of all patients was 45.6–18.4 years and 60.2%
(95% CI; 53.8%–66.3%) were male. The main mechanisms of injury were falls and/or motor vehicle crashes. Of the 251 observations, 206 observations were classified as negative by both primary radiologists, 33 were classified as positive by both primary radiologists, and 12 received discrepant positive and negative classifications from the two primary radiologists (95% agreement; Kappa=0.82, 95% CI 0.71–
0.91). After final diagnosis, adjudicated by a third independent radi- ologist, 36 patients (14.3%; 95% CI 10.3%–19.3%) had a CT scan that was positive for an acute intracranial lesion. Table 3 shows the
findings among patients with positive head CT scan results. Note that the number of findings add up to more than 36 because many patients had more than one CT abnormality.
All except one of the head CTs were positive for some type of blood (ICH); one patient had a nonhemorrhagic contusion. Of the 251 study patients, 225 (89.6%; 95% CI 85.2%–93.1%) had an initial GCS of 15 of whom 24 (10.7%) had a positive CT scan. Among patients with a GCS<15 (n=26), 12 (46.2%) had a positive CT scan.
Median values for all biomarkers were higher among CT positive patients. Table 4 details these and other patient characteristics.
The Spearman correlation coefficients were 0.49 for GFAP and UCH-L1, 0.37 for GFAP and S100B, and 0.77 for UCH-L1 and
Table4. Patient Characteristics
CT negative 85.7% (totaln=215) CT positive 14.3% (totaln=36) All subjects (totaln=251)
Age (mean–SD) 44.3–18.1 53.7–17.7 45.6–18.4
Sex
Female 41.4% (89) 30.6% (11) 39.8% (100)
Male 58.6% (126) 69.4% (25) 60.2% (151)
Race
White 66.5% (143) 91.7% (33) 70.1% (176)
Black 27.4% (59) 2.8% (1) 23.9% (60)
Other 6.1% (13) 5.6% (2) 6.0% (15)
Employment status (n=243)
Employed 47% (98) 41% (14) 46% (112)
Unemployed 23% (49) 18% (6) 23% (55)
Student 9% (19) 0% (0) 8% (19)
Homemaker 1% (3) 0% (0) 1% (3)
Retired 19% (40) 41% (14) 22% (54)
Alcohol
Yes 24.2% (52) 33.3% (12) 25.5% (64)
No 75.8% (163) 66.7% (24) 74.5% (187)
Drugs
Yes 15.4% (33) 5.6% (2) 13.9% (35)
No 84.7% (182) 94.4% (34) 86.1% (216)
Smoke
Yes 31.6% (68) 33.3% (12) 31.9% (80)
No 68.4% (147) 66.7% (24) 68.1% (171)
Mechanism of injury (n=250)O
MVC 39% (84) 17% (6) 35.9% (90)
Assault 12% (25) 11% (4) 11.6% (29)
Fall 50% (107) 75% (27) 53.4% (134)
Sports 3% (7) 0% (0) 2.8% (7)
Loss of consciousness (LOC)
Yes 63.7% (137) 75.0% (27) 65.3% (164)
No 32.6% (70) 19.4% (7) 30.7% (77)
Unknown 3.7% (8) 5.6% (2) 4.0% (10)
LOC information (n=250)
Self-reported 57.0% (122) 36.1% (13) 53.8% (135)
Witnessed 43.0% (92) 63.9% (23) 45.8% (115)
GCS
15 93.5% (201) 66.7% (24) 89.6% (225)
14 5.1% (11) 16.7% (6) 6.8% (17)
13 0.5% (1) 11.1% (4) 2.0% (5)
9–12 0.9% (2) 5.6% (2) 1.6% (4)
GFAP (median; 25th, 75th) 7.8 (2.7, 22.1) 110.5 (20.4, 431.8) 10.3 (3.5, 37.4)
UCH-L1 (median; 25th, 75th) 56.2 (24.4, 104.3) 132.3 (98.2, 269.2) 65.8 (39.6, 125.2)
S100B (median; 25th, 75th) 100 (70, 190) 215 (160, 410) 120 (70, 230)
CT, computed tomography; SD, standard deviation; MVC, motor vehicle collision; GCS, Glasgow Coma scale; GFAP, glial fibrillary acidic protein;
UCH-L1, ubiquitin carboxyl-terminal hydrolase-L.
GFAP, UCHL-1, AND S100B TO CLASSIFY HEAD CT FINDINGS 207
FIG. 2. Scatter plots for glial fibrillary acidic protein (GFAP), ubiquitin carboxyl-terminal hydrolase-L (UCH-L1), and S100B stratified by computed tomography (CT) results. (A) All biomarker values; (B) biomarker values£1200 pg/mL; (C) biomarker values£400 pg/mL.
Color image is available online at www.liebertpub.com/neu
208
FIG. 3. Scatter plots for glial fibrillary acidic protein (GFAP), ubiquitin carboxyl-terminal hydrolase-L (UCH-L1), and S100B stratified by presenting Glasgow Coma Scale (GCS) score (all values£400 pg/mL for resolution). (A) GFAP; (B) UCH-L1; (C) S100B.
CT, computed tomography. Color image is available online at www.liebertpub.com/neu 209
S100B. There were 45 subjects who had GFAP levels below de- tectable levels (4 had a positive CT scan), 2 patients had UCH-L1 levels below detection (none had a positive CT scan), and all S100B values obtained were in the detectible range.
Figure 2A shows scatter plots of the three biomarker results. The plots include all individual patients stratified by CT result. So the reader can better examine for potential false-negative values, only those patients with values less than or equal to 1200 pg/mL (Fig.
2B) and 400 pg/mL (Fig. 2C) for each biomarker are shown. This effectively reduces the scale on the y-axis for the corresponding biomarker also stratified by CT scan result. Figures 3A–C show similar scatter plots for all three biomarkers, but in this case stra-
tified by the Glasgow Coma Scale score; with all values capped at 400 pg/mL for better visual resolution. For these figures, only biomarker values less than or equal to 400 pg/mL were included because the lower values are of most importance.
All three biomarkers had similar values for the AUROC curves:
0.75 (95% CI; 0.65–0.85) for S100B, 0.79 (0.70–0.88) for GFAP, and 0.80 (0.71–0.89) for UCH-L1. When a direct comparison was performed, neither GFAP nor UCH-L1 differed significantly from S100B (p=0.21 andp=0.77, respectively; see Fig. 4A–D). The slight difference in AUROC curve values for the biomarkers when calculated individually was a result of the logistic regression pro- cedure excluding 20 patients with missing values for S100B FIG. 4. Area under the receiver operating characteristic (ROC) curve comparison for all (A) biomarkers in the model (n=231) and for each individual marker (n=251 for glial fibrillary acidic protein [B] and ubiquitin carboxyl-terminal hydrolase-L [C] andn=231 for S100B [D]). Color image is available online at www.liebertpub.com/neu
210 WELCH ET AL.
(complete case analysis), but the results were nearly identical. We also examined the predictive values of two biomarkers compared with one marker alone and, regardless of the pairing (for example, GFAP and UCH-L1 compared with either marker alone), the AUROC curves were no more predictive than the AUROC curve of a single biomarker.
Threshold values for the biomarkers were determined for our study population that provided for high sensitivity at the expense of specificity. For our primary outcome, UCH-L1 was 100% sensi- tive and 39% (95% CI 33%–46%) specific at a value‡40 pg/mL (specificity was 40%, 95% CI 33%–47% when using a cutoff of 41 pg/mL; Table 5). In contrast, GFAP was 100% sensitive and 0% specific at a cutoff of 0 pg/mL, indicating that using the GFAP value associated with 100% sensitivity within 6 h of injury, the test could not reliably determine which patients had negative head CTs. At a sensitivity of 100%, S100B had a specificity of only 2%. In addition, S100B was only 91% sensitive and 44%
specific at the recommended 100 pg/mL threshold. The combined biomarker strategy (both GFAP and UCH-L1) resulted in the same sensitivity and specificity estimates as that of UCH-L1 alone.
Table 5 provides a summary of the sensitivity and specificity at a variety of biomarker threshold values (with associated 95% CIs) and details the threshold biomarker values determined for sensitivity
‡95% and‡90% and the same resulting test characteristics for pa- tients evaluated at 4 h post-injury.
Of the three patients with abnormal CT findings that would have been missed by S100B (<100 pg/mL), one had a subdural hema- toma; the second patient had a subdural hematoma with a cortical hemorrhagic contusion and subarachnoid hemorrhage, and the third had a hemorrhagic contusion with subarachnoid hemorrhage and depressed skull fracture. For two patients, the mechanism of injury was a fall and the third was physically assaulted. All three presented with a GCS score of 15.
When implementing a combined strategy of all three markers (positive if GFAP‡100, UCH-L1‡40, or S100B‡100), the re- sulting sensitivity remained at 100%, but specificity decreased to 0.29 and 0.26 at 6 h and 4 h, respectively (not in table).
Finally, we examined the results for and GFAP and UCH-L1 when only considering the 231 patients having all biomarker data. For GFAP, specificity was 0.00, but for UCH-L1, 100% sensitivity was obtained at a concentration of 40.9 pg/mL (95% CI 0.89–1.00) and sensitivity was 0.40 (95% CI 0.33 to 0.47), neither reflecting a selection bias.
Discussion
This prospective observational study evaluated three biomarkers for potential usefulness to exclude an acute intracranial injury as determined by CT scan in patients with mild to moderate TBI.
Both S100B and GFAP have been studied in patients with mild to moderate TBI with the former being extensively evaluated. For UCH-L1, however, only limited information in patients with mTBI Table5. Sensitivity, Specificity, and95%Confidence Intervals for Biomarkers at6and4Hours Post-Injury
Biomarker
Threshold for a
positive test (pg/mL) Sample size Sensitivity (95% CI) Specificity (95% CI)
6 hoursO GFAP & UCH-L1N 100 GFAP 40 UCH-L1
251 1.00 (0.90, 1.00) 0.39 (0.33, 0.46) 4 hoursO GFAP & UCH-L1N 100 GFAP
40 UCH-L1
251 1.00 (0.89, 1.00) 0.37 (0.30, 0.44)
6 hoursO GFAPU(=100%) 0 251 1.00 (0.90, 1.00) 0.00 (0.00, 0.02)
GFAP (‡95%) 0 251 1.00 (0.90, 1.00) 0.00 (0.00, 0.02)
GFAP (‡90%) 0 251 1.00 (0.90, 1.00) 0.00 (0.00, 0.02)
GFAP 15 251 0.81 (0.64, 0.92) 0.67 (0.61, 0.74)
UCH-L1U(=100%) 41 251 1.00 (0.90, 1.00) 0.40 (0.33, 0.47)
UCH-L1 (‡95%) 44 251 0.97 (0.85, 1.00) 0.42 (0.36, 0.49)
UCH-L1 (‡90%) 67 251 0.92 (0.78, 0.98) 0.58 (0.51, 0.65)
UCH-L1 35 251 1.00 (0.90, 1.00) 0.34 (0.28, 0.41)
S100BU(=100%) 30 231 1.00 (0.89, 1.00) 0.02 (0.00, 0.04)
S100B (‡95%) 80 231 0.97 (0.84, 1.00) 0.30 (0.24, 0.37)
S100B (‡90%) 120 231 0.91 (0.75, 0.98) 0.54 (0.47, 0.61)
S100Ba 100 231 0.91 (0.75, 0.98) 0.44 (0.37, 0.51)
4 hoursO GFAPU(=100%) 0 251 1.00 (0.89, 1.00) 0.00 (0.00, 0.02)
GFAP (‡95%) 0 251 1.00 (0.89, 1.00) 0.00 (0.00, 0.02)
GFAP (>90%) 4 251 0.90 (0.74, 0.98) 0.30 (0.24, 0.37)
UCH-L1U(=100%) 44 251 1.00 (0.89, 1.00) 0.40 (0.33, 0.48)
UCH-L1 (‡95%) 64 251 0.97 (0.83, 1.00) 0.54 (0.47, 0.62)
UCH-L1 (‡90%) 74 251 0.90 (0.74, 0.98) 0.60 (0.53, 0.67)
S100BU(=100%) 80 231 1.00 (0.88, 1.00) 0.28 (0.22, 0.35)
S100B (>95%) 90 231 0.96 (0.82, 1.00) 0.36 (0.29, 0.44)
S100B (‡90%) 120 231 0.93 (0.76, 0.99) 0.53 (0.45, 0.60)
S100Ba 100 231 0.93 (0.76, 0.99) 0.42 (0.34, 0.49)
CI, confidence interval; GFAP, glial fibrillary acidic protein; UCH-L1, ubiquitin carboxyl-terminal hydrolase-L1
NIf either GFAP UCH-L1 was above the threshold, it was considered to be a positive test. If both were below the threshold, the test was negative.
OTime from reported injury to blood sample obtained.
USensitivity and specificity results for each target sensitivity level (100%,‡95%, and‡90%) and resulting biomarker threshold values.
aThis is at the recommended threshold of 100 pg/mL for S100B.
GFAP, UCHL-1, AND S100B TO CLASSIFY HEAD CT FINDINGS 211
is available, particularly related to early screening for CT abnor- malities in patients with mild to moderate TBI.
Therefore, we conducted the first study designed to evaluate test characteristics of the three biomarkers in the same patient cohort with the goal of determining optimal testing strategies and applying the results to a larger cohort of patients with mild to moderate TBI.
Our results demonstrate that early biomarker screening for patients with mild to moderate TBI potentially can provide the clinician with objective evidence needed to reduce CT use.
This study was not designed to determine biomarker capability to differentiate patients with or without any brain injury but rather to differentiate patients with head injury who have a negative versus positive head CT scan. Therefore, it is likely patients en- rolled in this study who had a negative head CT scan could still have had neuronal and glial cellular damage that resulted in ele- vated biomarker levels. This and other factors may have led to the high degree of overlap in biomarker levels (particularly with GFAP and S100B) between CT negative and CT positive patients. In fact, a recent study reported that MRI during the subacute phase (1–2 weeks) post-injury detected injuries in more than 20% of patients who had negative CT finding on initial evaluations. This indicates that blood biomarkers coupled with advanced MRI imaging are complimentary in the diagnosis of mTBI cases.20,29
Because there is no true ‘‘gold-standard’’ for the early diagnosis of mTBI, there still may be utility in considering the possibility that patients with elevated biomarkers may have sustained actual brain injury despite having a negative CT scan. This concept requires further study.
Of the three biomarkers evaluated in this study, UCH-L1 had the best test performance when used to differentiate between subjects with normal versus abnormal CTs. Within 6 h of injury, a UCH-L1 level above 40 pg/mL detected all 36 patients with an acute intra- cranial lesion as determined by a reference CT scan. Given the estimate of 39% specificity, we could have potentially eliminated 83 (95% CI 70–98 scans) of the 215 negative CT scans performed in our study population. The sensitivity and specificity when com- bining GFAP and UCH-L1 was the same as UCH-L1 alone, indi- cating that UCH-L1 was the driving biomarker at the given 6-h time interval. The value of these tests appears to be in the ability of a specified threshold value to reduce the need to obtain a head CT (‘‘rule-out’’ an abnormal CT) among patients with mild to moderate TBI as opposed to using a high value to ‘‘rule-in’’ a positive scan.
Three patients with abnormal CT findings would have been missed by S100B (<100 pg/mL) at the 6-h time limit. These three patients presented with a GCS score of 15, had important injuries, and would require at least observation. Because this study was not designed to determine need for surgery, we do not know whether the patients required surgery or other invasive neurosurgical procedures.
Analysis of biomarker test characteristics using AUROC curve analysis did not reveal any significant difference between S100B and either UCH-L1 or GFAP. AUROC curves, however, reflect the combination of sensitivity and specificity across the entire range of each. Rather than focusing on a single summary statistic such as the AUROC curve, it is often more useful to define the most important aspect of the test being studied.27,28,30
For our study, we determined that test sensitivity was the critical test characteristic, and lower levels of specificity would be tolerated.
This was because of thea prioridecision that the defined screening tests should miss little to no cases of acute intracranial lesions. It was hoped that with optimal sensitivity, that the specificity of the bio- marker would still be high enough to safely reduce the number of negative CT scans and the tests would be acceptable to practicing
physicians. Although on this derivation cohort, UCH-L1 had 100%
sensitivity with a fairly good (39%) specificity, the study sample size was not large enough to have a high level of certainty, and a validation study on a larger patient cohort is warranted.
The Scandinavian guidelines for treating adult patients with mild to moderate head injury recommend S100B as an alternative to head CT when evaluating low-risk patients with either a presenting GCS of 14 or a GCS of 15 with suspected or confirmed loss of consciousness or persistent vomiting.17For our patients, S100B was 91% sensitive when using the recommended threshold value of 100 pg/mL at 6 h post-injury.
Lowering the threshold for S100B to 30 pg/mL at 6 h allowed 100% sensitivity (95% CI 90–100) but had a specificity of only 2%
(95% CI 0–4), making the test ineffective for decreasing the use of head CT in this setting. At 80 pg/mL, however, the sensitivity at 6 h was 97% (with a specificity of 30%, and at 4 h, this same value had a sensitivity of 100%, with a specificity of 28%. Both of these specificities are less than the 39% specificity of UCH-L1 at the cutoff value of 40 pg/mL.
Given the results of previous studies,13,20it was somewhat sur- prising that GFAP did not perform as well, but this may be because the samples were collected before the bigger rise in serum GFAP levels (data not shown). One recent study suggested that elevated GFAP levels provided a diagnostic benefit above clinical screening alone, but this study included GFAP results that were obtained up to 24 h after mild, moderate, and severe TBI.20Our study included only patients with mild and moderate TBI who were evaluated much earlier (within 6 h of injury), making our results more ap- plicable and useful for ED screening for abnormal CT scans in patients with head injuries.
Results of biomarker testing performed within 4 h of injury showed a similar sensitivity for UCH-L1 with specificity that re- mained reasonably good (37%). For S100B at the same 4 h time window, sensitivity was 93% (lower 95% CI 76%) and specificity was 42%. Although this specificity would reduce the number of unnecessary CT scans, the lower sensitivity is unlikely to be ac- ceptable to justify the elimination of head CTs in these patients.
Experts and professional societies have suggested that the in- creasing use of head CT to evaluate patients with mTBI is unjus- tified.1,2,7,17Clinical decision rules that can guide physicians on the use of head CT are well developed and studied3,4,31,32but have not seemed to result in decreased utilization of inappropriate CT ex- aminations among patients with mTBI.1,33
A simple, sensitive, and rapid screening test that could be used in conjunction with validated clinical decision rules has the potential to reduce the use of CT scans for patients with mTBI and the associated radiation exposure and financial costs. It has been sug- gested that radiation has the potential to cause a small increased cancer risk to humans.5 Recognition of these potential risks has resulted in an initiative by the Food and Drug Administration to reduce radiation exposure as well as publish practice guidelines to minimize unnecessary CT examinations.34
The results of the current study provide evidence that assess- ments of serum levels of GFAP and UCH-L1 may achieve this goal by predicting, with a high sensitivity, acute intracranial lesions detected by CT while still preserving sufficient specificity to pro- vide an important reduction in CT use.
Limitations
There were a number of limitations that must be considered when evaluating this study and its results. Because this study was
212 WELCH ET AL.
designed to evaluate potentially optimal cutoff values for the bio- markers, performance of the biomarkers may be inflated. Derived CIs for specificity (and proportion of eliminated unnecessary CT scans) was conditioned on both the subjective decision of the CT scan and on the choice of an appropriate threshold with 100%
sensitivity, and do not reflect the variability inherent in those two processes. External validation of these potential thresholds is needed to assess the robustness of these findings.
Our study found that patients presenting with an initial GCS<15 had a high proportion of positive head CTs. Over a third of patients (6 of 17) with a GCS of 14 had positive scans. This is likely because of our small number of patients and the relatively high percentage of positive CT scans and may not represent the usual group of patients with mTBI who present with an initial GCS of 14. Gen- eralizability of these results is limited to the study population—i.e., patients with an initial GCS of 9–15 who would typically be re- commended by the ED physician for emergency head CT scan.
The definition of a positive CT included findings other than just traumatic ICH, but we did exclude isolated skull fracture from the definition. Practice variation led to the inclusion of a relatively high percent of patients with positive head CTs relative to that of pa- tients enrolled in U.S. sites. Given patients were enrolled in the United States and in Europe, selection of patients varied across sites. The majority of subjects had a GCS of 15, so these results may not apply to patients with a GCS score less than 15.
Finally, this study was not designed to determine clinical outcomes such as need for neurosurgical intervention or death but rather just to determine the association of specific biomarker values with a positive finding on head CT—an important first step in reducing CT use.
Conclusion
Our data suggest that when obtained within 6 h of injury, UCH- L1 and the combination of GFAP and UCH-L1 (at our pre-defined cutoff values) were very sensitive for a positive CT scan of the head among patients with mild to moderate TBI. If confirmed in a large trial with greater numbers of positive CT scans, UCH-L1 and the combination of GFAP and UCH-L1 could provide the objective evidence clinicians desire to reduce the use of CT scans among patients with mTBI. Further study related to other times post-injury and the utility of repeated test measurements will help further clarify the utility of these biomarkers.
Acknowledgment
This work is supported by the U.S. Army Medical Research and Material Command under Contract N. W81XWH-06-1-0517 and Contract N. W81XWH-10-C-0251. The views, opinions and/or findings contained in this report are those of the authors and should not be construed as an official Department of the Army position, policy or decision unless so designated by other documentation.
ClinicalTrials.gov Identifier: NCT01295346.
Banyan Biomarkers, Inc. personnel include Art Weber, Anna Shoshoo, Mike Catania, and Aarti Chawla.
Author Disclosure Statement No competing financial interests exist.
References
1. Melnick, E.R., Szlezak, C.M., Bentley, S.K., Dziura, J.D., Kotlyar, S., and Post, L.A. (2012). CT overuse for mild traumatic brain injury. Jt.
Comm. J. Qual. 38, 483–489.
2. Korley, F.K., Morton, M.J., Hill, P.M., Mundangepfupfu, T., Zhou, T., Mohareb, A.M., and Rothman, R.E. (2013). Agreement between routine emergency department care and clinical decision support re- commended care in patients evaluated for mild traumatic brain injury.
Acad. Emerg. Med. 20, 463–469.
3. Stiell, I.G., Wells, G.A., Vandemheen, K., Clement, C., Lesiuk, H., Laupacis, A., McKnight, R.D., Verbeek, R., Brison, R., Cass, D., Eisenhauer, M.E., Greenberg, G., and Worthington, J. (2001). The Canadian CT Head Rule for patients with minor head injury. Lancet 357, 1391–1396.
4. Haydel, M.J., Preston, C.A., Mills, T.J., Luber, S., Blaudeau, E., and DeBlieux, P.M. (2000). Indications for computed tomography in pa- tients with minor head injury. N. Engl. J. Med. 343, 100–105.
5. Smith-Bindman, R., Lipson, J., Marcus, R., Kim, K.P., Mahesh, M., Gould, R., Berrington de Gonzalez, A., and Miglioretti, D.L. (2009).
Radiation dose associated with common computed tomography ex- aminations and the associated lifetime attributable risk of cancer.
Arch. Intern. Med. 169, 2078–2086.
6. Brenner, D.J., and Hall, E.J. (2007). Computed tomography—an in- creasing source of radiation exposure. N. Engl. J. Med. 357, 2277–
2284.
7. Jagoda, A.S., Bazarian, J.J., Bruns, J.J., Jr., Cantrill, S.V., Gean, A.D., Howard, P.K., Ghajar, J., Riggio, S., Wright, D.W., Wears, R.L., Bakshy, A., Burgess, P., Wald, M.M., and Whitson, R.R. (2008).
Clinical policy: neuroimaging and decisionmaking in adult mild traumatic brain injury in the acute setting. Ann. Emerg. Med. 52, 714–
748.
8. Jeter, C.B., Hergenroeder, G.W., Hylin, M.J., Redell, J.B., Moore, A.N., and Dash, P.K. (2013). Biomarkers for the diagnosis and prognosis of mild traumatic brain injury/concussion. J. neurotrauma 30, 657–670.
9. Egea-Guerrero, J.J., Revuelto-Rey, J., Murillo-Cabezas, F., Munoz- Sanchez, M.A., Vilches-Arenas, A., Sanchez-Linares, P., Dominguez- Roldan, J.M., and Leon-Carrion, J. (2012). Accuracy of the S100beta protein as a marker of brain damage in traumatic brain injury. Brain Inj. 26, 76–82.
10. Zongo, D., Ribereau-Gayon, R., Masson, F., Laborey, M., Contrand, B., Salmi, L.R., Montaudon, D., Beaudeux, J.L., Meurin, A., Dousset, V., Loiseau, H., and Lagarde, E. (2012). S100-B protein as a screening tool for the early assessment of minor head injury. Ann. Emerg. Med.
59, 209–218.
11. Biberthaler, P., Linsenmeier, U., Pfeifer, K.J., Kroetz, M., Mussack, T., Kanz, K.G., Hoecherl, E.F., Jonas, F., Marzi, I., Leucht, P., Jo- chum, M., and Mutschler, W. (2006). Serum S-100B concentration provides additional information for the indication of computed to- mography in patients after minor head injury: a prospective multi- center study. Shock 25, 446–453.
12. Metting, Z., Wilczak, N., Rodiger, L.A., Schaaf, J.M., and van der Naalt, J. (2012). GFAP and S100B in the acute phase of mild trau- matic brain injury. Neurology 78, 1428–1433.
13. Papa, L., Silvestri, S., Brophy, G.M., Giordano, P., Falk, J.L., Braga, C.F., Tan, C.N., Ameli, N.J., Demery, J.A., Dixit, N.K., Mendes, M.E., Hayes, R.L., Wang, K.K., and Robertson, C.S. (2014). GFAP out-performs S100beta in detecting traumatic intracranial lesions on computed tomography in trauma patients with mild traumatic brain injury and those with extracranial lesions. J. Neurotrauma 31, 1815–
1822.
14. Yokobori, S., Hosein, K., Burks, S., Sharma, I., Gajavelli, S., and Bullock, R. (2013). Biomarkers for the clinical differential diagnosis in traumatic brain injury—a systematic review. CNS Neurosci Ther.
19, 556–565.
15. Muller, K., Townend, W., Biasca, N., Unden, J., Waterloo, K., Romner, B., and Ingebrigtsen, T. (2007). S100B serum level predicts computed tomography findings after minor head injury. J. Trauma 62, 1452–1456.
16. Zetterberg, H., Smith, D.H., and Blennow, K. (2013). Biomarkers of mild traumatic brain injury in cerebrospinal fluid and blood. Nat. Rev.
Neurol. 9, 201–210.
17. Unden, J., Ingebrigtsen, T., Romner, B.; Scandinavian Neurotrauma Committee. (2013). Scandinavian guidelines for initial management of minimal, mild and moderate head injuries in adults: an evidence and consensus-based update. BMC Med. 11, 50.
18. Barbosa, R.R., Jawa, R., Watters, J.M., Knight, J.C., Kerwin, A.J., Winston, E.S., Barraco, R.D., Tucker, B., Bardes, J.M., and Rowell, S.E. (2012). Evaluation and management of mild traumatic brain
GFAP, UCHL-1, AND S100B TO CLASSIFY HEAD CT FINDINGS 213
injury: an Eastern Association for the Surgery of Trauma practice management guideline. J. Trauma Acute Care Surg. 73, Suppl 4, S307–S314.
19. Papa, L., Lewis, L.M., Falk, J.L., Zhang, Z., Silvestri, S., Giordano, P., Brophy, G.M., Demery, J.A., Dixit, N.K., Ferguson, I., Liu, M.C., Mo, J., Akinyi, L., Schmid, K., Mondello, S., Robertson, C.S., Tortella, F.C., Hayes, R.L., and Wang, K.K. (2012). Elevated levels of serum glial fibrillary acidic protein breakdown products in mild and mod- erate traumatic brain injury are associated with intracranial lesions and neurosurgical intervention. Ann. Emerg. Med. 59, 471–483.
20. McMahon, P.J., Panczykowski, D.M., Yue, J.K., Puccio, A.M., Inoue, T., Sorani, M.D., Lingsma, H.F., Maas, A.I., Valadka, A.B., Yuh, E.L., Mukherjee, P., Manley, G.T., Okonkwo, D.O., Casey, S.S., Cheong, M., Cooper, S.R., Dams-O’Connor, K., Gordon, W.A., Hricik, A.J., Law- less, K., Menon, D., Schnyer, D.M., and Vassar, M.J. (2015). Mea- surement of the glial fibrillary acidic protein and its breakdown products GFAP-BDP biomarker for the detection of traumatic brain injury compared to computed tomography and magnetic resonance imaging. J.
Neurotrauma 32, 527–533.
21. Wilkinson, K.D., Lee, K.M., Deshpande, S., Duerksen-Hughes, P., Boss, J.M., and Pohl, J. (1989). The neuron-specific protein PGP 9.5 is a ubiquitin carboxyl-terminal hydrolase. Science 246, 670–673.
22. Mondello, S., Muller, U., Jeromin, A., Streeter, J., Hayes, R.L. and Wang, K.K. (2011). Blood-based diagnostics of traumatic brain in- juries. Expert Rev. Mol. Diagn. 11, 65–78.
23. Papa, L., Lewis, L.M., Silvestri, S., Falk, J.L., Giordano, P., Brophy, G.M., Demery, J.A., Liu, M.C., Mo, J., Akinyi, L., Mondello, S., Schmid, K., Robertson, C.S., Tortella, F.C., Hayes, R.L., and Wang, K.K. (2012). Serum levels of ubiquitin C-terminal hydrolase distin- guish mild traumatic brain injury from trauma controls and are ele- vated in mild and moderate traumatic brain injury patients with intracranial lesions and neurosurgical intervention. J. Trauma Acute Care Surg. 72, 1335–1344.
24. Bazarian, J.J., Blyth, B.J., He, H., Mookerjee, S., Jones, C., Kiechle, K., Moynihan, R., Wojcik, S.M., Grant, W.D., Secreti, L.M., Triner, W., Moscati, R., Leinhart, A., Ellis, G.L., and Khan, J. (2013).
Classification accuracy of serum Apo A-I and S100B for the diagnosis of mild traumatic brain injury and prediction of abnormal initial head computed tomography scan. J. Neurotrauma 30, 1747–1754.
25. Florkowski, C.M. (2008). Sensitivity, specificity, receiver-operating char- acteristic (ROC) curves and likelihood ratios: communicating the perfor- mance of diagnostic tests. Clin. Biochem. Rev. 29, Suppl 1, S83–S87.
26. DeLong, E.R., DeLong, D.M., and Clarke-Pearson, D.L. (1988).
Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837–845.
27. Bossuyt, P.M., Reitsma, J.B., Bruns, D.E., Gatsonis, C.A., Glasziou, P.P., Irwig, L.M., Moher, D., Rennie, D., de Vet, H.C., and Lijmer, J.G.
(2003). The STARD statement for reporting studies of diagnostic accu- racy: explanation and elaboration. Ann. Intern. Med. 138, W1–W12.
28. Sauerbrei, W., Abrahamowicz, M., Altman, D.G., le Cessie, S., and Carpenter, J. (2014). STRengthening analytical thinking for observa- tional studies: STRATOS initiative. Stat. Med. 33, 5413–5432.
29. Kou, Z., Gattu, R., Kobeissy, F., Welch, R.D., O’Neil, B.J., Woodard, J.L., Ayaz, S.I., Kulek, A., Kas-Shamoun, R., Mika, V., Zuk, C., Tomasello, F., and Mondello, S. (2013). Combining biochemical and imaging markers to improve diagnosis and characterization of mild traumatic brain injury in the acute setting: results from a pilot study.
PloS One 8, e80296.
30. Korevaar, D.A., van Enst, W.A., Spijker, R., Bossuyt, P.M., and Hooft, L. (2014). Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD. Evid. Based Med. 19, 47–54.
31. Smits, M., Dippel, D.W., de Haan, G.G., Dekker, H.M., Vos, P.E., Kool, D.R., Nederkoorn, P.J., Hofman, P.A., Twijnstra, A., Tanghe, H.L., and Hunink, M.G. (2005). External validation of the Canadian CT Head Rule and the New Orleans Criteria for CT scanning in patients with minor head injury. JAMA 294, 1519–1525.
32. Stiell, I.G., Clement, C.M., Rowe, B.H., Schull, M.J., Brison, R., Cass, D., Eisenhauer, M.A., McKnight, R.D., Bandiera, G., Holroyd, B., Lee, J.S., Dreyer, J., Worthington, J.R., Reardon, M., Greenberg, G., Lesiuk, H., MacPhail, I., and Wells, G.A. (2005). Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA 294, 1511–1518.
33. Stiell, I.G., Clement, C.M., Grimshaw, J.M., Brison, R.J., Rowe, B.H., Lee, J.S., Shah, A., Brehaut, J., Holroyd, B.R., Schull, M.J., McKnight, R.D., Eisenhauer, M.A., Dreyer, J., Letovsky, E., Rutledge, T., Macphail, I., Ross, S., Perry, J.J., Ip, U., Lesiuk, H., Bennett, C., and Wells, G.A. (2010). A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments.
CMAJ 182, 1527–1532.
34. Initiative to Reduce Unnecessary Radiation Exposure from Medical Imaging-February 2010. Center for Devices and Radiological Health- U.S. Food and Drug Administration. Available at:http://www.fda.gov/
downloads/Radiation-EmittingProducts/RadiationSafety/RadiationDose Reduction/UCM200087.pdf. Accessed: October 17, 2015.
Address correspondence to:
Robert D. Welch, MD, MS Department of Emergency Medicine Wayne State University Detroit, MI 48230 E-mail:rwelch@med.wayne.edu
214 WELCH ET AL.
