November 6th was a landmark day in the NFL's ongoing head trauma debacle. We learned that Tony Dorsett, the 59-year-old Hall of Fame running back who'd recently been battling depression and memory problems, had been diagnosed with early signs of chronic traumatic encephalopathy, or CTE. Not long after the story broke, I received a text from a friend that read: You see the Tony Dorsett CTE comments dude? NFL is def. not gonna be around in 10 years.
The diagnosis was unusual because until now, CTE could only be diagnosed by a neuropathologist on a post-mortem examination. But in an effort to become more proactive, researchers at UCLA have developed a brain scan that can potentially diagnose CTE in living patients. This remarkable technology was developed by a company called TauMark, which uses a PET scan in conjunction with a biomarker to identify the abnormal Tau protein that is a hallmark of Alzheimer's Disease, Traumatic Brain Injury, and CTE.
How do you test a test?
As I read the text from my friend I had one very basic, very important question: Is the TauMark Scan actually a good test for CTE? I work in a lab that develops tests to identify new diseases1 and before we bring a new test to the market, we understandably need to know if it actually works. We do that by calculating what's called the sensitivity and specificity of the test. Sensitivity is a statistical measure that tells us if the test correctly identifies sick people with the disease, while specificity tells us if the test correctly rules out people who don't have the disease. (They measure the instances of false positives and false negatives, essentially.) No test is perfect and there is usually a trade-off between them, but the closer these numbers are to 100%, the better the test is.
The classic example used to teach medical students about sensitivity and specificity involves airport security. There are a number of ways to try to identify a passenger with a bomb strapped to his chest and each has a different sensitivity and specificity. A simple metal detector is very sensitive—it will identify the bomb but it will also identify all kinds of other nonsense, including car keys and loose change. In theory, the chance that the bomb gets through the metal detector is low but the specificity of the metal detector is terrible because it fails to allow the average passenger with car keys in his pocket through the detector. In contrast, if we simply ask passengers if they're carrying a bomb, the sensitivity will be poor because those passengers carrying a bomb will presumably lie and the test will fail to correctly identify its target. On the other end of the spectrum, we could strip search every passenger; that would provide a very sensitive, very specific, very annoying way of ensuring that no bombs get through.
So is the CTE test a good one?
Because the TauMark Scan that was used on Tony Dorsett is so new, and because the number of ex-players tested is so small, we don't yet know what the sensitivity or specificity of this test is. The first eight NFL players to be tested with this technology all tested positive for CTE at UCLA. If the researchers go on to test hundreds of patients who never experienced head trauma and have no symptoms, those patients should not light up on their scanner. If they do show signs of CTE, it's a lousy test; if they don't it's an excellent test.
The fascinating group to examine will be ex-NFL players who did suffer concussions but are without symptoms. Will these patients test positive for CTE on the TauMark Scan? And if they do, is it because they're catching CTE before the symptoms have developed or because the scanner has mistakenly diagnosed an ex-player with CTE?
It's also important to note that the Tau protein has also been seen in Alzheimer's Disease. This distinction is critical because there are some treatment options for Alzheimer's that don't exist for CTE; Tony Dorsett currently believes he has a progressive, unrelenting, untreatable disease. The onus is on the physicians to be certain that none of these ex-players are misdiagnosed with CTE when, in fact, they have Alzheimer's Disease or some other neurodegenerative disease.
The quality of the TauMark Scan is crucial because there are a number of rival groups currently developing technology to diagnose CTE in the living. These new assays involve analysis of cerebral blood flow, augmentation of brain architecture, and analysis of chemicals that are secreted by damaged neurons. There are literally billions of dollars at stake and you will see undoubtedly see more tests come to market in the coming years. When they do, you should ask the same question: Is this new test actually a good test for CTE?
If you take away one notion from this, let it be that not all diagnostic tests are created equally. When I was in medical school, I was paired with a young doctor who ordered something called a CA-125 blood test (cancer antigen 125) for a patient with vague abdominal pain whom he thought might have ovarian cancer. (The CA-125 biomarker is often elevated in ovarian cancer and it seemed like a reasonable thing to do). But the next morning, a more senior physician criticized this decision, pointing out that so many benign conditions produce excess CA-125 that the test is essentially useless. We were told that in a great many cases of ovarian cancer, the CA-125 level is normal and that the sensitivity and specificity of the test is so poor that we shouldn't use it.2
Maybe it won't be a test, but tests
It's possible that several tests—potentially from rival pharmaceutical companies—will be necessary to accurately diagnose CTE. This is not to impugn the vital work that's being done at UCLA, it's simply to point out that because the work is on the forefront of science, we don't yet have enough information to determine how good the test is. The TauMark Scan may turn out to be the most sensitive, most specific test for CTE—but also it might not.
The other issue to confront is how we should evaluate future tests. While TauMark is figuring out the sensitivity and specificity of its own test, should Tony Dorsett travel to other facilities to try out emerging new methods? Ideally yes, but not all patients will have the time or resources to participate. And what if there is a discrepancy?
The people I've spoken to are very confident in the TauMark Scan and this seems to come from a combination of gut instinct and an understanding of the science behind the test. Let's hope they're right.
1 Our group at Cornell recently helped develop a DNA test for an outbreak of a rare fungal infection called Exserohilum rostratum in contaminated steroid vials that has affected over 10,000 patients across the United States.
2 The American Congress of Obstetricians and Gynecologists recommend against women with average risk of ovarian cancer having routine CA-125 screening
Matt McCarthy is board-certified in internal medicine. You can follow him on Twitter here.
Image by Jim Cooke