Thanks for visiting. I’ve been writing a blog since 2011, prior to my career shift to medical writing. The previous blog posts can be viewed on Blogger, and are informal and fun posts on a variety of pharmaceutical topics and give a hint to my personality. As I polish up those posts to a more professional style or create new ones, they will be posted here.
This week, we had a very interesting discussion at work about COVID-19 testing, prompted by a tweet from Elon Musk. Even in a group of people I consider brilliant, it was clear we were (including me) a little confused about COVID tests. So I decided to do a little research to answer some questions I had and post the answers with a little bit of context of science and statistics.
The bottom line is: in many cases, COVID-19 testing is a clue, not an answer. It can tell you if you definitely have, most likely have, probably have, probably do not have, most likely do not have COVID-19. Testing cannot tell you that you definitely do not have COVID-19.
Tests are helpful, but are only a piece of the puzzle. The most important things you can do to protect your health and others are to:
If you are well: Wear a mask AND keep >6 feet of distance from those you don’t live with. If you are outside, you should still wear a mask unless your exposure to others is brief and distant.
If you or someone in your household is sick: the sick person should separate completely from anyone else, including those in the house (self-isolate), and the rest of the household should stay home (quarantine).
The CDC has a Self Checker to help you make testing decisions, and the FDA has a helpful video.
What is the difference between the types of tests?
Types of tests for COVID-19: swabs indicate tests done by swab up the nose or through the mouth, the syringe indicates a blood test by finger stick or through the arm
Test type
PCR test
Other names
sometimes called nucleic acid test, qPCR, RT-PCR, molecular test, genetic test, diagnostic test, viral test, nucleic acid amplification test (NAAT), LAMP test
What if I get a positive result?
If you get a positive result, you have COVID-19. This test is the “gold standard” – PCR is a sensitive test that is very specific for COVID-19. Testing positive for COVID-19 without being infected is theoretically possible due to a lab mistake, but unlikely. You should self-isolate and people who live with you should quarantine for as long as your health care provider or health department recommends.
What if I get a negative result?
There is still a chance you have COVID-19. This test can only find COVID-19 if the swab picks up enough virus to test. You could have growing amounts of virus that the swab did not pick up. You should talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should be retested at a later date and if you should self-isolate or quarantine.
Test type
Antigen test
Other names
sometimes called rapid test, viral test, diagnostic test
What if I get a positive result?
If you get a positive result, you may have COVID-19. The likelihood of you having COVID-19 depends on how many people in your community have the virus (see below for explanation) and your exposure. You should self-isolate and your household should quarantine, but you may need to follow-up. Talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should have a PCR test.
What if I get a negative result?
There is still a chance you have COVID-19. This test can only find COVID-19 if the swab picks up enough virus to test. You could have growing amounts of virus that the swab did not pick up. You should talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should have a PCR test and if you should self-isolate or quarantine.
Test type
Antibody test *This test does not test for COVID-19 infection. This test is best used for research purposes, or to find out if your current health problems could be due to a recent COVID-19 infection where the virus is no longer able to be found by other tests.*
Other names
Serological test, serology, blood test, serology test
What if I get a positive result?
You have antibodies that are similar to the ones made to fight COVID-19. You may currently have a COVID-19 infection or may have had one in the recent past. You could also have been infected with a virus that was similar to COVID-19. You should get a COVID-19 PCR or antigen test to find out if you still have an active infection. If you do not have an active infection, a positive antibody test may not mean you are immune to COVID-19. You should continue to take precautions to avoid spreading COVID-19. Consider donating plasma to help others.
What if I get a negative result?
This does not mean you do not have COVID-19. This test does not test for the presence of the virus, and your body may not have made antibodies to the virus yet. If you need to test for COVID-19, you need a PCR or antigen test. You may not have ever had COVID-19. If you never had exposure or symptoms, this is the most likely answer. You may have had a COVID-19 infection in the past. Antibody levels can drop over time, so you could have had an infection but no longer have antibodies.
Why can’t the tests tell me that I definitely don’t have COVID-19?
There is a curve of infection that happens after you are exposed to COVID-19. The virus is in your body and multiplying. Once the virus gets high enough, sensitive PCR tests find the genetic material inside the virus shell and less sensitive antigen tests find pieces of the virus shell. As you recover from the virus, antibodies appear and the virus level decreases. Over time, the antibodies also disappear.
All tests have limits for how little of the virus they can find, called the limit of detection. Depending on when you are tested, your level of virus may be too low for the test to find. If you were tested earlier or later, you may get a different result because your levels of virus got higher or lower. At first, you could test negative because there is not yet enough virus at the spot they swab to test above the limit of detection. The date you were exposed and what symptoms you are experiencing are important clues to where you are on the curve.
In the early days of infection, your virus level could be negative because it is below the limit of detection of the test.
If your levels of virus are in a zone where the test is not accurate, you could get inconclusive results, which is likely what happened in Elon Musk‘s case.
This is the reason why if you have been in close contact with a COVID-19 case, the CDC still recommends you self-isolate for 14 days even if you have a negative test and do not have symptoms.
Why does the amount of positive cases in my community affect how I interpret my antigen test?
Antigen tests are an important tool in our battle against COVID-19, but due to the way the tests work, there is a possibility of testing positive when you do not have the virus (false positive) and testing negative when you do have the virus (false negative).
Due to an interesting quirk of statistics, your chance of getting a false positive increases when the chance of you catching COVID-19 is low. Your chance of a false positive decreases when the chance of you catching COVID-19 is high. How is this possible? The opposite is true for false negatives – when your chance of catching COVID-19 is high, you are more likely to get a false negative.
BD, who manufactures the Veritor™ Plus System for rapid COVID-19 (SARS-CoV-2) testing, has a nice graphic of this phenomenon on their webpage. They state here that their test has a specificity of 99.5%, which means 1 out 200 tests will be a false positive. They also state that their test has a sensitivity of 84%, which means if we used the test on 200 people who are all later confirmed to have COVID-19, 168 will test true positive, and 32 will have false negatives.
Positive predictive value is the chance that your positive is a true positive. Prevalence means how many people in your community have an active case of COVID-19. As you can see, when there are very few COVID-19 cases in your area, your chance of your positive result being correct could be as low as 9.1%! However, if your chance of COVID-19 infection is higher (due to high community spread or known exposure), your chance of your positive result being correct is >90%.
This seems to make no logical sense. The way the test works should not change based on what is going on in the outside world, right?
Let’s imagine we mail this antigen test to a secluded island nation that has never been exposed to COVID-19. The test is accurate 99.5% of the time, which means for every 200 tests performed, 1 will be a false positive. So, residents of this island who test positive have a 100% chance that it is a false positive. There are no false negatives or true positives in this case because COVID-19 does not exist on the island.
Now, let’s look at the state of NC (DHHS testing dashboard). Of the people who are getting tested for COVID-19, 7.9% are testing positive by PCR tests. (People getting tested are more likely to have COVID-19 than NC residents as a whole, because people getting tested are more likely to have had exposure or symptoms.) If we were to take 200 of those people in NC who had a PCR test in the last few days, 16 of them would have COVID-19. If we tested all 200 with the BD antigen test, 14 people would test positive. Of those, 1 would still be false positive from the test as we saw above. Because this test catches 84% of positives, 3 of the positive people will test false negative and the remaining 13 will test true positive. For those who tested positive, their chance of having COVID-19 is 93%. Their chance of a false positive is 7%. For those who tested negative, their chance of having COVID is 2%.
Now, let’s look at a hypothetical case where a workplace in Sampson County, NC wanted to test all 200 of their employees with the rapid antigen test, even though there had not been any known exposure. According to NC DHHS, there have been 538 cases per 100,000 residents in the last 14 days, which is about 1 person out of 200 people. We would still have our 1 false positive from the test, and 1 true positive. So of the 2 people who receive positive results, they each have a 50% chance of having a false positive.
On average, the chance of a false negative is slim – but if just 1 person tested false negative due to being at an early but still contagious stage of infection, having a party based on these results could get several people sick.
Now let’s imagine a super spreader event. There was a large indoor wedding where people sang together, laughed, and yelled to hear each over the music for hours. They hugged and shook hands while enjoying appetizers. Of the 200 people who attended, 130 people were later confirmed COVID-19 positive by the PCR test. If we tested all 200 attendees with the BD antigen test, 110 people would test positive. Of those, 1 would still be false positive from the test as we saw above. Because this test catches 84% of positives, 21 of the positive people will test false negative and the remaining 109 will test true positive. For those who tested positive, their chance of having COVID-19 is 99%. Their chance of a false positive is 1%. For those who tested negative, their chance of having COVID is 23%.
In each case, the 1 false positive out of 200 tests stays the same – but the number of total COVID-19 positive cases changes, which causes the chance of a positive actually being a false positive to change.
After seeing these examples, it’s easier to understand why the antigen test is recommended for people with exposure or symptoms of COVID-19 and not for general screening, and also why preventative isolation and a follow-up PCR test is important in cases where the risk of a false negative is high. Results from general screening may be useful to determine trends, and may catch some true positive cases, but it also may give a sense of false security because it does not catch early cases (as in the now famous event at the White House Rose Garden) and may sideline false-positive people who are not sick.
An important note on false negatives – unlike false positives, they are due to a combination of factors related to where you are in your COVID-19 course of infection and factors related to the test itself. Although BD reports an average false negative rate – your personal risk of a false negative rate may be higher or lower depending on your personal circumstances. Also, although the PCR test is much less likely to have false positives, false negatives like the example above can happen with the PCR test as well.
Does this mean that the number of positive tests reported in the media is wrong?
Each state has their own method of testing and reporting results. These results will always be an estimate because not everyone gets tested. In states that report antigen results, there is a possibility that false positives will get reported. The best state health departments can do is set clear rules for testing and reporting results and look at trends over time. In NC, DHHS reports antigen results separately. Right now they are <5% of the total.
If you have doubt that the increase in the cases is real, you can also look at hospitalization and death trends.
How should I interpret my test results?
Be informed and ask your health care provider to help you interpret the results based on your personal situation.
Err on the side of caution. If you have symptoms or confirmed exposure, follow CDC guidelines on how to keep others safe.
Do not use negative test results to have a maskless gathering. It’s safer to quarantine 14 days before getting together, keep masks on, or have a virtual visit.
Do not use a previous positive PCR, antigen, or antibody test result to act “immune.”
Pharmaceutical professionals keep one foot in science and another foot in health authority regulation, both rapidly evolving fields that require consistent training to keep pace. In addition, new opportunities often entice scientists to widen their expertise to contribute to new areas of drug development, which may require an entirely new set of skills. At some point, you are likely to reach the limit of what your corporate or personal budget for training allows, and at that point this list is just what you are looking for: quality training that is either free or subsidized!
These courses are ones I have taken and felt were worth the time investment. This list is current as of February 2021.
This course (IPPCR) is offered by the NIH to train new clinical investigators from October to June of each year. It covers all aspects of clinical trials including design, analysis, reporting, budgeting, regulations, and ethics. The textbook is available for about $75 and the course is free, self-paced, and entirely online. If you pass the final exam you will earn a certificate of completion. This course is a significant time commitment, there are about 40 lectures and most are 60-90 minutes, and you will need to allot additional time to reading the assignments and studying for the exam. I highly recommend going through Statistics on Khan Academy (see below) to prepare for the biostatistics in this course.
This 6-week course offered by the Office of Regulatory Affairs and Quality (ORAQ) at Duke University is free and available online via WebEx. This course is not self-paced, there are 1-hour webcasts from 12-1 pm EST on Fridays. They take attendance and there is reading and homework.
ORAQ also offers free seminars on regulatory topics that you can join via WebEx.
This online course consists of 6 self-paced modules, which took me about 6 weeks (they estimate 8 weeks). It’s a great overview of the drug approval process and the history of the FDA. It’s presented by the Program on Regulation, Therapeutics, and Law at Harvard and Brigham and Women’s Hospital. It is free to audit, or $199 to get a certificate. It is self-paced, so you start anytime enrollment is open.
FDA training
The FDA offers free online resources, including the online courses below.
If you are completely new to medical writing, this course will provide a complete overview to the field. It’s available online through ed2go through partnership with community colleges (I took it through Wake Community College for less than ed2go charges).
When writing up adverse events, you need to learn a whole other language. If you are new to terms like pyrexia, dyspnoea, and tachycardia, take this brief online self-paced course on the Latin and Greek behind medical terminology. It takes about 2 hours, and is free if you don’t require a certificate.
Government communications are required to be written in plain language, but some writers are unclear what plain language means. This free online self-paced course from the National Institutes of Health (NIH) will help you understand federal requirements for plain language.
Membership to AMWA will cost you $199/year ($80 for students), but offers a lot of educational perks for medical, scientific, or regulatory writers. AMWA offers a variety of paid courses, but there are some that are free for members (search for “complimentary“), and they email out a monthly free webinar (one that is usually paid) for members as well. You also get an included subscription to the AMWA Journal and all back issues online. Membership in AMWA also includes free chapter events that are a great bargain if you live near an active chapter (if you live in NC or SC, check out @AMWACarolinas on Twitter!).
Membership in the North Carolina Regulatory Affairs Forum is only $40 and includes 6 seminars per year on regulatory topics that are available by WebEx. They also offer a summer workshop (at an additional very reasonable fee) to prepare for the Regulatory Affairs Certification (RAC) exam.
This podcast is provided by Emma Hitt Nichols of Nascent Medical to promote her business and her 6-week course. She has many interviews with medical writers, and through listening you can learn about the many career paths available to medical writers as well as many tricks of the trade.
This podcast, sponsored by PSI, is excellent for statisticians and those who work with statisticians, and some episodes on leadership and influence are applicable to anyone in pharma.
Take a whirlwind tour of the past, present, and future of cancer in episode #62 of Peter Attia’s podcast. I’ve listened to this one a few times because so much interesting detail is packed into every minute of this episode.
The European Medical Writer’s Association journal published an entire issue devoted to regulatory writing in 2014, and it’s free for non-members to read.
Trilogy has several publications written by expert medical writers on a variety of topics.
Emma Hitt Nichols’ book on Freelance Medical Writing
Emma broadcasts a free webinar regularly to promote her 6-week course. The course is not free, but when I attended her webinar she provided her book for free to attendees.
Know of a great cheap or free course that I should list here? Contact me through the contact link above and let me know! Happy learning!
Scientists
are well versed in experimental bias, which is why we address it by using
experimental controls, masking our clinical trials, and using the scientific
method to approach questions. However, how do we control for bias within our own minds? Cognitive bias refers to any number
of ways that our brain prevents us from making entirely objective
decisions. In an article that Harvard Business Review published in June
2011, “Before You Make That Big Decision…”, several
types of cognitive bias are defined and discussed along with case studies, and
a 12-step checklist to root out bias is defined.
Major
decisions in pharmaceuticals are impacted
by cognitive bias. When developing a product, there are a million
decisions that can have a significant impact on the cost, timescale, clinical
success, and eventual marketability of your product. Many of these
decisions are originally made at the
bench level, and may not be able to be changed without considerable additional
time or expense as the project progresses through later stages of development.
For
example, a formulator may demonstrate a bias for a particular type of
formulation process because of previous experience and comfort, or the wish for
high visibility through the use of trendy new technology, or convenience
according to what equipment is on site and available. Decision makers
should recognize the potential for this bias and make sure the best formulation
is chosen regardless of the above
factors. Once this formulation makes it into human studies, there is
considerable inertia that makes change difficult, since the project team
doesn’t want to delay timelines by having to repeat animal studies or bridge
with additional human pharmacokinetic studies.
Bias
can be very costly to big pharma companies,
but attempts to avoid bias are not without cost. Multiple layers of peer
review, involving Marketing early in development where most compounds fail for
other reasons, and execution of checklists also take time, but could save
billions for that one “blockbuster in the rough.”
According
to the article, it is nearly impossible to detect your own bias, but through
learning about bias, we can better detect it in our peers and use this knowledge
to better challenge decisions. For example, when performing due diligence,
you must be alert for bias from the company under scrutiny, the fellow members
of your team, and in how your team prioritizes and reports the findings.
Here
are some types of bias from the article and how they could come up in pharma:
Self-interested Bias
This type of bias is hard to avoid. Almost every person on a project team is heavily vested in the success of their project. Part of this is due to corporate culture, which tends to reward those people who happen to be on successful projects. This bias can be minimized by shifting the focus from project success, which can be largely due to the luck of being assigned to a safe and effective compound, to excellence in contributing to the project. Another similar bias is loss aversion, a fancy business term for “fear of failure.” Pharma is understandably already risk-averse, but it is also disadvantageous to have people avoiding difficult projects, or killing projects that are a deviation from the norm without sufficient basis. If people on failing projects are rewarded for swiftly contributing to clinical evaluation and cost-effectively killing their project, there is less motivation to “succeed at all costs” or “run for the hills.”
In
a similar vein, even when project members’ fates are not tied to a project outcome, a project team can fall in love
with a concept after expending a lot of hard effort, which also makes an
objective analysis of the product’s value difficult. In this case, it is up to the peer reviewers or
due diligence team to make sure that they are getting a clear picture and not
an overly positive projection based on the best subset of data.
Groupthink
Groupthink
is the result of insufficient diversity on the team or strong dominant members
that quash all dissent before it can be fully
explored. If you have a group of scientists from similar
backgrounds, who have been working together in the same field for a long time,
groupthink can occur. Most Big Pharma companies indirectly solve
groupthink by aggressively promoting diversity and reorganizing fairly often, so you aren’t working with the
same people for more than a few years. Groupthink can be challenged
head-on in peer review by considering the people making up the team- was there
enough varied expertise? Were all voices heard?
“We find comfort among those who agree with us – growth among those who don’t.”
Frank A. Clark
Halo Effect
There is a whole book devoted to this
type of bias. Where does it come up for pharma? In audits of
suppliers and due diligence for in-sourcing, this bias can be difficult to
avoid. A related bias is the saliency
bias, where a previous success casts a rosy glow on a new, similar
project. The halo effect can come up in decisions regarding outsourcing.
If you have a company that you love and frequently use for analytical
capability, that positive association may bias you to choose them for
formulation work, even though it may turn out that their capabilities for formulation are
insufficient. As common as this bias is, at least it is easier to spot
than some other types of bias. Auditing and due-diligence teams will benefit
from reminding themselves of this potential bias before visiting a favorite
supplier, as tempting as a shortened visit would be.
Confirmation Bias
This
bias may be the most insidious for pharma. In confirmation bias, the team
generates one path forward and seeks only data to support the chosen path,
disregarding all else. In drug development, each decision builds over a
thousand smaller previous decisions. A
common pitfall in oral formulation development is dose. Early in development, a high dose is required, so you develop a melt granulation. Later in
development, when the dose has dropped to
10 mg, did the project team scale down the melt granulation, or evaluate a
cheaper dry blend process?
Availability Bias
There
is much scientific information to evaluate in the early stages of product
development. Even still, many times you have to move forward with less info
than you would like. Analytical testing is a bit like exploring a cave
with a flashlight, where the light cast by the flashlight
is the capability of your test. Is there anything lurking in the
shadows? It’s important to do a risk assessment based on what data is
missing at the time of the decision and evaluate “what
ifs.” What if the drug substance supply was not an issue? What
if you had another month to develop? How would the decision
change? Should a contingency plan be in place in case a critical factor
does change?
For
example, many times your first formulation is
developed while your salt program is ongoing. For now, you are
assuming your compound is insoluble, but what if a soluble salt is found? How will this change your
approach? Do you have a workable backup plan?
Sunk Cost Fallacy
Pharma
is very susceptible to the sunk cost fallacy because it is just so expensive to
develop a drug. The sunk cost fallacy is when you, for better or worse,
factor in past cost/resource into a decision for the future.
Consider
the simplistic hypothetical case where
you have a drug that you have already spent $500 million developing. The
Food and Drug Administration (FDA) then restricts your patient population,
driving the market forecast from blockbuster level to only $5 million a year
over a projected remaining patent life of 7 years. You have $5 million in
expected future costs prior to
launch. If you consider the sunk costs, this project is a loser, and you
may be tempted to cut your losses and save $5 million. However, if you
ignore the past money spent and focus only on the future, the return on
investment is pretty good.
The
sunk cost fallacy can also work in the opposite
way and be a powerful companion to the self-interest bias and related biases above, also known as the
“We Have to Make This Work Because We Have Already Spent Ungodly Sums on
It” bias.
Bias Assessment
Considering
the impact and cost of bias to Big Pharma, an organizational assessment to
determine how susceptible you are to bias
may be in order:
–
How aware are your project teams of cognitive bias and how to recognize
it? Is this awareness only at the executive level, or does it reach to
your bench-level decision makers?
–
How are your decisions controlled? Is there peer review? Are the peer
groups involved sufficiently diverse?
–
Is your corporate or departmental culture breeding bias? Are people
rewarded based on only project success? Have you ever rewarded a “positive failure”? Are
dissenting opinions welcomed?
–
Are there physical or process factors that could create bias in your
decisions? For example, scientists may have a bias toward equipment housed
in the same building as their office. If ordering a new excipient requires
multiple forms and a six-month auditing process, there will be a strong
preference for what’s already in the warehouse.
Pharmaceutical
employees weather a perfect storm of conditions that promote bias: high
financial stakes, a strong scientific drive
to produce successful results, considerable time pressure, and a highly
regulated environment resistant to change. A pharma company that promotes
awareness of bias and implements effective counter-measures at all levels of
the organization can sail through this storm toward better outcomes.
