Thanks for visiting. I’ve been writing a blog since 2011, prior to my career shift to medical writing. The previous blog posts can be viewed on Blogger, and are informal and fun posts on a variety of pharmaceutical topics and give a hint to my personality. As I polish up those posts to a more professional style or create new ones, they will be posted here.
The heart and soul of good writing is research; you should write not what you know but what you can find out about.
— Robert J. Sawyer
This week, we had a very interesting discussion at work about COVID-19 testing, prompted by a tweet from Elon Musk. Even in a group of people I consider brilliant, it was clear we were (including me) a little confused about COVID tests. So I decided to do a little research to answer some questions I had and post the answers with a little bit of context of science and statistics.
The bottom line is: in many cases, COVID-19 testing is a clue, not an answer. It can tell you if you definitely have, most likely have, probably have, probably do not have, most likely do not have COVID-19. Testing cannot tell you that you definitely do not have COVID-19.
Tests are helpful, but are only a piece of the puzzle. The most important things you can do to protect your health and others are to:
The CDC has a Self Checker to help you make testing decisions, and the FDA has a helpful video.
| Test type | PCR test |
| Other names | sometimes called nucleic acid test, qPCR, RT-PCR, molecular test, genetic test, diagnostic test, viral test, nucleic acid amplification test (NAAT), LAMP test |
| What if I get a positive result? | If you get a positive result, you have COVID-19. This test is the “gold standard” – PCR is a sensitive test that is very specific for COVID-19. Testing positive for COVID-19 without being infected is theoretically possible due to a lab mistake, but unlikely. You should self-isolate and people who live with you should quarantine for as long as your health care provider or health department recommends. |
| What if I get a negative result? | There is still a chance you have COVID-19. This test can only find COVID-19 if the swab picks up enough virus to test. You could have growing amounts of virus that the swab did not pick up. You should talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should be retested at a later date and if you should self-isolate or quarantine. |
| Test type | Antigen test |
| Other names | sometimes called rapid test, viral test, diagnostic test |
| What if I get a positive result? | If you get a positive result, you may have COVID-19. The likelihood of you having COVID-19 depends on how many people in your community have the virus (see below for explanation) and your exposure. You should self-isolate and your household should quarantine, but you may need to follow-up. Talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should have a PCR test. |
| What if I get a negative result? | There is still a chance you have COVID-19. This test can only find COVID-19 if the swab picks up enough virus to test. You could have growing amounts of virus that the swab did not pick up. You should talk to your health care provider about your symptoms and possible exposure to COVID-19 to find out if you should have a PCR test and if you should self-isolate or quarantine. |
| Test type | Antibody test *This test does not test for COVID-19 infection. This test is best used for research purposes, or to find out if your current health problems could be due to a recent COVID-19 infection where the virus is no longer able to be found by other tests.* |
| Other names | Serological test, serology, blood test, serology test |
| What if I get a positive result? | You have antibodies that are similar to the ones made to fight COVID-19. You may currently have a COVID-19 infection or may have had one in the recent past. You could also have been infected with a virus that was similar to COVID-19. You should get a COVID-19 PCR or antigen test to find out if you still have an active infection. If you do not have an active infection, a positive antibody test may not mean you are immune to COVID-19. You should continue to take precautions to avoid spreading COVID-19. Consider donating plasma to help others. |
| What if I get a negative result? | This does not mean you do not have COVID-19. This test does not test for the presence of the virus, and your body may not have made antibodies to the virus yet. If you need to test for COVID-19, you need a PCR or antigen test. You may not have ever had COVID-19. If you never had exposure or symptoms, this is the most likely answer. You may have had a COVID-19 infection in the past. Antibody levels can drop over time, so you could have had an infection but no longer have antibodies. |
There is a curve of infection that happens after you are exposed to COVID-19. The virus is in your body and multiplying. Once the virus gets high enough, sensitive PCR tests find the genetic material inside the virus shell and less sensitive antigen tests find pieces of the virus shell. As you recover from the virus, antibodies appear and the virus level decreases. Over time, the antibodies also disappear.
All tests have limits for how little of the virus they can find, called the limit of detection. Depending on when you are tested, your level of virus may be too low for the test to find. If you were tested earlier or later, you may get a different result because your levels of virus got higher or lower. At first, you could test negative because there is not yet enough virus at the spot they swab to test above the limit of detection. The date you were exposed and what symptoms you are experiencing are important clues to where you are on the curve.
If your levels of virus are in a zone where the test is not accurate, you could get inconclusive results, which is likely what happened in Elon Musk‘s case.
This is the reason why if you have been in close contact with a COVID-19 case, the CDC still recommends you self-isolate for 14 days even if you have a negative test and do not have symptoms.
Antigen tests are an important tool in our battle against COVID-19, but due to the way the tests work, there is a possibility of testing positive when you do not have the virus (false positive) and testing negative when you do have the virus (false negative).
Due to an interesting quirk of statistics, your chance of getting a false positive increases when the chance of you catching COVID-19 is low. Your chance of a false positive decreases when the chance of you catching COVID-19 is high. How is this possible? The opposite is true for false negatives – when your chance of catching COVID-19 is high, you are more likely to get a false negative.
BD, who manufactures the Veritor™ Plus System for rapid COVID-19 (SARS-CoV-2) testing, has a nice graphic of this phenomenon on their webpage. They state here that their test has a specificity of 99.5%, which means 1 out 200 tests will be a false positive. They also state that their test has a sensitivity of 84%, which means if we used the test on 200 people who are all later confirmed to have COVID-19, 168 will test true positive, and 32 will have false negatives.
Positive predictive value is the chance that your positive is a true positive. Prevalence means how many people in your community have an active case of COVID-19. As you can see, when there are very few COVID-19 cases in your area, your chance of your positive result being correct could be as low as 9.1%! However, if your chance of COVID-19 infection is higher (due to high community spread or known exposure), your chance of your positive result being correct is >90%.
This seems to make no logical sense. The way the test works should not change based on what is going on in the outside world, right?
Let’s imagine we mail this antigen test to a secluded island nation that has never been exposed to COVID-19. The test is accurate 99.5% of the time, which means for every 200 tests performed, 1 will be a false positive. So, residents of this island who test positive have a 100% chance that it is a false positive. There are no false negatives or true positives in this case because COVID-19 does not exist on the island.
Now, let’s look at the state of NC (DHHS testing dashboard). Of the people who are getting tested for COVID-19, 7.9% are testing positive by PCR tests. (People getting tested are more likely to have COVID-19 than NC residents as a whole, because people getting tested are more likely to have had exposure or symptoms.) If we were to take 200 of those people in NC who had a PCR test in the last few days, 16 of them would have COVID-19. If we tested all 200 with the BD antigen test, 14 people would test positive. Of those, 1 would still be false positive from the test as we saw above. Because this test catches 84% of positives, 3 of the positive people will test false negative and the remaining 13 will test true positive. For those who tested positive, their chance of having COVID-19 is 93%. Their chance of a false positive is 7%. For those who tested negative, their chance of having COVID is 2%.
Now, let’s look at a hypothetical case where a workplace in Sampson County, NC wanted to test all 200 of their employees with the rapid antigen test, even though there had not been any known exposure. According to NC DHHS, there have been 538 cases per 100,000 residents in the last 14 days, which is about 1 person out of 200 people. We would still have our 1 false positive from the test, and 1 true positive. So of the 2 people who receive positive results, they each have a 50% chance of having a false positive.
On average, the chance of a false negative is slim – but if just 1 person tested false negative due to being at an early but still contagious stage of infection, having a party based on these results could get several people sick.
Now let’s imagine a super spreader event. There was a large indoor wedding where people sang together, laughed, and yelled to hear each over the music for hours. They hugged and shook hands while enjoying appetizers. Of the 200 people who attended, 130 people were later confirmed COVID-19 positive by the PCR test. If we tested all 200 attendees with the BD antigen test, 110 people would test positive. Of those, 1 would still be false positive from the test as we saw above. Because this test catches 84% of positives, 21 of the positive people will test false negative and the remaining 109 will test true positive. For those who tested positive, their chance of having COVID-19 is 99%. Their chance of a false positive is 1%. For those who tested negative, their chance of having COVID is 23%.
In each case, the 1 false positive out of 200 tests stays the same – but the number of total COVID-19 positive cases changes, which causes the chance of a positive actually being a false positive to change.
After seeing these examples, it’s easier to understand why the antigen test is recommended for people with exposure or symptoms of COVID-19 and not for general screening, and also why preventative isolation and a follow-up PCR test is important in cases where the risk of a false negative is high. Results from general screening may be useful to determine trends, and may catch some true positive cases, but it also may give a sense of false security because it does not catch early cases (as in the now famous event at the White House Rose Garden) and may sideline false-positive people who are not sick.
An important note on false negatives – unlike false positives, they are due to a combination of factors related to where you are in your COVID-19 course of infection and factors related to the test itself. Although BD reports an average false negative rate – your personal risk of a false negative rate may be higher or lower depending on your personal circumstances. Also, although the PCR test is much less likely to have false positives, false negatives like the example above can happen with the PCR test as well.
Each state has their own method of testing and reporting results. These results will always be an estimate because not everyone gets tested. In states that report antigen results, there is a possibility that false positives will get reported. The best state health departments can do is set clear rules for testing and reporting results and look at trends over time. In NC, DHHS reports antigen results separately. Right now they are <5% of the total.
If you have doubt that the increase in the cases is real, you can also look at hospitalization and death trends.
CDC information on testing (for the public)
CDC information on testing (more detailed for health care workers)
Pharmaceutical professionals keep one foot in science and another foot in health authority regulation, both rapidly evolving fields that require consistent training to keep pace. In addition, new opportunities often entice scientists to widen their expertise to contribute to new areas of drug development, which may require an entirely new set of skills. At some point, you are likely to reach the limit of what your corporate or personal budget for training allows, and at that point this list is just what you are looking for: quality training that is either free or subsidized!
These courses are ones I have taken and felt were worth the time investment. This list is current as of September 2019.
This course is offered by the NIH to train new clinical investigators from October to June of each year. It covers all aspects of clinical trials including design, analysis, reporting, budgeting, regulations, and ethics. The textbook is available for about $75 and the course is free, self-paced, and entirely online. If you pass the final exam you will earn a certificate of completion. This course is a significant time commitment, there are about 40 lectures and most are 60-90 minutes, and you will need to allot additional time to reading the assignments and studying for the exam. I highly recommend going through Statistics on Khan Academy (see below) to prepare for the biostatistics in this course.
This 6-week course offered by the Office of Regulatory Affairs and Quality (ORAQ) at Duke University is free and available online via WebEx. Each session starts around September, but you can register as early as February. This course is not self-paced, there are 1-hour webcasts from 12-1 pm EST on Fridays. They take attendance and there is reading and homework.
ORAQ also offers free seminars on regulatory topics that you can join via WebEx.
This online course consists of 6 self-paced modules, which took me about 6 weeks (they estimate 8 weeks). It’s a great overview of the drug approval process and the history of the FDA. It’s presented by the Program on Regulation, Therapeutics, and Law at Harvard and Brigham and Women’s Hospital. It is free to audit, or $99 to get a certificate. This course starts again October 3, 2019. It is self-paced, so you start anytime enrollment is open.
The FDA offers free online resources, including the online courses below.
Khan Academy offers free courses on Statistics and Probability, AP Statistics, and Health and Medicine that can keep you busy for months.
Dr. Kristin Sainani of Stanford teaches this excellent free self-paced course that starts with a brief review of statistical concepts, then uses medical case studies to challenge your interpretation of statistics in clinical reports. After this course, you will be better able to understand and critically evaluate statistical claims in journal articles and clinical study reports. It’s 23.5 hours, but well worth your time. Dr. Sainani is an excellent teacher, I enjoyed several aha! moments in this course when she was explaining the mathematical underpinnings of statistics.
If you are completely new to medical writing, this $75 course will provide a complete overview to the field. It’s available online through ed2go through partnership with community colleges.
When writing up adverse events, you need to learn a whole other language. If you are new to terms like pyrexia, dyspnoea, and tachycardia, take this brief online self-paced course on the Latin and Greek behind medical terminology. It takes about 2 hours, and is free if you don’t require a certificate.
Government communications are required to be written in plain language, but some writers are unclear what plain language means. This free online self-paced course provides clear examples of how to use plain language in health communications, and exercises for you to practice. There are 8 modules, each module takes approximately 30 minutes.
Membership to AMWA will cost you $195/year, but offers a lot of educational perks for medical, scientific, or regulatory writers. AMWA offers a variety of paid courses, but there are some that are free for members (search for “pocket training” or “complimentary“), and they email out a monthly free webinar (one that is usually paid) for members as well. You also get an included subscription to the AMWA Journal, all back issues online, and proceedings of the past annual conferences are summarized and posted online. Membership in AMWA also includes free chapter events that are a great bargain if you live near an active chapter.
Membership in the North Carolina Regulatory Affairs Forum is only $40 and includes 6 seminars per year on regulatory topics that are available by WebEx. They also offer a summer workshop (at an additional very reasonable fee) to prepare for the Regulatory Affairs Certification (RAC) exam.
This podcast is provided by Emma Hitt Nichols of Nascent Medical to promote her business and her 6-week course. She has many interviews with medical writers, and through listening you can learn about the many career paths available to medical writers as well as many tricks of the trade.
This podcast, sponsored by PSI, is excellent for statisticians and those who work with statisticians, and some episodes on leadership and influence are applicable to anyone in pharma.
Take a whirlwind tour of the past, present, and future of cancer in episode #62 of Peter Attia’s podcast. I’ve listened to this one a few times because so much interesting detail is packed into every minute of this episode.
This free magazine will keep you up to date on news, regulatory updates, and best practices for Chemistry, Manufacturing, and Controls (CMC).
This is a great monthly email newsletter for medical writers full of tips, tricks, and tools.
The European Medical Writer’s Association journal published an entire issue devoted to regulatory writing in 2014, and it’s free for non-members to read.
Trilogy has several publications written by expert medical writers on a variety of topics.
Emma broadcasts a free webinar regularly to promote her 6-week course. The course is not free, but when I attended her webinar she provided her book for free to attendees.
Know of a great cheap or free course that I should list here? Contact me through the contact link above and let me know! Happy learning!
Scientists are well versed in experimental bias, which is why we address it by using experimental controls, masking our clinical trials, and using the scientific method to approach questions. However, how do we control for bias within our own minds? Cognitive bias refers to any number of ways that our brain prevents us from making entirely objective decisions. In an article that Harvard Business Review published in June 2011, “Before You Make That Big Decision…”, several types of cognitive bias are defined and discussed along with case studies, and a 12-step checklist to root out bias is defined.
Major decisions in pharmaceuticals are impacted by cognitive bias. When developing a product, there are a million decisions that can have a significant impact on the cost, timescale, clinical success, and eventual marketability of your product. Many of these decisions are originally made at the bench level, and may not be able to be changed without considerable additional time or expense as the project progresses through later stages of development.
For example, a formulator may demonstrate a bias for a particular type of formulation process because of previous experience and comfort, or the wish for high visibility through the use of trendy new technology, or convenience according to what equipment is on site and available. Decision makers should recognize the potential for this bias and make sure the best formulation is chosen regardless of the above factors. Once this formulation makes it into human studies, there is considerable inertia that makes change difficult, since the project team doesn’t want to delay timelines by having to repeat animal studies or bridge with additional human pharmacokinetic studies.
Bias can be very costly to big pharma companies, but attempts to avoid bias are not without cost. Multiple layers of peer review, involving Marketing early in development where most compounds fail for other reasons, and execution of checklists also take time, but could save billions for that one “blockbuster in the rough.”
According to the article, it is nearly impossible to detect your own bias, but through learning about bias, we can better detect it in our peers and use this knowledge to better challenge decisions. For example, when performing due diligence, you must be alert for bias from the company under scrutiny, the fellow members of your team, and in how your team prioritizes and reports the findings.
Here are some types of bias from the article and how they could come up in pharma:
Self-interested Bias
This type of bias is hard to avoid. Almost every person on a project team is heavily vested in the success of their project. Part of this is due to corporate culture, which tends to reward those people who happen to be on successful projects. This bias can be minimized by shifting the focus from project success, which can be largely due to the luck of being assigned to a safe and effective compound, to excellence in contributing to the project. Another similar bias is loss aversion, a fancy business term for “fear of failure.” Pharma is understandably already risk-averse, but it is also disadvantageous to have people avoiding difficult projects, or killing projects that are a deviation from the norm without sufficient basis. If people on failing projects are rewarded for swiftly contributing to clinical evaluation and cost-effectively killing their project, there is less motivation to “succeed at all costs” or “run for the hills.”
In a similar vein, even when project members’ fates are not tied to a project outcome, a project team can fall in love with a concept after expending a lot of hard effort, which also makes an objective analysis of the product’s value difficult. In this case, it is up to the peer reviewers or due diligence team to make sure that they are getting a clear picture and not an overly positive projection based on the best subset of data.
Groupthink
Groupthink is the result of insufficient diversity on the team or strong dominant members that quash all dissent before it can be fully explored. If you have a group of scientists from similar backgrounds, who have been working together in the same field for a long time, groupthink can occur. Most Big Pharma companies indirectly solve groupthink by aggressively promoting diversity and reorganizing fairly often, so you aren’t working with the same people for more than a few years. Groupthink can be challenged head-on in peer review by considering the people making up the team- was there enough varied expertise? Were all voices heard?
“We find comfort among those who agree with us – growth among those who don’t.”
Frank A. Clark
Halo Effect
There is a whole book devoted to this type of bias. Where does it come up for pharma? In audits of suppliers and due diligence for in-sourcing, this bias can be difficult to avoid. A related bias is the saliency bias, where a previous success casts a rosy glow on a new, similar project. The halo effect can come up in decisions regarding outsourcing. If you have a company that you love and frequently use for analytical capability, that positive association may bias you to choose them for formulation work, even though it may turn out that their capabilities for formulation are insufficient. As common as this bias is, at least it is easier to spot than some other types of bias. Auditing and due-diligence teams will benefit from reminding themselves of this potential bias before visiting a favorite supplier, as tempting as a shortened visit would be.
Confirmation Bias
This bias may be the most insidious for pharma. In confirmation bias, the team generates one path forward and seeks only data to support the chosen path, disregarding all else. In drug development, each decision builds over a thousand smaller previous decisions. A common pitfall in oral formulation development is dose. Early in development, a high dose is required, so you develop a melt granulation. Later in development, when the dose has dropped to 10 mg, did the project team scale down the melt granulation, or evaluate a cheaper dry blend process?
Availability Bias
There is much scientific information to evaluate in the early stages of product development. Even still, many times you have to move forward with less info than you would like. Analytical testing is a bit like exploring a cave with a flashlight, where the light cast by the flashlight is the capability of your test. Is there anything lurking in the shadows? It’s important to do a risk assessment based on what data is missing at the time of the decision and evaluate “what ifs.” What if the drug substance supply was not an issue? What if you had another month to develop? How would the decision change? Should a contingency plan be in place in case a critical factor does change?
For example, many times your first formulation is developed while your salt program is ongoing. For now, you are assuming your compound is insoluble, but what if a soluble salt is found? How will this change your approach? Do you have a workable backup plan?
Sunk Cost Fallacy
Pharma is very susceptible to the sunk cost fallacy because it is just so expensive to develop a drug. The sunk cost fallacy is when you, for better or worse, factor in past cost/resource into a decision for the future.
Consider the simplistic hypothetical case where you have a drug that you have already spent $500 million developing. The Food and Drug Administration (FDA) then restricts your patient population, driving the market forecast from blockbuster level to only $5 million a year over a projected remaining patent life of 7 years. You have $5 million in expected future costs prior to launch. If you consider the sunk costs, this project is a loser, and you may be tempted to cut your losses and save $5 million. However, if you ignore the past money spent and focus only on the future, the return on investment is pretty good.
The sunk cost fallacy can also work in the opposite way and be a powerful companion to the self-interest bias and related biases above, also known as the “We Have to Make This Work Because We Have Already Spent Ungodly Sums on It” bias.
Bias Assessment
Considering the impact and cost of bias to Big Pharma, an organizational assessment to determine how susceptible you are to bias may be in order:
– How aware are your project teams of cognitive bias and how to recognize it? Is this awareness only at the executive level, or does it reach to your bench-level decision makers?
– How are your decisions controlled? Is there peer review? Are the peer groups involved sufficiently diverse?
– Is your corporate or departmental culture breeding bias? Are people rewarded based on only project success? Have you ever rewarded a “positive failure”? Are dissenting opinions welcomed?
– Are there physical or process factors that could create bias in your decisions? For example, scientists may have a bias toward equipment housed in the same building as their office. If ordering a new excipient requires multiple forms and a six-month auditing process, there will be a strong preference for what’s already in the warehouse.
Pharmaceutical employees weather a perfect storm of conditions that promote bias: high financial stakes, a strong scientific drive to produce successful results, considerable time pressure, and a highly regulated environment resistant to change. A pharma company that promotes awareness of bias and implements effective counter-measures at all levels of the organization can sail through this storm toward better outcomes.
References
