Transcript of video
Professor Evans, there are several basic concepts in clinical trials. And what does it mean, for example, that the trial is underpowered? Because now the clinical trial terminology is at the forefront, it’s out in the newspapers. So people have to understand that this the basic concepts. So what does it mean as a trial underpowered? What is NNT, number needed to treat? And there are pluses and minuses and that kind of basic concepts. What’s the primary and secondary endpoints of the clinical trials? Because clearly a few trials have been moving goalposts and has been common data in the medical community. Well, if we’re going to, we’ll try and take nearly all of our examples from the current situation with COVID-19. If we are going to study mortality that will require a fairly large number of people. Because even in the situation in a hospital, fortunately, not everyone will die. And if we have, let’s say 10% of people dying, then in order to find a difference that would probably be quite important, let us say of reducing that 10% mortality rate within 30 days of beginning treatment down to let us say 70% mortality 7% mortality. So, we go from 10% down to 7%. We’re going to need a large number of patients to be able to find out whether such a difference is actually occurring. And we do statistical analysis on that. But if the numbers are too small in the trial, then That is a trial we call underpowered. The power of the study to detect a real difference, if it exists, was too low. And this was true for some of the early trials that were carried out on potential treatments for COVID-19. Whereas if we study thousands of patients, then it’s unlikely that the trial will be underpowered for mortality as an outcome, provided we’re dealing with differences that are reasonable. If we wanted to detect a difference between a 10% mortality rate and a 9.9% mortality rate, we would need 10s of thousands of patients. And that, of course, is not a difference that would be very useful to individual patients. So underpowered trials are a problem, and so It’s underpowered in relation to the outcome you study. Now, if you made mortality your primary outcome, you would need a lot of patients. So very often what people do is that they make mortality, a secondary outcome, and they make their primary outcome something that is easier to study and for which we need fewer patients. And that often in this kind of situation is the time to recovery of the disease. problem with that is that it can be slightly subjective. You can define someone is reaching a level of recovery based on a clinical assessment, but it may be that you based on viral load or something of that kind, which is an objective assessment. So we may be able to have an objective assessment for a primary outcome that is easier to study the mortality. The problem is that when we look at recovery, we have a definition for it. But it may be that people don’t meet those definitions. And so it becomes obvious in the trial, that the outcome you set out as a primary one is not going to give you any useful data. And so there can be legitimate reasons for changing it. But the difficulty is that if people know what the results are showing, they can change the answer to be the change the question, and therefore know that they’ll get the answer they want. in epidemiology, this is called the Texas sharpshooter syndrome, where the Texas gunman stands at the side of a barn and fires his gun at the barn, and then afterward walks up to the side of the barn and draws target. You need in a trial to have a target specified beforehand, and then do the trial and see what the results are rather than changing the target while the trial is running. In general, that can be legitimate reasons for changing your outcome. But you’ve got to be very careful and make sure that you’re not doing it. Having already fired your gun and seen where the bullets fall. You need to do it before you know where the bullets are falling. Now, when we come to measuring the outcome, one of the things that we can do is, we say, what is the mortality rate? And let’s say that we have a treatment difference of 10% down to 5%. Right? So that means that in every hundred people, there will be five people who do not die as a result of having treatment. And so for every 20 people, there will be one person who does not die. And when we turn that upside down, we say then that the number needed to treat to prevent one death will be 20 when we have our difference between 10% and 5%. That would also be the case if there was a difference between 20% and 15% or between 50% and 45%. And so it is a measure of the number of patients who need to treat to prevent one death. In that case, now, sometimes, instead of death, we look at a particular event like myocardial infarction or a stroke. The problem is with this number is that it isn’t a pure number. It depends on how long you followed patients up for. And it also has some other statistical problems with it. So it’s not one that I particularly like, even though it sounds quite a nice thing to say, Oh, well, this drug needs 20 patients needed to treat to get the benefit, whereas this drug needs 50 patients needing to be treated. And if you’ve used the same rules for both, then NNT can be quite helpful. But you’ve got to be careful to make sure that your definition of the NNT, which isn’t a pure number, used exactly the same definition when you make comparisons between treatments.