In this video I discuss the difficulties of assessing the effectiveness of treatment. These include spontaneous remission, regression to the mean, nonspecific treatment effects, and biases which can influence patient and clinician perspectives on improvement. Disclaimer: This channel is for educational purposes only and is not intended as medical advice for the diagnosis or treatment of psychological disorders.

Video Transcript

I’m Michael Corayer and this is Psych Exam Review. In the previous video, I mentioned some ways of trying to evaluate drug treatments and think about whether a certain pharmaceutical intervention is actually helpful for a patient. And so in this video, I want to go into this topic in more detail and think about how we can assess the effectiveness of treatments in general, whether we’re talking about biomedical interventions or something like psychotherapy.

So we have this main question, which is, how do we know that a treatment has actually worked? How do we know if it’s effective? Now, a simple way to try to answer this would be to say, well, did the patient get better? If the patient got better, then the treatment worked. If the patient didn’t get better or got worse, then the treatment didn’t work. But unfortunately, it’s not so simple. And so in this video, I’m going to go through a number of the reasons why it’s not so simple, why we can’t just look at improvement as the only way of assessing whether something is effective.

So first, we’ll look at a few reasons why improvement itself does not indicate an effective treatment. And the first of these is what’s called spontaneous remission. And so this is the idea that sometimes illnesses spontaneously go away on their own or symptoms just dissipate over time. Now, this is true for something like a cold. Maybe you’ve gotten a cold. Your symptoms are kind of bad. After a few days, the cold has run its course, and you no longer feel ill. But it’s not the result of any particular treatment that you gave. This is part of the reason why we can see all sorts of folk remedies for things like a sore throat or a minor cold is because generally, the cold is probably going to go away within a few days anyway. And so it doesn’t really matter what you do during those days. And so people might come to believe that what they’re doing during those days is really helping the cold when in fact, it’s just running its course and it’s sort of naturally, those symptoms are dissipating. So we can apply this to psychiatric illness and we can wonder, can we actually distinguish between a natural improvement, maybe something like an episode of depression is just going away on its own versus is the treatment that I’m getting actually the cause of why it’s going away. So that’s what we call spontaneous remission.

And it’s closely related to another idea, which is called regression to the mean. And so regression to the mean is the idea that symptoms have a range of severity over time. And so sometimes they’re worse and sometimes they’re better, even if you continue to have the symptoms. So they’re not completely disappearing, but they might be getting better. And then later they might get worse. And one of the problems we have is if they were worse and then they get better is that because they’re just sort of naturally fluctuating or is that the case that the treatment that you’re getting is actually causing that improvement in the symptoms? So you have some mean or some average level of severity for a particular symptom? And then the question is, are you returning to that average from worst or are you actually improving, right? So this is compounded by the fact that people tend to seek treatment for psychiatric illnesses when their symptoms are at their worst. So my symptoms get really bad, I go, I decide now I should start therapy or I should get antidepressants or something like this. And then I start getting better. But it’s hard to know if that’s truly the effectiveness of the treatment or a return to the average.

That’s what I tried to draw in this little graph here. You can see here’s some mean level, let’s say, over a long period of time. This is sort of your average severity of a particular symptom. And so sometimes it’s less severe, sometimes it’s more severe. But let’s say we have a time where it really increases in its severity. It gets up quite high. It’s at its worst. Now, this might be the point where I say, OK, I need to get treatment for this And then the question is over the next few weeks, if I return back to my mean, is this effectiveness or is this regression to the mean or return to the average? And this is another problem that we have in trying to assess whether or not somebody’s truly improved because of the treatment or for some other reason.

Now, another issue here is that often we rely on data from patients. We talk to patients and we want to know how they feel about certain symptoms, especially when we have symptoms that are hard to assess objectively. And so this brings in the problem that people are biased. And those biases can influence how they feel about the treatment. And this might not be an objective assessment of if they’re actually getting better. And so there’s a number of things that can cause patients to think they’re getting better that are not specific to the treatment. And so these are what we call nonspecific treatment effects, things that might help people to feel they’re improving even though they’re not the result of the particular drug the person’s getting or the particular type of therapy that they’re receiving.

And so one thing is provider confidence. So if they see a provider who says yes, you know, we have a treatment for this, we’re going to put you on, you know, the the latest, best medication or we’re going to engage in practices that have been shown to improve your symptoms and the confidence of the provider in doing that might help the person already to feel better. And related to this is the feeling that they have a sense of control over their illness. So just going to get help is a way of saying, you know what, I’m going to get better. I’m going to improve myself. I’m going to get the medications I need or I’m going to engage in the therapy that I need. And so I’m taking control of my life. For many psychiatric illnesses that’s an important part of feeling a sense of improvement. People feel okay, I’m in control. I’m not sort of subject to this disorder. I’m going to do the work that’s necessary or I’m going to engage in the practices that I need to do in order to improve.

And related to that is that there are other lifestyle improvements that often occur as a result of getting medication or of getting therapy. And so in some cases, people will improve their diet or they’ll stop drinking alcohol. And this might be because of interactions with medications. The doctor says, OK, you need to drink less alcohol. And it might just be that drinking less alcohol is actually helping them to feel better and feel an improvement in their symptoms. And it’s not actually due to the medication they’re receiving.

Or the therapist might be providing a warm relationship in the sense of empathy that the person might not otherwise be getting in their life, and so it might not matter exactly which type of therapy the therapist is engaging in. It might have been the fact that they have someone they can talk to and someone who they feel cares about them and wants them to improve.

And of course, the therapist also wants the person to improve, wants to see them get better. And this is referred to as the therapeutic alliance, the idea that you have the provider and the patient get together and they both want to see improvement in the symptoms and the therapist is helping support this. And so because they both are motivated to see improvement this might improve certain symptoms, not specific to what the actual treatment is.

And lastly, it can be difficult to isolate any of these from the specific treatment. That’s why these are called non-specific treatment effects. And another one of these that I’ve talked about in the previous video is the placebo effect. And you’re probably familiar with the placebo effect, but it’s the idea that belief that one is receiving treatment can alleviate certain symptoms. That’s sort of generally what we refer to as the placebo effect.

And of course, we try to control for this. We’re aware that this occurs not just for psychiatric illness, but for a number of other medical treatments. And so the sort of gold standard of trying to get around this is the randomized control trial or RCT and this is where we give half the patients the real drug, we give half a placebo pill, which is an inert substance, and then we see do the people getting the placebo pill improve? Do the people getting the medication improve even more? So if the placebo pill group improves the same as the medication group, then we say, well, the medications no different from a placebo pill. It’s basically not effective. But if the medication group does even better than the placebo group, so in other words, the placebo group has improved their symptoms, probably because of all these non-specific treatment effects that I just described, then if the medication group has that improvement plus even more, we can say that additional improvement is the result of the medication.

Now one of the problems that we have with these randomized control trials is that sometimes patients can figure out whether they’re getting the placebo or not. So if you start having certain side effects or interactions as a result of the real medication that might indicate to you, hey, I’m getting the real drug. I know I’m not in the placebo group now because I am having these interactions with the drug where it’s causing these side effects. Now, of course, it is possible to have side effects from a placebo pill. And this is actually what’s called the nocebo effect, where there’s negative effects of taking an inert substance. So you get a placebo pill and then you get some side effect like headaches or something that would be an example of a nocebo effect. And so that can keep people sort of guessing they don’t know if they’re in the placebo group or the medication group, but it’s possible that some people might know. They might think that, well, I must be in the placebo group or I must be in the medication group because I’m having these side effects.

Okay, and another problem we have with the patient’s perspectives is reconstructive memory. So this is something I discussed in the unit on memory. And now we can apply it to mental illness. And one of the problems we have is people are trying to recall some of their symptoms over the course of maybe weeks or months and they may falsely remember their symptoms as being worse. So the idea of reconstructive memory is each time we recall something, we sort of reconstruct it. We put the pieces back together and that means there’s potential for bias or errors to occur anytime we recall memory.

And so if I’m thinking about how bad were my symptoms two months ago, I might recall them as being worse. And I remember I might remember just the worst parts of those where I might exaggerate some of the symptoms that I was experiencing before. And this bias might lead me to believe that the treatment is worthwhile. So I think, well, you know, two months ago, man, I was really bad. I remember that as being the worst time in my life. And so as a result, now how I feel today seems to be much better when in fact, maybe the symptoms aren’t actually that different. And these biases can play a role because people are biased to want to believe that they’re at least a little better. I mean, they want to believe the treatment is helping them. They don’t want to believe well, I’m just, you know, wasting my time going to therapy every week or I’m wasting money taking these pills and I’m suffering from these side effects for no reason. It’s not justified. So people have a bias to believe that they’re doing at least a little bit better.

And another way this can play a role is they might think, well, maybe I’m about the same, but I’d be even worse off if it weren’t for my doctor giving me this prescription, or if it weren’t for the therapist that I’m seeing every week. Man, imagine how bad I’d be. You know, I managed to maintain the severity of my symptoms, thanks to this treatment, and therefore it’s effective.

And there are also cases where people become dependent on this therapeutic relationship where their symptoms actually improve but if they say that, well, I’m better, I don’t need treatment anymore, then maybe I don’t need to keep saying this therapist who I have this great, warm relationship with and who really is motivated to help me. And maybe I’ll miss that. Maybe I don’t want to give that up. And this can lead to us called malingering. This is where somebody claims symptoms after they’ve actually passed. And they may do this because they want to continue receiving care, maybe they really enjoy the therapy sessions or they enjoy the attention from somebody who’s actually listening to them. Maybe they don’t have that in their life outside of this therapeutic relationship. Maybe they don’t get much sympathy or empathy from others, and so they might have a bias towards continuing to believe that their symptoms are still there, they need to keep getting this treatment or else they’d be worse off, when in fact, maybe they’re actually much better and they don’t need treatment anymore. Maybe the treatment has actually been more effective than the patient believes.

And next we can consider the clinician perspectives. And of course, clinicians also have biases. Now we might think that they’re well aware of these. They’re, you know, professionals in this area. They’re familiar with how to assess the effectiveness of a treatment. But nevertheless, they’re going to have a bias towards the particular treatment that they practice. And it’s going to be very difficult to completely overcome that or ignore it. So if somebody’s practicing something like cognitive behavioral therapy, well, then obviously they believe that’s an effective treatment or else they wouldn’t be doing it. So automatically, we have sort of a bias towards believing this particular treatment or this particular drug is effective because, you know, that’s the person providing that treatment or prescribing that drug. And this can lead them to be biased in how they reflect on the effectiveness of their own practice.

They might only count positives and this might not be intentional. But when they think about their treatments, they might immediately bring to mind people who they saw large improvements in. This person was really bad and then they were much better, that shows me that this therapy is effective. So they might focus on success stories versus the failures. They might not spend as much time considering why did a particular treatment not work for a patient or maybe they, you know, blame the patient for certain failures for maybe not being committed enough to the therapy. Or maybe they weren’t taking the medication regularly enough or something like that. They might come up with some excuses for explaining away the failures.

Or they might wonder about people who stopped the treatment. So they might say, well, you know, this person was seeing me and then they’re not seeing me anymore. It must be better. Their symptoms must have improved to the point where they no longer need treatment. But that might not necessarily be the case. Perhaps they tried a different treatment or they went to see another therapist maybe even doing the same type of therapy. But maybe this didn’t work for them. The therapist might be seeing this as a success. I saw this person for three months, and then I don’t see them anymore because they were better rather than saying, well, maybe they chose to see somebody else or engage in some other type of therapy. Or maybe they gave up on treatment because they weren’t really improving.

OK, now by thinking about all these biases and these problems that we have in assessing the effectiveness of treatments, this helps us to understand the dangers of pseudo–therapies. And these are things that are not effective treatments, but many people believe in them. And it’s not simply the case that we can say, well, people who are promoting these therapies are just trying to scam people. That certainly occurs. But there are also what we could call true believers. Both patients and practitioners, people who really have an earnest belief in the treatment. And they’re well-intentioned; they actually want to help people and this combination of patients who have hope and trust and they feel a sense of control by engaging in this treatment and practitioners who really do want to help people, they have an honest sense of empathy for their patients. They want to relieve suffering but maybe the treatment they’re engaging it is not actually helpful.

But all of these biases and things like spontaneous remission or regression to the mean can cloud their judgment and make them think that they’re more effective than they actually are. That the treatments are more effective than they actually are. And this can help us to understand certain types of pseudo-therapies that we see, whether it’s a homeopathy or herbal medicines for certain symptoms, or things like beliefs in energy fields or healing crystals or eye movement desensitization. These therapies that don’t have good track records, don’t have good empirical evidence to support them. We might wonder how people believe in them or why people would continue paying money to receive these treatments. But by understanding these biases and getting a sense of how difficult it is to really figure out if something’s actually working, if a treatment really is effective, I think we can better understand these and then hopefully better understand other types of treatment.

And that’s what we’ll look at in the next part in the next video. So hope you found this helpful. If so, please like the video and subscribe to the channel for more. Thanks for watching!