Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul 26;13(1):12107.
doi: 10.1038/s41598-023-34938-7.

The difference between 'placebo group' and 'placebo control': a case study in psychedelic microdosing

Balázs Szigeti  1 , David Nutt  2 , Robin Carhart-Harris  3 , David Erritzoe  2
Affiliations

The difference between 'placebo group' and 'placebo control': a case study in psychedelic microdosing

Balázs Szigeti et al. Sci Rep. .

Abstract

In medical trials, 'blinding' ensures the equal distribution of expectancy effects between treatment arms in theory; however, blinding often fails in practice. We use computational modelling to show how weak blinding, combined with positive treatment expectancy, can lead to an uneven distribution of expectancy effects. We call this 'activated expectancy bias' (AEB) and show that AEB can inflate estimates of treatment effects and create false positive findings. To counteract AEB, we introduce the Correct Guess Rate Curve (CGRC), a statistical tool that can estimate the outcome of a perfectly blinded trial based on data from an imperfectly blinded trial. To demonstrate the impact of AEB and the utility of the CGRC on empirical data, we re-analyzed the 'self-blinding psychedelic microdose trial' dataset. Results suggest that observed placebo-microdose differences are susceptible to AEB and are at risk of being false positive findings, hence, we argue that microdosing can be understood as active placebo. These results highlight the important difference between 'trials with a placebo-control group', i.e., when a placebo control group is formally present, and 'placebo-controlled trials', where patients are genuinely blind. We also present a new blinding integrity assessment tool that is compatible with CGRC and recommend its adoption.

PubMed Disclaimer

Conflict of interest statement

B.S. declares no conflict. D.N. is an advisory to COMPASS Pathways, Neural Therapeutics, and Algernon Pharmaceuticals; received consulting fees from Algernon, H. Lundbeck and Beckley Psytech; received lecture fees from Takeda and Otsuka and Janssen plus owns stock in Alcarelle, Awakn and Psyched Wellness. D.E. received consulting fees from Aya, Mindstate, Field Trip, Clerkenwell Health. R.C.H. is an advisor to Beckley Psytech, Mindstate, TRYP Therapeutics, Mydecine, Usona Institute, Synthesis Institute, Osmind, Maya Health, and Journey Collab.

Figures

Figure 1
Figure 1
The activated expectancy bias (AEB) model, consisting of 3 binary nodes (TRT, PT and TE) and a continuous value node, the outcome (OUT). In the equations, BX/NX stand for a random Bernoulli/normal variable, respectively. The binary nodes (TRT, PT and TE) represent Bernoulli variables (BTRT, BPT, BTE), where the values of 0/1 correspond to placebo/active. To generate AEB model data, first Treatment (TRT) is determined by Eq. 1 and then the Perceived treatment (PT) by Eq. 2, where pCG is the probability of correct guess, i.e. the correct guess rate, and then Treatment expectancy is fixed according to Eq. 3. Finally, the outcome score is calculated by Eq. 4 which has components of natural history (NNH), direct treatment effect (NDTE) and activated expectancy bias (NAEB), see Supplementary table 1 for the numeric value of all parameters.
Figure 2
Figure 2
Correct guess rate (CGR) adjustment to estimate the outcome of a perfectly blinded trial based on data from an imperfectly blinded trial. First, scores (purple histogram at top) are separated into four strata corresponding to all possible combinations of treatment and guess. Both treatment and guess are binary with potential values of placebo/active, thus, the four strata are (using the treatment/guess notation): PL/PL, AC/PL, PL/AC and AC/AC. Next, statistical models of these strata are built using kernel density estimation (KDE). Note that two strata correspond to correct guesses (PL/PL and AC/AC; red) and two to incorrect guesses (AC/PL, PL/AC; blue). Next, n/2 random samples are drawn from the correct guess KDEs, such that the relative sample sizes of the correct guess strata are preserved, i.e. the ratio nPL/PL/nAC/AC is same as in the original data, see Supplementary materials for a numeric example. Similarly, n/2 random samples are drawn from the incorrect guess KDEs, such that the ratio nAC/PL/nPL/AC is same as in the original data. These random samples are then combined, resulting in a pseudo-experimental dataset with CGR = 0.5 (purple distribution at bottom), corresponding to effective blinding. The random sampling from KDEs is repeated 100 times, for each CGR-adjusted pseudo-experimental dataset is analyzed to estimate the direct treatment effect, see Estimate of treatment effects. The ‘CGR adjusted treatment effect/p-value’ is the mean treatment estimate / p-value across these 100 samples. Estimates at other CGR values can be obtained similarly, e.g. a trial with CGR = 0.6 can be approximated by drawing 0.6*n random samples from the correct guess KDEs and 0.4*n random samples from the incorrect guess KDEs, etc.
Figure 3
Figure 3
Correct guess rate (CGR) curves of the activated expectancy bias (AEB) model. Each panel shows the estimated treatment p-value (blue; scale shown on left y-axis) and effect size (red; scale shown on right y-axis), with their corresponding confidence interval, as a function of CGR. Horizontal purple dashed line represents the p = .05 significance threshold, vertical green dashed line corresponds to the simulated trial’s original CGR, while the black dashed line corresponds to a perfectly blinded trial (CGR = 0.5). The model was analyzed with 2*2 = 4 configurations of parameters, corresponding to the possibilities of the direct treatment effect (DTE) and activated expectancy bias (AEB) either being active or inactive, see Fig. 1. For the DTE off; AEB on case (bottom left) generates a false positive finding when CGR is not considered during analysis (green dashed line intersects p-value estimate below 0.05), but CGR adjustment recovers the lack of treatment effect (black dashed line intersects p-value estimate above 0.05). For the DTE on; AEB on case (bottom right), both analyses correctly identify that there is a treatment effect; however, non-CGR adjusted analysis overestimates the effect size by ~ 40%, see Table 1 for numeric results.
Figure 4
Figure 4
Correct guess rate (CGR) curves for self-blinding microdose trial outcomes. Each panel shows the estimated treatment p-value (blue; scale shown on left y-axis) and effect size (red; scale shown on right y-axis), with their corresponding confidence interval, as a function of CGR. Horizontal purple dashed line represents the p = .05 threshold, vertical green dashed line corresponds to the trial’s original CGR (= 0.72), while the black dashed line corresponds to a perfectly blinded trial (CGR = 0.5). Outcomes in the top row (Positive and Negative Affection Scale (PANAS) and Mood visual analogue scale) are significant according to unadjusted analysis (green dashed line intersects p-value estimate below 0.05), but become insignificant after CGR adjustment (black dashed line intersects p-value estimate above 0.05), arguing that these findings could be false positives driven by AEB. Energy VAS remains significant even after CGR adjustment, although the effect size is reduced by ~ 40%. This finding suggests that microdosing increases self-perceived energy beyond what is explainable by expectancy effects, although the remaining effect is small (Hedges’ g = .34). Finally, CGR adjustment has little impact on the cognitive performance score as both the p-value and the effect estimate remain close to a constant. This finding suggests that this measure is not affected by AEB, possibly because cognitive performance was not self-rated, rather measured by objective computerized tests, see Table 2 for numerical results.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. von Similon M, et al. Expert consensus recommendations on the use of randomized clinical trials for drug approval in psychiatry-comparing trial designs. Eur. Neuropsychopharmacol. 2022;60:91–99. doi: 10.1016/j.euroneuro.2022.05.002. - DOI - PubMed
    1. Karanicolas PJ, Farrokhyar F, Bhandari M. Blinding: Who, what, when, why, how? Can. J. Surg. 2010;53(5):345–348. - PMC - PubMed
    1. Colagiuri B. Participant expectancies in double-blind randomized placebo-controlled trials: Potential limitations to trial validity. Clin. Trials. 2010;7(3):246–255. doi: 10.1177/1740774510367916. - DOI - PubMed
    1. Bausell RB. Snake oil Science: The Truth About Complementary and Alternative Medicine. Oxford University Press; 2009.
    1. Baethge C, Assall OP, Baldessarini RJ. Systematic review of blinding assessment in randomized controlled trials in schizophrenia and affective disorders 2000–2010. Psychother. Psychosom. 2013;82(3):152–160. doi: 10.1159/000346144. - DOI - PubMed