Skip to main content
Premium Trial:

Request an Annual Quote

PCORI's Draft Methodology Report Broadly Addresses Patient Heterogeneity in Treatment Response


The Patient-Centered Outcomes Research Institute has released the first draft of its methodology report, which aims to improve the design of studies comparing the impact of different medical interventions on health outcomes.

Although this first draft of the report doesn't specifically address how such comparative effectiveness research studies should be designed to address differential treatment responses in molecularly defined patient populations, it does outline standards for studying the heterogeneity of treatment effects in various patient subgroups based on whether the investigations are confirmatory, descriptive, or exploratory.

"Heterogeneity of treatment effect analyses can contribute to the goals of PCORI by either 1) estimating the effects of a treatment in subgroups of patients or 2) predicting whether a specific person will benefit from treatment," the report notes.

Furthermore, the report acknowledges that "subgroups" can be defined by a variety of patient characteristics, including genetic differences.

"The US Food and Drug Administration and the Journal of the National Cancer Institute both recommend reporting of treatment by sex and other subgroups," the methodology committee writes in the report. "In addition to sex, other subgroup variables that may be important to report include demographic characteristics (e.g., age), behaviour (e.g., smoking), pathophysiology (e.g., measures of disease severity), genetic markers, and comorbid conditions (e.g., diabetes status in cardiovascular disease trials)."

PCORI, an organization formed by the 2010 Patient Protection and Affordable Care Act, funds research that compares the impact of different medical interventions on patient outcomes. Through the CER it funds, the institute hopes to provide the kind of information that patients and healthcare providers need to make informed medical decisions.

Some proponents of personalized medicine feel that PCORI's CER research agenda isn't sufficiently focused on conducting studies that would produce the kind of data healthcare providers need to adopt molecularly guided strategies for individualizing care. The difficulty lies in the fact that CER has traditionally been designed and conducted to inform healthcare decisions for broadly defined patient populations.

In the methodology report, the committee acknowledges that conductingCER that produces data useful for most people is problematic because individuals usually want "personalized" information that addresses their unique medical situation.

"[I]f research doesn't account for 'our' characteristics, then the results aren't as likely to be relevant to us. What people want is information that takes into account all their unique characteristics and conditions," the report states. "Personalizing research results is becoming ever more challenging as we learn more about the genetic variation that makes each of us different from the 'average' patient."

This statement illustrates the challenges ahead for PCORI in advancing CER that yields data with applicability across patient populations while conducting research that factors in the heterogeneity among patient subsets in terms of how they respond to medical interventions based on their specific traits, environmental conditions, medical history, as well as their unique molecular characteristics.

"Like any funding agency, PCORI cannot be all things to all people. If it places the highest priority on research questions that will offer the greatest good for the greatest number of people, then it might be thought to signal a focus on common conditions or majority populations to the dismay of advocates for those with rare but serious diseases or minority populations," the methodology committee states in the report. "Conversely, focusing on less common conditions or smaller populations with the loudest advocates could drain research funding without producing adequate progress against widespread health threats."

While it's clear that PCORI understands the contradictory goals of advancing CER as personalized medicine becomes more mainstream, it is not yet known how the committee plans to address this problem. Groups that believe healthcare providers should be using strategies to individualize treatments have expressed concern that PCORI's research agenda seems to focus too heavily on advancing studies that wouldn't sufficiently account for patients' molecular heterogeneity and would advance more one-size-fits-all care.

As PCORI has crafted its research and funding focus, advocacy groups such as the Personalized Medicine Coalition have requested more specificity from the institute on how CER should factor in patients' molecular heterogeneity on treatment response. In particular, Amy Miller, PMC's VP for public policy, has expressed concern that PCORI may be conflating personalized medicine with patient-centeredness, a key theme of all of the institute's research aims (PGx Reporter 5/16/2012).

Additionally, with regard to the design of CER, Miller has said that "head-to-head drug comparisons … must take into account any pharmacogenomic information in the labels of the drugs that are going to be compared." For researchers applying for PCORI grants, it will be critical for them to know in detail how they should factor PGx information into their CER research proposals.

Other stakeholders feel that personalized medicine and traditional CER principles are at odds with each other. The Center for Medicine in the Public Interest this week released a report that asserted that applying CER to inform decisions will reduce investments in innovation, such as personalized medicine. In the report, titled Promoting Innovation and Health: Personalized Medicine or Comparative Effectiveness Research?, University of North Carolina, Chapel Hill's John Vernon and CMPI's Robert Goldberg compared rates of innovation in research and development fueled by CER and biomarker-based treatment strategies.

"We estimate that the shift [toward CER] will cause a decline and loss in R&D innovation over ten years in terms of between $38 billion and $74 billion and would reduce the number [of new drug products by 57] over the decade," wrote Vernon and Goldberg in the report. The study authors "conservatively" estimated that the social value of biomarker-based innovation could be $10 trillion over 10 years.

PCORI's methodology report offers some guidance applicable to personalized medicine in a section discussing "standards for studying patient heterogeneity of treatment effect." The committee cautions researchers that multiple subgroup analyses in a single study increases the risk of finding heterogeneity of treatment effect where there is none and that the use of subgroups can reduce the statistical power of studies.

The report also includes a table outlining "The Essential Characteristics of the Three Different Types of [Heterogeneity of Treatment Effect] Analyses," which it describes as confirmatory, descriptive, and exploratory trials. For example, for confirmatory trials, the committee recommends that the design include a small number of subgroups (one or two), a "strong" scientific rationale for the study hypothesis, a prespecified analytical strategy, and sufficient power for testing the hypothesis.

"The standards for heterogeneity of treatment effects outline the benefits and pitfalls of subgroup analyses," Andrea Hofelich, director of communications of the National Pharmaceutical Council, told PGx Reporter via e-mail. "They specify work that may explore, hypothesize and confirm what works best for subgroups of patients (e.g., those with KRAS mutation versus those without KRAS mutation). The standards in their current form do not yet identify what works best for individual patients." NPC recently posted a series of videos featuring experts discussing CER and the issue of patient heterogeneity in such studies.

While the guidelines in the methodology report are very basic, it is clear from the first draft of the document that PCORI is considering how best to conduct CER at the patient subgroup level. For the time being, PCORI recommends that "in designing studies, researchers should identify participant subgroups of interest and, where feasible, design the study with adequate precision and power to reach conclusions specific to these subgroups." The committee notes in the report that it intends to include more information on subgroup analysis in CER in later systematic reviews.

"The preliminary methodology report includes a minimum set of standards that are generally good research practices," Hofelich said. "It also focuses on prioritization, patient engagement, research conduct, and peer-review."

Ultimately, PCORI will rely on the strategies outlined in the methodology report to help guide its research funding decisions. The committee noted that over the next three years, it plans to refine the strategies and expand the scope of its proposed CER standards.

"One aim is to counter overgeneralization from studies in narrow populations or in highly controlled research settings to the broader range of people and settings in 'real life,'" the committee says in the report. "But another aim, undoubtedly, is to learn more about how well the different treatment choices can work, and for whom they work best and are safest."

PCORI plans to begin soliciting comments on the draft methodology report in July.