CHICAGO – While patient access to in-person genetic counseling still faces several hurdles, online education is one way institutions are attempting to extend the reach of this expertise. But a new study suggests that while patients may express interest in this method of learning about their genomic tests, the content had better be compelling or their attention will fade.
The researchers, from Columbia University Irving Medical Center in New York, enlisted a team of geneticists, psychologists, genetic counselors, research assistants, and community members — but no communications specialists — to develop videos and website content. They hoped to determine whether basic and inexpensive videos could be an effective tool to educate patients on genetic testing
According to results presented at the recent American Medical Informatics Association meeting in Washington, DC, the videos were simply not effective. Despite this, the researchers are optimistic that the results can help inform the effective design of an online portal to educate patients about genetic testing.
The videos developed for this study were "visually basic, with minimal animation" — thanks to budget limitations from two small National Institutes of Health grants — and a voice-over, according to the paper presented at AMIA.
The Columbia team conducted a secondary analysis of Ashkenazi Jewish and Latino participants in a study called Individualized Medicine through the Application of Genomics (IMAGene) to gauge the effectiveness of online video education. IMAGene was conducted as part of the Electronic Medical Record and Genomics (eMERGE) consortium, which has as one of its goals the assessment of the ability to interpret genomic data from minority populations.
The genetic panels themselves were not meant to be diagnostic, but rather to search for signs of increased risk for certain conditions.
Of the 471 people enrolled in IMAGene, 390, or 83 percent chose website access to educational material rather than traditional sessions with genetic counselors. Those opting for online access were more likely to be non-Latino, US-born, English-speaking, younger than 45, privately insured, and with better baseline genetic knowledge than those who wanted face-to-face education.
Participants took a 15-question survey of their baseline genomic knowledge, then repeated the questionnaire following their exposure to the videos.
Those who chose web-based education were invited to learn about the testing online and watch videos developed for this study, but were not required to access the website. They were then given the questionnaire again and had two attempts to correctly answer at least 12 of the 15 questions. Columbia had genetic counselors contact those who could not meet that threshold before they were able to have genomic testing for IMAGene.
The videos covered the topics of: genes; inheritance and mutations; genomic screening for genetic conditions; interpretation of various types of test results; and personal, familial and reproductive implications of the results.
Columbia made a total of 26 minutes of videos in English and Spanish across seven clips, with the average English-language video about 3.5 minutes long. The Spanish-language video was a single, 10-minute clip, but only four study participants viewed that one, so researchers did not analyze findings from it, they said.
The study ran from July 2016 to August 2018; one video on pharmacogenetics was removed from the website in March 2017, leaving six English videos up for the duration of the study period.
Researchers measured website exposure by logging the number of times and duration they accessed the site. They also counted the amount of time spent actually playing videos and whether they completed each video by getting to the last "page" of video since each video was produced similar to animated slides in Microsoft PowerPoint, the authors said.
While new versions of some of the videos were posted about six months into the study period, there was no difference in exposure time following the change.
Participants who chose online education performed better on the baseline assessment, but there was no difference in the change in assessment scores between the groups after genomic education, the researchers found.
Of the 390 who asked for website access, 303 (78 percent) actually logged on, and they did so a total of 1,166 times, an average of 3.8 times per user. The average session lasted 5.8 minutes, with the longest going for 50.5 minutes.
However, only 45 percent of the 303 people who logged on watched any video at all. Viewers watched about 13 minutes or 3.8 videos per user and those who watched past the first five seconds of each video typically stayed until the end. Just 17 percent of the 390 who opted for website access had more than 10 minutes of video exposure.
Participants who watched at least one video to the end were more likely to view other videos than those who gave up early.
Exposure time varied slightly across several demographic groups, but there was no correlation between length of exposure and effect on improvement of knowledge of genomics in the post-education assessment.
The researchers said that a "significant factor" in this result was the study design. "Our educational tool was used at home and was self-guided, did not have any required content, and the education was not supervised in a clinical or research appointment," they wrote.
Other studies may have had more supervision, but the researchers said that that type of situation is of limited utility. "[T]o be truly scalable, education will need to be delivered in a manner that requires minimal provider supervision and is convenient for the individual," the AMIA paper said.
Several earlier studies that the Columbia researchers cited — including one from the NIH's National Human Genome Research Institute and another from the Wellcome Sanger Institute — have found similar wide acceptance of the idea of online pretest counseling, including by video.
Few of these papers have delved into how individuals interact with web-based video education platforms or whether there is any link between usage and increased genomic knowledge, though.
"This is likely not the most engaging manner in which information could be provided, but our strategy was limited by the research budget," the authors conceded. "Potentially platforms that immediately engage users with interesting and motivating stories and visually interesting illustrations could capture the viewers' attention and motivate viewers to continue to watch and increase overall usage," they said.
"We are in an era where we expect things that are exciting and engaging," lead author Julia Wynn, a senior genetic counselor and clinical research manager at Columbia, said in presenting the paper at AMIA.
These simple videos were neither exciting nor engaging. To achieve that, they will have to spend more money. Wynn suggested that maybe a celebrity would help get people more engaged.
Michael Millenson, a healthcare policy and patient engagement consultant and author in Highland Park, Illinois, who was incredulous when he saw the results, noted that the study had several shortcomings that reduced the relevance of its findings.
He said it was "disturbing" that the Columbia team was able to develop a rigorous methodology to evaluate responses, but went through with the study anyway despite not having the money to produce compelling content.
"If you're rigorously analyzing something that has little chance of working, all you've done is rigorously proved the obvious," Millenson said. "That's like the difference between a blog post and submitting an article to a peer-reviewed journal," Millenson said of the researchers' decision to self-produce videos without any input from communications professionals.
The methodology of evaluation may be rigorous, but Millenson said there was a "flaw" in the methodology of production of what was being evaluated. "You can't cut corners on the product and then get a valid result of analyzing the product that means something," he said.
"Genomic information is incredibly complicated and incredibly important. The communication of that information to the public in an understandable manner is urgent," Millenson said.
With the shortage of genetic counselors, care providers and testing laboratories do need alternate means of educating the public about genomics.
Millenson also voiced a concern from the standpoint of individual and public health. If medical professionals and academics are not able to educate the populace about genetic testing, then the communication will come from those with a profit motive. "Their communication is going to drown out the evidence-based communication, and that's not good for us as a society," Millenson said.
He noted that this is happening already with claims made by commercial purveyors of genetic tests.
Wynn said that her team is in fact talking to unspecified companies that have more expertise in the production of patient educational materials in hopes of conducting additional research. "We need to reframe how we are assessing [educational materials] as well," she said.
Wynn did say that there were some benefits to the low-budget segments Columbia produced. "Genetic knowledge is not actually the best measure of how effective these videos were," she said. For example, some study subjects said that the videos corrected their misunderstandings and helped them better understand the testing process.
However, these videos did not capture the imagination of large segments of the study population.
"I think we all are making a valiant effort to understand that technology has some ways to meet some of the growing demands on genetic providers and we, probably, as genetic providers, don't have the expertise nor the funds nor the resources to actually make this work," Wynn said in a follow-up interview. "I think this can work and I think our ability to measure whether or not it is working is getting better, but our ability to produce what is going to eventually work is probably the furthest behind from the academic side."
Wynn, who also cochairs the genetic counseling working group of the Clinical Sequencing Exploratory Research Consortium, said that while some might be doing better in terms of production and engagement, "they are not assessing whether or not it works because they are not in research."
She said that this study does have value from an analysis perspective because it measured what people are watching, when they are watching, and for how long. "That is going to give us a wealth of knowledge about how to correctly implement these [online resources]" Wynn said.
"Administering a knowledge quiz pre- and post-test is certainly not the only measure and maybe isn't the best measure," she added. It would be worthwhile to look at other metrics, such as patient satisfaction with the decisions they make after receiving this kind of information and whether they feel empowered by having such information, Wynn said.
"We want them to make informed decisions that help them and help their healthcare," Wynn said. "We don't necessarily want them to be able to tell us how many genes there are."