Following a "thorough reassessment" of its Cancer Biomedical Informatics Grid program, the National Cancer Institute has decided to end some caBIG projects and migrate others to a new effort called the National Cancer Informatics Program.
NCI Director Harold Varmus announced the changes in a letter to the community last week. He noted that some caBIG projects have been "scaled back" or "eliminated" and others will be integrated into the NCIP, a new project that will "leverage the investments made in, and lessons learned from, caBIG."
The changes are in response to recommendations from a working group set up last year by NCI's Board of Scientific Advisors to assess the impact of caBIG, which kicked off in 2004 with the aim of integrating informatics resources across NCI-supported cancer centers. As outlined in a March 2011 report, the working group determined that the fruits of the caBIG program have not been "commensurate with the level of investment" in the initiative.
Varmus did not provide specifics about the planned restructuring, but said that the NCI is organizing a meeting of clinical scientists, informaticists, managers of core facilities, and representatives of advocacy groups who will discuss NCIP's development.
The group will assess the informatics needs in the cancer research community with an eye toward identifying the best ways to meet those needs and ensure that data is available and shared, Varmus said.
NCIP's management will be the responsibility of the new director of the Center for Biomedical Informatics and Information Technology, a position that NCI is seeking to fill following the departure of Kenneth Buetow last month.
Buetow vacated the role to take a position at Arizona State University as the director of computational sciences and informatics in the school's Complex Adaptive Systems Initiative (BI 3/9/2012). George Komatsoulis, NCI CBIIT's deputy director, is serving as the acting director for the time being (BI 3/2/2012).
Komatsoulis told BioInform this week that he sees the changes to caBIG as an opportunity to take the project "to the next level."
"Every decade or so, you are going to want to take a look at what you are doing, ask whether its time to reassess your strategy, alter your goals," he said. Given that caBIG "was almost eight years old ... [this] was an entirely appropriate thing to do."
Komatsoulis said that NCI's informatics arm intends to apply "the lessons that we learned doing caBIG" to NCIP "to make sure that we have an even more successful program this time."
He said NCI has complied with all of the initial recommendations made by the BSA's working group — which included ending some activities in the caBIG portfolio, although he declined to discuss in detail which programs are being discontinued.
NCI will continue to support caBIG activities that the group identified as strong points, such as establishing community-developed standards and tools for imaging and tissue banking.
He also said that the NCI informatics team has begun bi-weekly meetings with NCI's National Cancer Advisory Board's newly established Informatics Oversight Committee to ensure that the informatics program "continues to move forward in a way that will provide the maximum value to our community."
He said that NCI is retaining its informatics staff although their job descriptions will likely change as NCIP evolves.
For example, the group is looking to move several caBIG tools into a "full open source development model" where members of the cancer research community are responsible for their development and NCI acts as more of a partner, Komatsoulis said.
"Clearly as that happens, we are going to wind up changing the portfolios of our individual staff members," he said.
Komatsoulis stressed that the changes to caBIG will not affect NCI's efforts to support the cancer research community.
In fact, under NCIP, "our goal is to be more inclusive, more transparent," he said.
On the financial front, he said that the NCI's informatics arm still has "a substantial budget," although he could not provide additional details since the exact dollar amount has not been set.
Rethinking caBIG
The BSA's working group's report, published last March, discussed the results of a four-month assessment that included interviews with 59 individuals from 46 institutions. Those interviews looked at caBIG's impact on NCI-designated cancer centers, cooperative clinical trials groups, and research initiatives such as the Cancer Genome Atlas.
The group called for a return to caBIG's "original mission and premises," stating that while those goals are "worthy and remain highly relevant to the future of cancer research," there is "strong consensus" in the community that the current project has "expanded" far beyond its initial mandate "to implement an overly complex and ambitious software enterprise of NCI-branded tools."
The report further noted that many caBIG tools have "produced limited traction in the cancer community, compete against established commercial vendors, and create financially untenable long-term maintenance and support commitments for the NCI," including "a vast management network of external contractors that consumed at least $60 million in overhead costs in the past seven fiscal years."
The document also notes that caBIG's budget grew from approximately $15 million in 2004 to more than $47 million in fiscal year 2010. Furthermore, the project received between $87 million and $100 million from the American Reinvestment and Recovery Act in fiscal years 2009 and 2010, bringing the total cost of the program from 2004 to 2010 to about $350 million.
These funds have been used to develop and maintain more than 70 caBIG software tools for use in clinical settings, molecular biology, imaging, and biospecimen banking.
In spite of these expenses, the working group found that the expected impact for most tools in the caBIG portfolio has not been "commensurate with the level of investment."
For example, few NCI-designated cancer centers have adopted the full caBIG clinical trial management solution, the group said. Rather, individual components have been used in small pilot projects with "little impact on the centers' mainstream operations."
Also of note is a separate survey that was conducted by the European Association for Cancer Research, which found that among 746 European respondents, between 84 percent and 96 percent never use caBIG's tools (BI 11/7/2011).
Other caBIG projects mentioned in the report include the caGrid program, an environment for grid-based cloud computing whose development costs amounted to $9.8 million. According to the document, the working group did not find evidence that the platform had lived up to its stated goal of empowering "a new class of tools to 'accelerate the discovery of new approaches for the detection, diagnosis, treatment, and prevention of cancer.'"
The report notes that IT directors and other involved in testing and deploying caGrid at their institutions found the system "very difficult" to implement, maintain, and use.
The working group also determined that some caBIG tools were developed "at significant expense and without a clear justification, particularly since a number of similar commercial and open software tools already existed."
As an example, the report states that the caArray package, which cost more than $9 million to develop, was "not widely adopted by the community because of the availability of well-established alternatives that were either free or commercially available," such as ArrayExpress, the Gene Expression Omnibus, and GeneSpring.
"A small fraction of the total caArray investment would have been sufficient to help implement innovative or critically missing features … in the more widely used array analysis tools rather than creating a completely new solution from scratch," the report states.
The working group also found caBIG's program management structure — handled by contractors Booz-Allen Hamilton, SAIC-Frederick, and Sapient — to be "overly complex and expensive."
Such "third-party management" has introduced “too many management layers between the community doing the work and the caBIG leadership,” which has resulted in a “disconnect” between the program's goals and "the daily directives that have been used to manage the projects," the working group stated.
The Silver Lining
It's not all bad news for caBIG, however. As Komatsoulis pointed out, the report acknowledges that the program has helped "create and manage community-driven standards for data exchange and application interoperability" as well as to develop a "semantic infrastructure that allows data to be harmonized across cancer centers."
For example, the report notes, caBIG has driven the use of community-defined vocabularies that have been incorporated into "the majority of the electronic clinical trial management tools used by virtually all cancer centers" as well as in commercial solutions such as Oracle Clinical and Velos; and in bespoke systems developed by larger cancer centers.
The working group also noted that caBIG has supported the development, maintenance, and dissemination of software tools that have been developed by the academic research community, including tools for biomedical data storage, retrieval, and mining; integrative genomics, systems biology and computer modeling; and visualization tools.
Specific examples cited in the report are caTissue and GenePattern, which "received almost uniformly positive reviews," according to the report.
Additionally, the report notes that some tools from the caBIG life science domain, such as the caTissue suite, have been adopted by several cancer centers and NCI-supported programs.
Among other recommendations, the working group called for an immediate moratorium on ongoing software development projects internally and through commercial contracts.
The document recommends that the moratorium include tools being developed as part of the CTMS suite, caGRID, cloud computing, electronic health records, and caBIG 2.0, though it is not clear which of these programs have been eliminated as part of the restructuring announced last week.
Have topics you'd like to see covered in BioInform? Contact the editor at uthomas [at] genomeweb [.] com.