Skip to main content
Premium Trial:

Request an Annual Quote

Accelrys Pins Success of Upcoming DS 1.5 Release on New Usability Testing Strategy

Premium

With its Discovery Studio 1.5 modeling platform scheduled to begin shipping just after the new year, Accelrys is putting the finishing touches on a system that CEO Mark Emkjer recently called one of the "most significant product releases in our company's history."

To prepare for the launch, the company carried out several rounds of usability testing to ensure that the new product met user needs — a "relatively new" strategy for Accelrys, according to David Edwards, director of life sciences at the company.

Edwards said that Accelrys hired an expert in human-computer interactions to lead the testing process, which the company carried out in two phases that began in the summer. "We really felt very strongly that it was the key to the success of the company and this product in the life sciences space to do this," he said.

DS 1.5, which the company previewed at its AccelrysWorld user meeting in London earlier this month, is an upgrade of the DS Modeling 1.1 and 1.2 packages and integrates four of the company's legacy modeling and simulation packages: Catalyst, Cerius2, Insight II, and Quanta. The release will also be the first to incorporate SciTegic's Pipeline Pilot workflow technology to enable researchers to share and reuse user-designed protocols, and, eventually, to integrate third-party software modules.


"We really felt very strongly that it was the key to the success of the company and this product in the life sciences space to do this."

Accelrys began developing the Discovery Studio platform in 2002, when it was envisioned as an integration platform for a suite of disparate products that Accelrys picked up in a flurry of acquisitions in the late 1990s. Soon after acquiring SciTegic last September, Accelrys began discussing its plans to make Pipeline Pilot the new backbone for Discovery Studio. After closing the acquisition, Emkjer said that the company was getting closer to its "lifetime dream" of integrating its entire product portfolio behind the Discovery Studio interface [BioInform 10-04-04]. Several months later, Steve Levine, senior director of strategic partnerships, told BioInform that Pipeline Pilot would give Accelrys "a much shorter path to the open framework we envisioned" for the DS platform [BioInform 05-23-05].

With the long-promised product nearing completion, Accelrys needed to make sure it was on the right track, Edwards said. "One of the challenges for Discovery Studio is that we're bringing together our four top brands — Catalyst, Cerius2, Insight II, and Quanta — and each of those have very different histories, and each of those user bases likes specific aspects of the product. So we had a challenge to make all four camps of users happy, rather than all four unhappy. We wanted to pay particular attention to that because we understand that the success or failure of this product really rides a lot on the usability of the product — not just what science is in it."

Standardized Testing

This summer, the company brought together around 30 users for a first round of usability testing in the US. After gathering feedback in one-on-one sessions and making a number of changes to the software, Accelrys then carried out a second round of tests in Europe this fall. Changes based on feedback from those sessions will appear in the final version of DS 1.5, Edwards said.

Accelrys used standard methods and metrics from the usability testing community, Edwards said, which provided a "standardized score that you can compare with other companies' scores."

The scoring is based on a number of tasks that reflect a user's daily workflow, including how long it takes to accomplish certain tasks, and how quickly users are able to learn new tasks without the aid of a manual. "So you can study not just whether the user can find the button they need to perform a certain operation or calculation, but also how learnable the interface is in a short space of time," Edwards said. "Those metrics are standard across all software platforms, and that's what we were really aiming to test, rather than whether they like our homology modeling product."

With no life science computing peers to use as a yardstick, Accelrys used general-purpose software products like Microsoft's Windows to gauge its success.

In the first round, "We definitely scored better than Windows ME," Edwards said, though he admitted that some might consider that particular version of Windows to be "a pretty low bar." Nevertheless, he said, "We went back and redesigned some of the user interface — so that means adding certain user aspects that customers had suggested, removing some buttons, moving things from one place to another."

After testing European customers on the same set of tasks in the fall, "the usability score increased by about 20-25 percent," he said.

'Philosophical Debate' and Customer Feedback

One of the primary questions that Accelrys hoped to answer through customer testing arose over the decision to move Discovery Studio, which was previously a Windows-based client, to encompass Linux as well. "Windows and Linux have a very different look and feel, and we had a philosophical debate about whether you make the user interface the same on both platforms, or do you make it consistent with the look and feel of that platform?" Edwards said.

The two rounds of testing indicated that "it's more important for the application to look and feel the same way on different platforms, because the user then still knows how to do things in the same fashion on whichever machine they happen to log into," he said.

Another important change that the company made based on user feedback was to regroup its software modules by function rather than their legacy product names. Users "don't really care what the module name is — it's somewhat confusing — they just know what they want to do within the application," Edwards said.

"We have this great product called Catalyst, but the product name doesn't tell you that this is what we use to generate pharmacophores and understand QSAR relationships in 3D," he added. "So when you rename the tasks, 'pharmacophore modeling,' it becomes a lot more intuitive where to find things."

While acknowledging that this appears to be "a very obvious example," Edwards noted that it "breaks a tradition or a common way that modeling companies have put their products together."

Edwards posited that this reliance on legacy modules is one reason that established life science companies don't carry out usability tests, but prefer to "refine their user interface over a period of years." He added that "companies in the life sciences space who are just starting to develop products are doing this a lot more than companies who, let's say, have had a product on the market for five or six years."

This is certainly true in some relatively new sectors of the bioinformatics market, such as pathway informatics, where vendors like Ingenuity Systems have carried out human-computer interaction studies to ensure that their software meets the needs of the user community [BioInform 10-17-05]. Some companies in the microarray analysis sector, such as Spotfire and Stratagene's Iobion software unit, have also turned to usability testing to enhance their products [BioInform 01-17-05].

Edwards said that Accelrys conducted some limited usability studies for previous versions of Discovery Studio, "but we didn't have these statistical metrics so we could benchmark ourselves. With the upcoming launch of DS 1.5, however, "we had an opportunity to change the look and feel and we wanted to guarantee that we were on the right track."

Dan Severance, associate director of computer-aided drug design at Kalypsys and a participant in the first round of testing this summer, said the experience was "an interesting exercise."

Accelrys "had some interesting ideas in the implementation, but a few of them were just a little cumbersome, and it was just a matter of figuring out how to do what they were trying to do," Severance said. "The basic idea was sound, it was just figuring out how to implement it in a more straightforward manner so that it would actually do what they had intended it to do."

Severance said that he hasn't seen the software since the summer, but that an Accelrys representative contacted him by phone to update him on the company's progress. "It sounded like they had addressed a lot of the issues that I had with it," he said.

As for whether he plans to purchase Discovery Studio based on what he's seen so far, Severance was noncommittal. "Possibly," he said, "but a lot depends on the science behind it. The usability is certainly a step in the right direction, but I can write scripts to do a lot of stuff, and in the end, I don't use an interface all that much."

— Bernadette Toner ([email protected])

Accelrys to Release Free Version of DS Visualizer

About a month after the commercial launch of DS 1.5, Accelrys plans to make a version of its molecular viewer, called DS Visualizer, freely available through its website to all commercial, academic, and government users.

The launch will resurrect the company's prior strategy to provide a free version of the viewer, which generated "great uptake from customers who wanted to have visualization but didn't want to have the modeling package," but ultimately impacted sales of the company's commercial viewer, David Edwards, director of life sciences at Accelrys, told BioInform.

"Now, there's a goal from the company's perspective to get our tools used by as many people as possible, and certainly making something free to compete with some of the other freeware that's out there is a good way to go," he said.

Edwards said that PyMol and other free molecular visualization tools are gaining popularity, but many users are "in some ways struggling to use PyMol because the user interface isn't the most friendly thing in the world."

DS Visualizer offers a better user interface than freely available packages, Edwards said, and will enable scientists who do their modeling in Discovery Studio to share results with their colleagues who don't have the full commercial version of the software.

While the company will "take a hit from a product sales perspective" with the release of the free viewer, "we think that the hit is worth it in order to be able to get the Discovery Studio Visualizer out and used by a wider scientific audience," Edwards said.

"I think we're trying to be a lot more pragmatic about the way things are in the world and not try and swim upstream so much," he added.

Filed under

The Scan

Dropped Charges

The US Justice Department has dropped visa fraud charges against five Chinese researchers, according to the Wall Street Journal.

More Kids

The Associated Press says Moderna is expanding its SARS-CoV-2 vaccine study to included additional children and may include even younger children.

PNAS Papers on Rat Clues to Human Migration, Thyroid Cancer, PolyG-DS

In PNAS this week: ancient rat genome analysis gives hints to human migrations, WDR77 gene mutations in thyroid cancer, and more.

Purnell Choppin Dies

Purnell Choppin, a virologist who led the Howard Hughes Medical Institute, has died at 91, according to the Washington Post.