Looking back at Cedars-Sinai’s experience, Dr. M. Michael Shabot, Cedars-Sinai’s medical director of enterprise information services, says that, at the time, there was little awareness of the difficulties involved in a large CPOE implementation. "Given the money and time that is spent on this system, it has to be meaningful to the physician," says Shabot. "The reason for it has to be very clear."

A Technology in Its Infancy

When the UPenn researchers published their CPOE study in JAMA, Williams was in the midst of his complicated CPOE project at St. Jude’s. Williams acknowledges that after reading the JAMA study, he recognized some possible shortcomings in his own CPOE system. As a result, he recently requested the Meditech system provide a link between procedures and medication orders so that if the procedure (such as open-heart surgery) is canceled, the correlating medication order would automatically be canceled as well. In addition, his team is working closely with Meditech to improve integration of the pharmacy and physician order management to make sure that the orders are going through in real-time, without delay.

In a written response to the JAMA article, circulated among his staff, Williams wrote: "The safety and effectiveness of CPOE arises from the understanding that it is primarily a process change, a major transformation in the procedures surrounding the delivery of patient care."

Even those who question the study’s methodology agree that it highlighted the stress that a CPOE rollout can put on a hospital and the need to constantly monitor and fix the systems. After implementing a homegrown system at Brigham and Women’s, Dr. David Bates, chief of the division of general medicine, wrote in the Journal of Biomedical Informatics, "We routinely tracked errors and problems that were created, and made thousands of changes to the original application." If he had one thing to do over, Bates says he would have devoted even more resources to this area. "It is just impossible to get it all right at the outset, because the processes involved are so complex," he adds. And some say it is harder to make all of the necessary changes if you’re installing a system designed by a vendor. Bates says, and other practitioners agree, that just as with any new technology, CPOE must be aggressively monitored and tinkered with if it is going to succeed.

The Leapfrog Group’s Classen is leading a project to create what he calls a "CPOE flight simulator," which hospitals and outpatient clinics will be able to use to test their CPOE and decision support systems to gauge whether they "prevent or cause harm" among patients. "What’s clear is that not all medication errors are created equal," Classen says. "We need to find the ones that cause harm or death."