by Kim S. Nash

CIO 100 Award-winners Are ‘Risk Busters’

Jul 27, 201219 mins

Many of the honorees in the 25th annual CIO 100 awards program are using IT innovation to reduce risks that threaten employees, customers and entire business models.

CIOs who help create corporate strategy also find themselves responsible for the risks that come with it. These are not the traditional IT-centered aspects of risk management, such as disaster recovery and data security. Rather, CIOs must now manage fundamental corporate risks as serious as outdated business models that threaten to kill the company or faulty processes that could harm employees or customers.

Squashing looming business risks stands out as a key theme among the winners of our annual CIO 100 awards. We honor companies whose projects put IT innovation in the center of their growth strategies. Many of this year’s winning projects demonstrate how CIOs lead the way in changing how colleagues work, thereby mitigating profound risks to the company.

“We in this profession have to frame our work in terms of risks and payoffs to the business, not technology projects to check off,” says Paul Stamas, vice president of IT at Mohawk Fine Papers, a paper mill that is expanding into digital publishing. Stamas, his CEO and the rest of the company kicked off a corporate transformation two years ago, after they saw a bleak future for Mohawk if it continued to rely on dead-tree paper manufacturing as its only business, he says. In his winning project, Stamas raced to set up an IT infrastructure in the cloud that would let the company experiment with new business models.

It’s easier to identify serious business risks when you see the company as a collection of processes–such as dreaming up new products or interacting with customers, says David Kruzner, co-head of consulting at iGate, a consultancy and outsourcing company. The places where those processes slow down or fall apart point to risks that can rot the very core of the company, Kruzner says.

CIOs are as accountable for making change successful as the CEO or the head of the department where the change is happening, he says. “CIOs are co-owners of change.”

To smooth the way, CIOs have to take several steps to make sure colleagues are ready for change. Before any work begins, you should meet with the non-IT peers who are asking for the project and have a brass-tacks talk. Discuss how the company has agreed to spend the money and how everyone’s excited about the business case but none of it will materialize until–and this is the brassy part–business leaders commit part of their time, staff and budget to the work. “I’ve seen CIOs say, ‘If I don’t have you, I’m not doing it,'” Kruzner says.

This year’s CIO 100 winners got those commitments and used IT-led innovation to vanquish various types of business risk.

Risk: Losing Customers, Damaging the Corporate Reputation

Remember the ancient days in 2010 when you couldn’t get the iPhone on any network other than AT&T? The moment Apple and Verizon finally struck a deal that year, Shankar Arumugavelu, CIO of Verizon Wireless, knew that pent-up demand for a Verizon-compatible version of Apple’s phone, set to launch in 2011, might sink his customer order system. He did not want IT problems to hamper Verizon’s sales to existing customers, new customers and especially, in competitive spirit, AT&T defectors, he says. He also didn’t want to risk damaging Verizon’s reputation in these times of angry tweets and intense media coverage of iPhone mania.

The IT group designed the Iconic Phone Launch portal, a Web-based system made to handle an onslaught of orders for popular devices. The team finished the design in October 2010, which left Verizon three months to build, test and deploy it before Apple released the new phone. In addition to high transaction volumes, the portal had to handle pre-orders and back orders, which were unheard of before smash hit phones like the iPhone and Droid Razr, he says.

Another complication: Many AT&T customers switching to Verizon would want to keep their old cell numbers. Arumugavelu made sure the third parties that port phone numbers could connect their systems to the portal. “We had to make sure that didn’t become our weakest link” and cost the company customers, he says. “First impression is everything. We take it seriously.”

On a regular day, Verizon Wireless sells about 60,000 devices. On the day in February 2011 when the iPhone 4 debuted on Verizon, the system handled peak volumes of 160,000 orders per hour. The portal also cut order-processing time from eight minutes to four. “This was a big event,” he chuckles. And a flawless one, made all the sweeter as he watched AT&T’s systems falter.

If the portal had started to slow, Verizon was ready with a Plan B, which entailed processing orders without using functions deemed nonessential. For example, if an order was placed by an existing Verizon customer, the system wouldn’t bother with real-time credit checks, to save time and decrease network traffic, he says. Instead, under this Plan B, the credit checks would come later, during less-busy hours.

It was critical, Arumugavelu says, that the portal not only address the big business risks that would result from the deluge of orders but also handle the smaller risks that crop up along the way. Building real-time fraud detection, including credit checks, into the ordering process means Verizon can stop iffy transactions before they go through, eliminating the need to revoke orders later, after losing money as bad actors skip out on payments. Such technology-enabled changes in business processes have saved Verizon $300 million so far, he says.

Verizon Wireless plans to use the portal as the foundation of a new point-of-sale system that works across all its channels–physical stores, website, customer call center and indirect outlets such as other retailers.

Risk: Getting Beat by Competitors, Getting Stuck for Multimillion-Dollar Refunds

Boeing predicts that in the next 20 years, the air travel industry will need 34,000 new planes to meet flying demand worldwide. That’s potentially $4.5 trillion worth of planes. But all that business could easily remain out of Boeing’s reach, executives at the $68.7 billion company realized.

Why? Boeing has an order backlog that, if nothing changed in its processes or technology, would take seven years to fill. Delays in producing the company’s vaunted Dreamliner, for example, have already cost Boeing millions. One customer, Air India, ordered 27 Dreamliners in 2006 and received the first one just this June, three years late. The airline demanded up to $1 billion in compensation for its trouble; Boeing offered $145 million. The companies recently agreed on an undisclosed sum.

Simply put, Boeing has to get its factories working faster, says CIO Kim Hammonds, “for Boeing to realize its potential in the marketplace.”

Before Hammonds thought about technology changes, she worked with non-IT leaders to figure out how to improve the workflow by making changes in the engineering, manufacturing, quality-assessment and supply-chain processes. Each of these monster systems involves hundreds of applications. To focus the evaluation, Hammonds’ team looked at how individual people did their jobs daily to see where the existing technology helped or hurt.

A mechanic on the shop floor trying to open a set of plans might have to wait 10 seconds to see a digital 3-D drawing and its associated documents, which tell him what parts are needed to build the object the illustration shows. Lots of small waits add up to slow manufacturing, she says. But Boeing needs big changes, too, or it won’t achieve the production rates it needs. Right now, the company makes 3.5 Dreamliners a month, but it expects to build 10 a month in 2014, Hammonds says.

To get there, manufacturing, engineering, quality management and supply chain applications were upgraded, and the interfaces between the systems were overhauled to simplify processes such as ordering parts, updating contracts and scheduling plane production. In all, Boeing rolled out 35 new applications, enhanced 100 others and tweaked 50 interfaces. The project is one of the largest in Boeing’s history, with more than 500,000 man-hours spent on it.

Stepping up Dreamliner production is just part of Boeing’s larger goal of increasing manufacturing rates by 40 percent within the next three years. That will take years off its backlog and make room for new orders, Hammonds says. “We needed to do this.”

Risk: Death, Environmental Damage, Financial Losses

Jim Noble, senior vice president of IT at Talisman Energy, “watched with horror” as the BP oil spill unfolded in the Gulf of Mexico in 2010. He had left BP two months before the explosion at BP’s drilling site, which killed 11 people and released nearly 5 million barrels of oil.

Noble doesn’t know exactly what went wrong for BP, but he takes the oil industry’s inherent risks to heart. “In our business, we want people to go home at night safe and sound. You don’t have that if you’re working at a hedge fund or in retail.”

So Talisman built a system designed to reduce the risks it faces. It can cost up to $100 million to put together a deep-water drilling expedition and $1 million a day to run it, but only about one in three expeditions turns up valuable oil. Refining the process of building and running a successful rig will save money and lives, he says.

Noble’s team pieced together a system that tells Talisman more than it ever knew about what’s happening at a deep-water drilling site, almost as soon as it happens.

Rigs erected 150 miles offshore plunge a mile through the ocean and another mile or more into the sea floor. Weather and other atmospheric changes, problems with equipment or a lack of information can lead to poor decision-making. “Standing on one of these drilling rigs with 30-foot waves breaking over you, it’s dark and noisy,” he says. “You’re just not giving the job your full, undivided attention. That’s when people make mistakes.”

Talisman enabled high-bandwidth communications at the drill pipe to transmit thousands of megabits of data per second from sensors on the drill bit, including temperature, pressure, speed and depth. To avoid the half-second or one-second latency of satellites, Talisman is implementing a technique used by the military to bounce microwave signals off the troposphere, a moist layer of the Earth’s atmosphere.

Talisman also built a real-time dashboard to bring together geoscience, drilling and engineering data that had previously remained in three separate systems. Visualizations of the data show scientists (warm and dry at Talisman’s headquarters) what’s happening at the wells so they can advise the crews to adjust the drilling equipment accordingly. They can guide the drill bit to avoid obstacles as it digs, for example, and respond to emergencies faster and with more accurate information.

Before the new processes and technology were in place, Talisman experts could make drilling decisions every five to 15 minutes. Now adjustments can be made every second, he says.

Talisman has been able to reduce staffing on the rigs, as well as the number of times employees had to perform “interventions” on the machinery to fix or avoid a problem. Lost-time incidents, where work is stopped or slowed because of safety issues, are down. No one wants another BP disaster. “Failure to detect and adequately respond to these changing conditions can cost hundreds of millions to billions [of dollars] and, unfortunately, people’s lives,” he says. “[We] want to do something to avoid it ever happening again.”

Risk: Failing Compliance Audits, Catastrophic Patient Treatment Delays

Last year the emergency department of the Christiana Care Health System handled 172,000 patient visits, including 3,841 traumas–incidents where a person’s limbs, organs or life itself are at risk. Information about what happens to each patient must be recorded to comply with regulations, to keep accreditations and to be available in case of audits, internal peer reviews or legal requests. Until recently, a good chunk of that work was done by sharp-eyed staff with pen and paper.

For example, a clinician treating an emergency room patient often wants to consult with a neurologist, cardiologist or other specialist on call. The clinician would have to take a minute or two to find a clerk to make the call. The clerk would do so, then log the calls on paper, noting the time of the outbound call and of the response. About 150,000 calls to and from clerks are logged per year. It wasn’t possible to do any systematic analysis of this paper information, and gathering data for audits and certification requirements or to respond to complaints or lawsuits was “a huge burden. [The paper logs] were stored on pallets in an Iron Mountain warehouse that required forklifts to pull the books for people to review,” says Randall Gaboriault, CIO. “It would take hours and hours to go over logs.”

When a trauma patient enters the emergency department, a nurse must document which areas of the body each clinician treats. Christiana nurses had traditionally done this by observing and writing the information down. They had to know people, even behind their surgical masks, or ask who was who amid the chaos. Such documentation is necessary not only for internal performance monitoring and legal issues but also to qualify for accreditation as a level one–the highest level–trauma center.

Gaboriault led a project to automate and streamline these processes with workflow and collaboration applications, combined with interactive visualization software. Now, an emergency clinician signs in to a workflow application on a computer right in the exam room and requests, via the software, that the clerk contact a specialist. The clerk does, and logs the information electronically from his desk. The clinician doesn’t have to leave the patient to find the clerk.

In trauma cases, each clinician swipes his smart badge at a computer outside the trauma bay. The application accesses the hospital’s human resources database to identify him and to populate an interactive display. While there, he can touch the monitor to claim an area of the patient’s body to treat, such as the airway or left chest. The doctor enters the bay, where he can see another screen displaying statistics and data about the patient. The display also shows names and pictures of the clinicians at each area of the patient’s body, for everyone’s reference.

“They have all the same information at the same time,” Gaboriault says. The treatment bay is quieter and the documentation nurse isn’t confused. “Even if it’s seconds of friction out of the process, it speeds up the time between knowledge and action.”

Compliance records are more accurate and no one pours over paper logs to fulfill data requests.

Risk: Excessive Downtime Costs, Deficient Patient Care, Data Errors

In 2008, the Cancer Treatment Centers of America (CTCA) did something many healthcare organizations struggle to do: implemented a full electronic health record system. Having gone fully computerized, patient care was now dependent on 100 percent IT uptime–a nirvana not achieved, says Chad Eckes, CIO of the CTCA, which runs a network of five hospitals across the United States.

The electronic health record system and other software require weekly updates and patches, which meant that the system had a planned downtime of four hours each week. Unplanned downtime last year averaged an additional 43 minutes per year. When the system was unavailable, up to 3,000 doctors, nurses and other staff members had to use workarounds, Eckes says.

Patient safety was not at risk, he says; read-only copies of patient records were available locally to the medical staff during these periods, which were usually 11:00 p.m. to 3:00 a.m. But clinicians had to write out the necessary information and enter it later. They also were unable to care for as many patients during the downtime as they could when the systems were up, necessitating extra staffing. Downtime, Eckes says, “is inexcusable in a world where you have applications impacting patient care.”

Re-keying data always heightens the risk of errors, he says. Plus, using paper, even for a short time, disrupts the workflow of the staff, especially for younger clinicians who may have never worked in a paper world.

Eckes built a private cloud in a backup data center that mirrors the virtualized servers and 342 applications in its primary data center. The systems communicate with each other continuously, moving the work of physicians, nurses and other users between production environments without disruption, he says. The CIO likes to perform a dramatic test where he cuts power to a full rack of equipment and the SAN that runs the electronic health record software and watches the system self-heal by bouncing to the backup cloud. In the 18 months since the system launched, the company has had no downtime in the data center. “The end user sees nothing. Not a pause or blip. The response time is sub-second.”

Risk: Injury, Death, Property Damage

Brampton, Ontario, is the second-fastest-growing city in Canada, but the city budget isn’t keeping up, putting the fire department in a tough situation. It has to serve more citizens, who are packed more densely, while it controls costs. This risk caused by this familiar “do more with less” demand becomes more intense when you consider that human safety is at stake.

In 1995, for example, Brampton’s fire department handled 7,597 emergencies, including fires, car accidents and hazardous materials incidents. Last year it was 17,607–more than double.

“It’s not sustainable to continue to add staff and vehicles as the city grows. We have to look at how to sustain growth with fiscal pressures,” says Rob Meikle, CIO of the City of Brampton.

The city’s IT group worked closely with the fire department to determine how to route vehicles more efficiently, to save time and improve service. But several factors complicate routing.

In a fast-growing city, traffic patterns continuously change, which disrupts time-tested fire truck routes. Also, the fire department used to dispatch trucks from the station nearest the location of the incident, even though a truck and crew that was already out of the station may have been closer. There was no electronic system to track that information. Staff members used radio to share information about their whereabouts and the details of the emergency.

For Matt Pegg, deputy fire chief, the goal is to put responders on the scene as fast as possible, equipped with as much information as possible. But any new tools, systems or procedures had to be impeccably reliable, Pegg says. “In the world of fire, the only thing worse than not giving me a tool I need is giving me a tool I can’t rely upon.”

IT worked with the fire department to build a new system where trucks carry location-based sensors so the routing software can find the nearest fire truck, rather than just the nearest station.

Next, the team had to deal with the physical risks of navigating heavier and less-predictable traffic. Before, signals would detect the flashing lights of an approaching truck and change to red, to stop traffic and let the truck weave through the intersection. As the city grew, however, more buildings went up, blocking the trucks’ flashing lights. Now, GPS data from trucks is transmitted to the system that controls traffic signals by a new data network, which the city was already installing for smart buses and a rapid transit system. That happens as soon as a truck is dispatched. The signals then immediately start to “precondition” intersections, meaning they adjust green, yellow and red lights to clear traffic from the truck’s route, Pegg says, and ensure fire vehicles always have a green light.

“In the past, we were required to stop at a red light, then proceed through it. That’s dangerous,” he explains. “This is an efficient and fast response to the specific emergency but also improves public safety along the way.”

Risk: The Wrong Business Model at the Wrong Time

As its name suggests, Mohawk Fine Papers is–or was (we’ll get to that)–a maker of high-end specialty paper. Fancy stationery, book covers and the like. But the 81-year-old, family-owned business faces a triumvirate of forces gathering strength in the past few years and threatening to put the company out of business: the shift from paper to online publishing, the environmental push to conserve trees, and global competitors that offer cheaper printing services.

“Our survival was at stake,” says Paul Stamas, vice president of IT at the $300 million Mohawk.

Senior executives set a goal two years ago to become an agile company that could create new products and services to embrace, rather than fight, the disruptive trends, Stamas says.

“This was no IT-only exercise, Stamas says. “The CEO issued a call to action to the rest of the company [saying] ‘We have to change or we won’t survive.'”

Mohawk closed one plant and stopped making low-margin products such as brochure paper, outsourcing that work to Asian companies. It started making envelopes to go with the paper product lines it had kept. It also started selling products directly to consumers and businesses, cutting out some of the distributors it had previously used. The company wanted to be closer to its ultimate customers to sense trends more quickly, Stamas says.

Mohawk also got into a new business: online photo sharing. It now runs a site that offers printing services for calendars and books. “We’re driving demand for our paper but creating an entirely new business,” Stamas says. It also bought a plastics company so it could sell magnets and signs customized with photos–products with higher profit margins than paper.

In April, Mohawk felt confident enough in its new endeavors to drop the “fine papers” portion of its name from marketing materials and other corporate communications.

With an IT staff of just five people, cloud computing has enabled Mohawk to experiment with new business ideas and partnerships, then extricate itself from them quickly if necessary, Stamas says. For example, the company learned its envelope business from six smaller manufacturers, connecting with them via the cloud. Mohawk combines Amazon’s cloud with its own EDI, warehouse management and financial systems. When Mohawk was ready to start making its own envelopes, it dropped the partners with little impact on its internal IT systems, he says.

Mohawk is now smaller and makes fewer products but is twice as profitable as it’s ever has been, Stamas says. He and his staff, he says, “are business process orchestrators. I’m not thinking much about the technology.”

Follow Senior Editor Kim S. Nash on Twitter: @knash99. Or read her blog, Strategic CIO.