Virtualization a Challenge for IP Contact Centers

IP contact centers can benefit from virtualization as much as any other technology, but get ready to roll up your sleeves.

IP contact centers can benefit from virtualization as much as any other technology, but because of the real-time nature and need for reliability, deploying virtual machines in this environment requires care in selection of hardware, segmentation of applications and the use of replicating virtual machines on the fly. (Learn more about IP contact centers.)

Contact-center elements such as voice, interactive voice response (IVR), automated call distribution (ACD) and conference bridging require near-real-time response, so virtual machines supporting them need enough dedicated horsepower, says Patrick Conroy, CTO of contact-center software vendor Callfinity, which also installs its products.

Designing the contact center requires dedicating CPU and memory to the real-time applications that need them, Conroy says. Virtual machines not running real-time applications can share remaining resources without dire consequences.

Contact centers also include less-time-sensitive business applications that pop up on call agents' PC screens to handle callers' needs and that can handle greater delay.

"So your file server or your Exchange server or whatever the case may be may wind up running low on CPU or RAM, but it's not as critical as your phone system or your ACD or your IVR," he says.

Element separation

Separation of contact-center elements on different servers also is important, Conroy says. For instance, databases tapped by multiple applications should be put on separate servers where they are unaffected by CPU and memory demands of other applications to ensure their availability.

In a typical deployment, Conroy will place a single, critical contact-center application such as telephony on a physical host with other applications that are less critical, "so if you need more capacity for your telephony layer, you take away resources from your less important virtual servers."

Others take a more cautious approach. For instance, contact-center vendor Aspect Software deploys only its own applications on virtual servers within a given physical host, says Roger Sumner, senior vice president of technology and architecture for the firm. No third-party business applications are allowed, he says.

"Over time I think the technology will be there within virtual machines to allow mixing applications," he says, "but we'll be very guarded in that area because we want to ensure that applications get delivered in an appropriate amount of time."

Meanwhile, Aspect is gathering data about the effects that other applications on the same physical host have on the real-time data center applications. Virtual environments allow setting of parameters that help ensure contact-center applications get the performance required from CPU, memory and bandwidth, says Sumner. "But when you exceed those parameters the behavior of our applications isn't as predictable," he says.

The goal is to set benchmarks that help customers design virtual deployments that mix call-center and business applications on the same hosts and allow for live migration without risking such disturbances.

"We want to show them those contact-center applications that mix well with business applications," Sumner says, so customers maximize use of server resources and have flexibility about where applications can reside.

On the fly

Virtual environments can create new virtual machines on the fly to address performance dips, but this live migration can cause separate problems, Sumner says. "There can be as much as a five-second disruption. If you're switching voice, that's fairly significant for us to deal with," he says. So again he steers clear of live migration for time-sensitive applications.

In any case, live migration characteristics vary from vendor to vendor of virtualization platforms, and businesses need to know about the differences. "Be aware of what their live migration characteristics are and design around them," says Chris Wolf, an analyst with the Burton Group. "That might result in more VMs that average a smaller load versus, say, fewer VMs that run a higher concurrent load."

Live migration can result in proliferation of virtual machines, which in turn creates more challenges, says Francois Richard, the director of infrastructure engineering at Nuance Software, which makes speech-recognition software used in contact centers. Monitoring should be segmented so like applications can be viewed at once. For example, if speech-recognition software is located on multiple virtual machines on multiple physical hosts, monitoring should be set up to view all those instances at once, he says.

Some contact-center delays can be pinned on the hardware used to support the virtual machines rather than on the software, says Wolf. Converting virtual-machine demands for CPU, memory or I/O capacity creates an overhead as these virtual demands are translated into the use of physical resources. These translations take time that can cause delays in applications that require frequent updates to these page tables, Wolf says.

AMD has developed chips to handle these translations more quickly in hardware, and Intel is close behind, he says. So businesses should consider these chips when choosing hardware on which to run contact-center virtual machines.

"In the past I've seen organizations blame latency issues on network or storage I/O when in fact it's directly related to physical memory latency," Wolf says. "Businesses have been really afraid of the latency issues and of being able to do virtualization on a large scale, but with this hardware I see more of them looking to go in that direction in 2009."

High availability -- redundancy that is critical to contact center reliability -- is another challenge in virtual environments, Wolf says. "Your typical high-availability solution today is only able to detect failure of the physical server and relocate the virtual machine as a result of that," he says. "In general it has very little insight into what is happening inside the virtual machine."

The virtual machine as black box

The virtual machine is treated as a black box, he says, which means an application can hang up inside a virtual machine without the high-availability software knowing about it. "That virtual machine would continue to operate as if nothing is going wrong," he says.

Similarly, if server hardware fails, the virtual environment needs to respond quickly to maintain calls, Wolf says. "What I mean by that is if there is a partial failure on the physical host I don't want complete loss. I might have, say, a network card go down, which in a contact center can be critical in terms of the amount of I/O I'm going to lose," he says.

In that case it would be desirable to live migrate the virtual machines on the affected host to a healthy machine. One solution called continuous availability keeps a hot standby virtual machine ready in the same state as the active virtual machine.

"With that I'm able to run a single VM on multiple physical nodes simultaneously and keep that virtual machine in lockstep so if I do have a physical node failure, the VM continues to run, and I do not lose any application state as a result," he says. Marathon Technologies has such a product, he says. "That's the type of intelligence that still needs to evolve," he says.

Beyond server virtualization, virtualizing the desktops of contact-center agents can produce further benefits but requires care to avoid degraded voice quality, says Robert Iglehart, senior vice president of IS at Thomas L. Cardella (TLC) & Associates, a contact-center provider in Cedar Rapids, Iowa. The firm uses Citrix desktops for the business applications used by agents, but relies on Avaya hard phones for voice.

"There's enough other things to worry about than to worry about what's the person hearing from me at the other end," he says. "Am I cutting out? Is there a big delay? Are we having a walkie-talkie effect? We just really don't have any tolerance for that, and that's what's keeping us from virtualizing the voice."

For contact-center applications that are not demanding of CPU, virtual machines are ideal, Iglehart says, for all the same reasons virtual machines are used in general -- redundancy and moving virtual machines on the fly as capacity demands.

"It's more reliable because you're concentrating your applications in the data center on better equipment. Storage is put on a SAN. "You don't have desktops on agent stations getting beaten around every day," he says.

TLC supports specific business applications for each of its customers, Iglehart says, and virtualizing the desktops reduces maintenance of these applications. Each customer wants a unique set of scripts, reports and order-entry systems, he says.

"We end up supporting a lot of different applications for our clients, so there's a lot of change that happens a lot more than in other industries," he says.

Changes can be implemented on a virtual desktop to see whether they are compatible, then rolled out centrally where they are accessed by the remote machines, he says.

"Virtualization makes it almost a nonissue to add more seats and new locations because all you need is a network connection," he says. "The applications are already where you need them."

The overriding factor in virtualizing contact centers is having thorough knowledge, either in-house or via consultants, of virtualization platforms, experts say.

"I wouldn't suggest somebody trying to cut their teeth on virtualization with their call-center deployment," Wolf says. "If they don't have the expertise, they should get it. There's more than one way to skin a cat in virtualization, and if you start out wrong it's going to cost you more in the long run than it will if you get some help up front."

This story, "Virtualization a Challenge for IP Contact Centers" was originally published by Network World.


Copyright © 2009 IDG Communications, Inc.

7 secrets of successful remote IT teams