SBC COMMUNICATIONS: More Lines, Less Waiting with Network Management Tools

In the old days, when telecommunications companies held a monopoly on local and long-distance service, lost business wasn’t a major concern. But when the competitive barriers came down in the late ’90s, customer retention became a huge issue. That’s when PacBell developed its Equipment Capacity Optimization Systems (ECOSystems), a homegrown suite of network capacity management tools first developed and deployed in 1997. ECOSystems allows engineers to automatically detect and respond to network capacity shortages with its data analysis and visualization functions. And with this fast, accurate data, SBC Communications?which acquired PacBell in 1998 and began rolling out ECOSystems throughout the parent company in 1999?now has faster network capacity management, improved customer service capability, and new metrics and business intelligence to measure network capacity and plan expansion.

For Noah Yates, the value of ECOSystems is more straightforward; as the 31-year veteran tells it, ECOSystems has given him his life back.

Before 1997, if Yates, a network engineer at SBC’s PacBell subsidiary in San Diego, wanted to know when the telephone company’s network capacity was near exhaustion, he had to do it manually. In technical terms, this meant pulling data from several different legacy systems and compiling it into demand versus capacity charts, which he would then pore over in an attempt to determine equipment utilization, network capacity and future demands. “It meant six and a half hours in the office on a Saturday, eating lunch at my desk, with no phone calls or interruptions,” Yates says. And even then, because the information was anything but real-time, the data might be hopelessly outdated by the time Yates finished analyzing it. “We were exhausting [network] capacity without even knowing it,” he says. Sometimes these shortages weren’t discovered until Yates and his colleagues tried to fill customer orders for new lines or services. Then the engineers would have to place rush orders for new equipment installations, which cost the company 25 percent to 35 percent more in operating expenses than planned expansion, or customers would walk.

To SBC executives who have deployed ECOSystems throughout the company’s four subsidiaries in 13 states, the project’s biggest measurable success is that it’s taken an all-day task for engineers and diminished it to under 10 minutes. “We have better, faster tools, which give us better predictability,” says Andy Geisse, vice president of enterprise software solutions, the IT group at SBC’s PacBell offices in San Ramon, Calif. Christopher Rice, senior vice president of SBC’s network planning and engineering group at SBC headquarters in San Antonio, Texas, notes the IT and business collaboration that resulted in ECOSystems. “I’ve never been more impressed by IT folks [collaborating] with engineers,” Rice says.

Frankly, the capabilities provided by ECOSystems couldn’t come at a better time. Like most telecom companies, SBC has hit tough times. Last October, the company delivered lower-than-projected third-quarter results and announced it would eliminate several thousand jobs and cut capital spending by approximately 20 percent in 2002. In this cost-conscious environment, ECOSystems is credited with having saved SBC $22.3 million in operating expenses since its initial deployment. By 2003, SBC expects, savings will reach $10 million annually.

In awarding the company a 2002 Enterprise Value Award, judges were struck both by the capabilities and cost savings embodied in ECOSystems. “I’m impressed by [SBC] going right down to the core of what they do, and making it significantly better,” says judge John Glaser, CIO and vice president of Partners HealthCare System in Boston.

Born of Necessity

The need for better network capacity management became glaring during the Internet boom of the late ’90s. “Prior to the Internet, we were able to do capacity management on more of a planned basis?we knew the peak times were [on workdays and holidays],” says Ronald Ong, the technical director of IT and engineering systems who headed the ECOSystems development team. “With the Internet, conditions are not as predictable.” Suddenly, businesses required more lines, bigger bandwidth and more functionality, all on a 24/7 basis. Rather than risk exhausting capacity or losing customers to an increasing number of competitors, PacBell undertook the ECOSystems project.

To date, developers have rolled out four separate ECOSystems applications, two of which have been deployed long enough to meet the time requirements necessary for Enterprise Value Award consideration. These are:

• Rule-Based Capacity Management

  This first tool, deployed in June 1997, monitors the capacity, demand and maintenance records for roughly 30 different types of equipment (switches, relays and so on) throughout the network. Engineers use this tool to create graphical images that show them instantly whether their systems are at or near capacity.

• Sonet Analysis and Visualization

  Deployed in August 1998, this tool allows engineers to generate demand versus capacity charts as a topography graph. These graphs use colors to show where systems are at or below capacity. At a glance, engineers can see which parts of the network need attention, reducing their analysis time from about six hours to eight seconds.
  But more than just allowing engineers to monitor systems, those ECOSystems tools send out e-mail alerts whenever there is a network capacity issue that could result in disconnections or busy signals. Then engineers can view the problem, diagnose it and write a justification for the work as required by California’s Public Utilities Commission before it becomes an expensive rush job.

  Today, 700 engineers are using ECOSystems, and as a group they receive an average of 150 e-mail alerts daily, resulting in an average of 500 system reviews. The engineers are now nipping potential capacity situations before they occur, resulting in fewer rush jobs. These reduced operating expenses (engineers’ time and rush jobs eliminated) account for the $22 million SBC has saved so far since deploying ECOSystems.

  IT as Hero

  There have been small technical challenges in adapting ECOSystems to accommodate different terminologies at the various SBC subsidiaries. But there has been no significant user resistance, executives say. “It’s been kind of a Nirvana system,” says Edward Glotzbach, SBC’s CIO and executive vice president. “Not only was ECOSystems not that expensive to deploy, but it’s also proven extremely useful, and everyone loves [IT] for it.” To Glotzbach, the field acceptance of ECOSystems gives IT a business-side credibility it can leverage in future projects. “When you can improve the quality of life of the employee serving the customer, that always makes IT a hero,” he says.

  Next, developers want to incorporate artificial intelligence technologies to make ECOSystems respond not just to current network conditions but also to something that engineers can do today only through guesswork: Anticipate and plan for future demand capacities. “This system tells me where I’m at with today’s network, but I want it to tell me what I can do in the future,” Yates says. “That won’t be easy.” But it also can’t be as hard as life before ECOSystems.