Technology Nightmare: How to Protect Your Network from the Threat of Rogue IT Employees

Terry Childs, a network administrator for the City of San Francisco is accused of creating a super-password on the switches and routers in the city’s Fibre WAN and using it to block everyone else’s access to administrative functions. According to reports, Childs had beendetected tampering with the network and had reacted with hostility when disciplined after a confrontation with a supervisor.

As a result of Childs’ alleged actions, administrators are unable to access the routers and switches, although the network continues to function. Childs was charged with four counts of computer tampering and held on $5 million bail.

A week after the incident the city still hadn’t gotten access, and details were still sketchy. However a few things are obvious.

“This should never have happened in an organization of this size,” Cameron Laird says flatly.

The need to protect organizations from rogue employees existed long before computers were invented, notes Laird, the vice president of Houston, TX, security consultancy Phaseit. “There are principles people have been working out for a couple of millennia,” Laird says. “I think we’re best off working from models that enjoy more experience than we do in IT. For instance accounting and auditing where we’ve got a few hundred years experience.” Some of those principles, like access control, have been incorporated into IT culture. Some, like least privilege are only beginning to be widely incorporated. Some, like dual authorization, haven’t made it into the culture yet.

Unfortunately in the Childs case, many of those principles were apparently ignored. Reports of the incident indicate that while the city was routinely logging administrative activity on the network, they failed to act quickly and decisively when they found the first signs of Childs’ activities.

Best practice in these situations is to immediately deny access to the system pending a review. For example, the Nuclear Regulatory Commission’s rules for nuclear power plants require that access to important systems be immediately revoked if any suspicious activity is detected.

Another problem is that the city apparently did not effectively apply the principle of least privilege. A network administrator obviously needs wide-ranging access to the system being administered, but that is not the same as unquestioned, unrestricted access. Childs apparently had the ability to create a super password and alter other administrator’s privileges at will. While his activities were logged, logging amounts to locking the barn door after the horse is stolen.

In theory, employees at any level should be granted only those privileges absolutely needed to do their job. Since this requires a separate set of privileges for everyone but the lowest ranking employees, this is usually impractical. As a result we tend to assign employees to groups with the same privilege levels, whether that specific employee needs all those specific privileges or not.

The key to making this effective is granularity. For access to some critical functions—like changing administrative privileges—the granularity should be very fine indeed. Since relatively few people in the organization have or need such access to those critical functions, this is much easier to manage.

“For an organization of any size, say the government of a city of a million people, you really need to get serious about how you manage privileges,” Laird says.

Laird points out that in the military, NASA and other organizations, critical actions require more than one person’s action. The classic example is launching a nuclear missile where at least two officers have keys to the firing console and both must use them simultaneously. Requiring more than one person to perform a potentially damaging action means that it will probably require collusion of two or more people and lessen the likelihood of a problem.

Much of what happened in San Francisco falls under the heading of identity management and access control. There are a number of companies with good IM/AC packages out there, but generally there is more focus on identity management (setting and protecting passwords) than access control (deciding what those passwords let you get to). Certainly an action like creating a super password should require more than the appropriate privilege level.

But much more significantly, all the software in the world can’t substitute for awareness of the issue and a willingness to take steps to prevent it.

Rick Cook has written thousands of articles and several books on computers and management. He is also the author of several fantasy novels full of bad computer jokes.