As a result of Childs' alleged actions, administrators are unable to access the routers and switches, although the network continues to function. Childs was charged with four counts of computer tampering and held on $5 million bail.

A week after the incident the city still hadn't gotten access, and details were still sketchy. However a few things are obvious.

"This should never have happened in an organization of this size," Cameron Laird says flatly.

The need to protect organizations from rogue employees existed long before computers were invented, notes Laird, the vice president of Houston, TX, security consultancy Phaseit. "There are principles people have been working out for a couple of millennia," Laird says. "I think we're best off working from models that enjoy more experience than we do in IT. For instance accounting and auditing where we've got a few hundred years experience." Some of those principles, like access control, have been incorporated into IT culture. Some, like least privilege are only beginning to be widely incorporated. Some, like dual authorization, haven't made it into the culture yet.

Unfortunately in the Childs case, many of those principles were apparently ignored. Reports of the incident indicate that while the city was routinely logging administrative activity on the network, they failed to act quickly and decisively when they found the first signs of Childs' activities.

Best practice in these situations is to immediately deny access to the system pending a review. For example, the Nuclear Regulatory Commission's rules for nuclear power plants require that access to important systems be immediately revoked if any suspicious activity is detected.

Another problem is that the city apparently did not effectively apply the principle of least privilege. A network administrator obviously needs wide-ranging access to the system being administered, but that is not the same as unquestioned, unrestricted access. Childs apparently had the ability to create a super password and alter other administrator's privileges at will. While his activities were logged, logging amounts to locking the barn door after the horse is stolen.

In theory, employees at any level should be granted only those privileges absolutely needed to do their job. Since this requires a separate set of privileges for everyone but the lowest ranking employees, this is usually impractical. As a result we tend to assign employees to groups with the same privilege levels, whether that specific employee needs all those specific privileges or not.