Skills in virtualization—which is both a hot new technology and one that can save enterprises lots of money in data-center costs—seem to have overcome that faint praise, however, and become genuinely hot.

"If you look at what demand is out there and the cost savings you can achieve with [virtualization], that speaks volumes. That's why it still gets the budget dollars approved for it," according to Brian Gabrielson a regional vice president of Robert Half Technology, the IT-recruiting division of global contract-employment and recruiting giant Robert Half International (RHI).

RHI's latest quarterly IT Hiring and Skills Report on the IT market predicted 8 percent of companies will hire additional IT staff this quarter, while 6 percent expect to lay some off to accommodate shrinking IT budgets.

"If either of my kids were ready to graduate from college this year and had any interest in technology, virtualization is the way I'd point them," said Gary Federico, technical recruitment manager at Advizex Technologies, Inc., which specializes in VMware-based virtualization.

"Consolidation is a key component of almost everything that's going on, so a lot of positions out there that aren't 'virtualization' positions, have virtualization as a big component to the job," he says. "People who have really good virtualization credentials are still pretty rare, though."

Virtualization, like application development, is a broad term with as many sub-specialties as there are sub-specialties within IT, and no one is expert in all of them, or even most.

So which are the five most sought-after virtualization skills?

1. Experience

It's not a skill per se, but IT pros with solid, practical experience building and managing virtual infrastructures are still rare.

"There are a lot of people out there with their VCP certification," Federico says. "VCP is a good certification and it's genuinely valuable, but compared to something like a Cisco Certified Internetwork Expert, where you have to pass both the test and a lab exam, it doesn't guarantee much hands-on experience."

Virtualization can be so complex and subtle that even technically savvy recruiters like Federico (who spent years as a Hewlett-Packard solutions engineer) can't thoroughly vet VCPs whose depth of expertise may not be what they say it is. "I have to have them go back and talk to my real technical experts," Federico says. "They can tell."

2. Architecture

The ability to understand an existing IT infrastructure and plan a virtual infrastructure that is stable, efficient and flexible is the top request from customers doing either new virtualization installs or upgrades, Gabrielson says.

"These are people who have a keen eye for the business side of the organization, and understand what you're trying to accomplish, as well as having the technical skills," Gabrielson says.

Architects have to have a deep understanding of both the virtualization technology and the IT that is already in place in order to avoid bottlenecks and inefficient load-balancing schemes. They also need to make resource sharing as efficient as possible and avoid bogging down the network with the data moving among virtual machines, virtual storage and the real infrastructure underneath, Federico says.

"It's a real trick," he says.

3. Virtual Storage

"Virtually every position request we get for virtualization has some component of storage in it," Gabrielson says. Storage is such an integral part of the pattern of data use within an organization that no IT infrastructure, virtual or not, can succeed while gumming up the storage arrangements.

Virtualizing storage (via storage area networks, primarily) should make the interface between virtual servers and virtual storage simpler, but idiosyncracies in one can cause major problems in the other.

4. Heterogeneity

A year ago it might have been enough to be a real, true expert on VMware's virtualization software; and it's still a vital component to any virtualization guru's expertise. The release of Microsoft's Hyper-V in Windows Server 2008 broke open the doors, however. Analysts such as Burton Group's Chris Wolf predict VMware and Hyper-V will have to co-exist in data centers for a long time.

Virtualization 'experts' who can handle one vendor's technology but not the other are likely to be only virtually useful as well.

5. Virtual security

Being a system administrator for virtual servers, or a whole VM infrastructure, means more than just making sure the NICs don't fall out the back of the physical servers. VM sys admins have to control server sprawl and have enough expertise in security to keep the servers safe even when users or the app dev crew spawn off rogue VMs like butterflies in spring.

"Security has become the de facto requirement for almost every job in the data center," Gabrielson says. "In virtualization, it's just that much more important because virtualization touches everything else."