Virtualization and private clouds have delivered hardware savings in a big way for most enterprises. Software savings, however, are proving more elusive, and that's been particularly true when it comes to virtualizing Oracle databases, according to several sources.
While virtualization has enabled server consolidation ratios of 3:1 or more, businesses may see little or no reduction in associated software costs. In some cases enterprise software licensing can be so expensive that it overshadows the cost of the rest of the system stack combined, says David Welch, chief technology officer at House of Brick Technologies, an integrator with expertise in Oracle software licensing issues.
(Editor's note: During the months of reporting this story, we contacted Oracle 13 times, and the vendor's spokespeople declined to comment on this story on three separate occasions.)
Some consultants and a handful of IT executives say they face licensing obstacles with many enterprise software vendors, but Oracle agreements can present some of the most confusing compliance issues, especially for customers that fall under Oracle's processor-based licensing models. "Using Oracle on VMware provides no licensing savings, just operational cost savings," says R Wang, principal analyst and CEO at Constellation Research.
"Virtualization is an area where customers get creamed all the time," says Craig Guarente, CEO of software auditing and compliance consultancy Palisade Compliance. Both Guarente and Welch say that, all too often, organizations end up overpaying for new Oracle licenses as well as software maintenance after consolidating servers or adding new licenses in a virtual environment.
Several IT executives contacted for this story declined to speak on the record. One executive, "Tom," has dealt with Oracle as the CIO for several Fortune 500 corporations, and agreed to speak only on condition that his name and company affiliation not be used. "Part of the challenge is knowing what others are paying," he says, and that information is considered confidential. But professional negotiators can be helpful because they know what the average company is paying for any given product, he adds.
Some Oracle customers are "p.o.'d" about virtualization, Guarente says.
Story continues on next page.
Negotiating with Oracle: Experts' tips
- When executing an Oracle software license agreement, visit the Oracle contracts page and print out or store copies of all online policy documents to which the contract refers as a class.
- Be sure to include the Oracle Partitioning Policy document (PDF), Oracle Processor Core Factor Table (PDF), Software Investment Guide, Licensing Data Recovery Environments (PDF) and Oracle Software Technical Support Policies (PDF). These have information not otherwise specified in the customer's Oracle Software Licensing Agreement (OSLA).
- The documents can change over time, so it is important to have copies of the policies that are in force at the time the OLSA is executed.
- For more information on Oracle licensing in a VMware environment read the VMware white paper, "Understanding Oracle Certification, Support and Licensing for VMware Environments" (PDF).
- Avoid overspending on new Oracle licenses when moving servers to a virtual environment, especially when it comes to vSphere clusters. Oracle may ask users to pay a licensing fee for every server in a cluster -- not just the servers where the Oracle products are installed and/or running -- "but that condition is not spelled out in the software license or associated policy documents," says Guarente.
- Inventory your Oracle licensing paperwork before starting a server consolidation project to see if you can allow maintenance contracts to lapse selectively for Oracle licenses associated with retired physical servers. Note that Oracle's contracts group individual licenses within so-called license sets. Oracle's matching support levels policy states that you can't cancel maintenance and support on a subset of licenses within a license set without also cancelling those licenses. That may trigger a re-pricing for the rest of the agreement. So renegotiating the contract may not save you any money.
- Carefully consider the implications of any invitation to migrate all of your Oracle licenses into a single agreement. Such initiatives rarely work in the customer's favor, professional negotiators say. Keep Oracle software licensing contracts separate whenever possible for maximum flexibility. Also keep in mind that Oracle's License Sets policy works against allowing maintenance and support to lapse on all product licenses within a given Customer Service Identifier (CSI).
- According to what's commonly known as Oracle's 10-day rule, you're entitled to leave one node unlicensed in a vSphere cluster -- even if you dedicate every other physical server in the cluster to Oracle -- for up to 10 cumulative days, so long as that node is named as a designated failover node. The failover node must also share a storage array with the source node. See Oracle's Software Investment Guide and Licensing Data Recovery Environments documents for details.
Sources: House of Brick, Palisade Compliance
-- Robert L. Mitchell
One client called Palisade after installing Oracle database software on two servers in an eight-server vSphere cluster. Oracle discovered the configuration during a software audit and demanded that the customer buy a license for every processor and server in the cluster.
We haven't noticed other enterprise software vendors that assert the full-cluster licensing claim. David Welch, chief technology officer, House of Brick Technologies
That's unusual, says Welch. "We haven't noticed other enterprise software vendors that assert the full-cluster licensing claim," he says. Oracle's per-processor licensing fees start at $47,500, so charges can add up quickly. To make matters worse, the customer had also unknowingly installed the more expensive Enterprise Edition of the Oracle database instead of the Standard Edition.
Before the audit, the company had been paying Oracle $50,000 per year. The list price for licensing the new configuration: Just over $1 million. The customer ended up reconfiguring the vSphere infrastructure to limit the number of servers in the cluster, and bought more Oracle licenses. Guarente wouldn't say exactly how much the customer saved, but did say the customer paid a fraction of that original $1 million bill.
The company could have achieved the same goal without paying anything additional had it reconfigured the cluster to run Standard Edition before the audit took place. Or it could have gone back to Oracle and bought more licenses, Guarente says. "The first option would have resulted in no additional fees to Oracle. The second would have resulted in some fees, but far less than what they ended up paying."
Bob Clarke, manager of enterprise services for the Office of Technology in the State of Indiana, is considering his options for saving money. Like the customer described above, he also faces the requirement to license every core and server in the cluster.
"It gets very pricey," he says, but he hopes to make the numbers work by consolidating some of the existing licenses for Oracle software used by separate departments. "We will create a vSphere cluster that's dedicated to Oracle. We will comply with their licensing by doing it that way," he says.
But user experiences vary widely. "Bill," a CIO at another Fortune 500 company who also asked not to be named, says he has never been asked to pay to license every core in a server used in a virtual environment when not all are running Oracle, nor has he been asked to buy Oracle licenses for every server in a vSphere or any other virtual server cluster.
The key is in the negotiations and the customer's relationship with Oracle. "It's battle. You have to work hard," he says, but he's always been able to negotiate a satisfactory contract. Then again, he admits, he's never been through an audit.
Being asked to limit vSphere clusters just to Oracle isn't always efficient, especially for large data centers, and it's not necessary, professional negotiators say. "There is nothing in the contracts that say you have to pay for all the servers in a cluster," Guarente says. Although the software vendor has published separate policy documents that attempt to address this issue, Welch says there's nothing in black and white to back up the assertion that every server in a vSphere cluster must be licensed if Oracle software is installed or running on any one of those servers.
"The biggest issue for me is that these documents are not part of the contract the customer signed," says Guarente. "Oracle has very interesting policies around virtualization, and none of those policies are located in the contracts."
The problem with soft partitioning
One of the most relevant policy documents on the matter, the Oracle Partitioning Policy (PDF), distinguishes between hard partitioning and soft partitioning technologies with respect to licensing. With hard partitioning, customers can limit the number of processor licenses needed to those running Oracle. With soft partitioning, they cannot.
Oracle defines Microsoft Hyper-V and VMware as soft partitioning, although many technologists will argue that the features and methods used in VMware are no different than those that appear in Oracle's list of hard partitioning technologies, Guarente says. "Given that this is a competitor's offering, Oracle will always consider this as a form of soft partitioning," he explains.
The one exception, he adds, is under Microsoft's Windows Azure cloud offering, which uses Hyper-V. In that case, Oracle can be deployed within a virtual machine and the customer pays only for the resources used. "The reliance is on the provider to control the size of the virtual machines," he says.
Even if the customer's software license refers to the Partitioning Policy, that document includes a note stating that the policy "is for informational purposes only," and "does not constitute a contract or a commitment to any specific terms."
So does the policy apply or not?
Guarente says when he challenged one contract based on published policy material for an organization Oracle was auditing, he received that very same response from License Management Services, Oracle's auditing group. "They came back to us and said the documents were for educational purposes only and not binding. Fascinating." That business was able to successfully defend its position with Oracle, he adds.
"It's the art of working with Oracle versus the science of compliance. It's all very muddy," Guarente says. Furthermore, the list of approved hard partitioning technologies typically does not appear within the list of documents referenced in the Oracle Software Licensing Agreement (OSLA) that customers sign, says David Blake, CEO at UpperEdge, an IT sourcing consultancy.
Nonetheless, if one follows the Oracle Partitioning Policy, every processor and core within a single physical server must be licensed for Oracle in a VMware environment, even if the Oracle software is installed or running on just one core. House of Brick's Welch tells his clients to pay to license every processor and core in that situation. "We tell everyone that Oracle is right. That policy has been there since 2002," he says.
The subcluster debate
Running Oracle on a dedicated group of servers within a vSphere cluster, sometimes referred to as a subcluster, is a different matter, however. "Oracle likes to tell prospects and customers that if they are running Oracle on any physical server within a large vSphere cluster they have to license the entire cluster for Oracle. Nothing could be further from the truth," Welch says.
"The contract states that you must license any physical server on which you have installed or are running Oracle binaries. But you don't have to license other servers in that cluster," he insists. "Amazingly, many organizations don't know that."
Welch says he has negotiated on this point many times for enterprise clients. "When we have asked Oracle auditors if they could show us where it says in writing that every server in a vSphere cluster must be licensed for Oracle they said it was an unwritten policy."
Blake concurs. "Overall, the policy does not specifically call out the cluster scenario, so we advise clients to challenge Oracle" when a request is made to license every server in a vSphere cluster, he says.
In some cases, Welch adds, Oracle has asked some customers to license failover nodes in the cluster as well, but you're entitled to leave one node unlicensed, even when every physical server in a vSphere cluster is dedicated to Oracle, he says. Oracle's Software Investment Guide includes what's commonly known as the 10-day rule. "It says that you can run Oracle on an unlicensed physical server for up to 10 cumulative days per year as long as it's a named, designated node, and you fail back."
So why is this happening? Oracle is such a large organization that "people just make the assumption that there's no way that an Oracle account representative could be representing something to an organization that would not be contractual or binding," Welch says.