VMware Bug is Worse Than a Glitch To Users Who Depend on It

There is an unfortunate trend at VMware these days. The trend is in the release of their updates.

With VMware VI3 Update 1, VMware placed on their website an ISO image that had a bad installation. After a bit of tweaking, it was remastered with the same build number as the bad one. This was fixed by appending the character ‘a’ to the end of the build number. Should it not have been a new build number entirely?

This caused quite a bit of confusion.

When VMware VI3 Update 2 was release, VMware placed incorrect sizes for the ISO images on their website, apparently due to some automation issue. However, a worse problem awaited people on August 12th; A problem that would disable licensing and keep new VMs from being booted. But already running VMs worked just fine.

There is a patch available.

There is also a letter from Paul Maritz, explaining the issue.

The fact that these issues exists is problematic at best. But the lack of communication is what was really upsetting folks.

I communicated with my customers already based on the complexity of the Update 2 upgrade (you should also upgrade Virtual Center). Yet, VMware should have communicated in big bold letters that a problem existed long before they did. Yes they did start to communicate, I got several emails to that affect, but it was already very late in the day when they were sent.

The post on the forums about this is one of the largest threads there is for its short time period.

However, there is an even bigger problem. That is not all customers are properly testing their own installations of VMware ESX before they place them into production.

No this is not to say that VMware is not at fault, but it is to say that not everything that comes from a vendor is gospel and you should test with this in mind.

One of the tests was faulty for this; specifically a test with the time changing to sometime in the future or past. This is a security test that should be made to make sure authentication, authorization, and other aspects of security do not break.

It is not impossible that someone will have set the clock incorrectly and that can affect some tools adversely and gain access to which you do not have the rights. Most licensing schemes for example build into it time based locks, one way around this is to reset the system clock.

All patches, updates, and software should first be tested before the implementation of said software within production. All security, usage, and functionality tests should also be made before placing software in production.

It is an unfortunate trend that we have been trained by Microsoft to apply patches without much testing. We accept those patches will protect us, we do not do our own due diligence to prove this is correct or not, else we are susceptible to zero day attacks. We then have to patch the patch when Microsoft sends us broken code.

This trend is being applied to all systems and software in use within production. Yes some larger organizations have dedicated testing facilities and hopefully run the appropriate tests, but smaller organizations do not. They should and I imagine everyone admits that.

Yes VMware’s trend is upsetting, but we as the users of any software need to also improve our own testing and not take any vendor’s word for being safe, secure or even functional.

VMware has now forced us to add one more test to our testing procedures that should probably have been there from the start. But why have they not tested with this procedure already themselves? I am sure they will now.

Bottom line: buck the trend, test before you implement in production but make sure your own tests are adequate.

Virtualization expert Edward L. Haletky is the author of “VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers,” Pearson Education (2008.) He recently left Hewlett-Packard, where he worked in the Virtualization, Linux, and High-Performance Technical Computing teams. Haletky owns AstroArch Consulting, providing virtualization, security, and network consulting and development. Haletky is also a champion and moderator for the VMware discussion forums, providing answers to security and configuration questions.